Nanowire-Based Soft Wearable Human–Machine Interfaces for Future Virtual and Augmented Reality Applications

A virtual world has now become a reality as augmented reality (AR) and virtual reality (VR) technology become commercially available. Similar to how humans interact with the physical world, AR and VR systems rely on human–machine interface (HMI) sensors to interact with the virtual world. Currently, this is achieved via state of-the-art wearable visual and auditory tools that are rigid, bulky, and burdensome, thereby causing discomfort during practical application. To this end, a skin sensory interface has the potential to serve as the next-generation AR/VR technology because skin-like wearable sensors have advantages in that they can be ultrathin, ultra-soft, conformal, and imperceptible, which provides the ultimate comfort and immersive experience for users. In this progress report, nanowire-based soft wearable HMI sensors including acoustic, strain, pressure sensors, and physiological sensors are reviewed that may be adopted as skin sensory inputs in future AR/VR systems. Further, nanowire-based soft contact lenses, haptic force, and thermal and vibration actuators are covered as potential means of feedback for future AR/VR systems. Considering the possible effects of the virtual world on human health, skin-like wearable artery pulses, glucose, and lactate sensors are also described, which may enable imperceptible health monitoring during future AR/VR practices.     

Elastomeric Haptic Devices for Virtual and Augmented Reality

Since the modern concepts for virtual and augmented reality are first introduced in the 1960's, the field has strived to develop technologies for immersive user experience in a fully or partially virtual environment. Despite the great progress in visual and auditory technologies, haptics has seen much slower technological advances. The challenge is because skin has densely packed mechanoreceptors distributed over a very large area with complex topography; devising an apparatus as targeted as an audio speaker or television for the localized sensory input of an ear canal or iris is more difficult. Furthermore, the soft and sensitive nature of the skin makes it difficult to apply solid state electronic solutions that can address large areas without causing discomfort. The maturing field of soft robotics offers potential solutions toward this challenge. In this article, the definition and history of virtual (VR) and augmented reality (AR) is first reviewed. Then an overview of haptic output and input technologies is presented, opportunities for soft robotics are identified, and mechanisms of intrinsically soft actuators and sensors are introduced. Finally, soft haptic output and input devices are reviewed with categorization by device forms, and examples of soft haptic devices in VR/AR environments are presented.     

Recent Advances in Wearable Sensors and Integrated Functional Devices for Virtual and Augmented Reality Applications

The advancement in virtual reality/augmented reality (VR/AR) has been achieved by breakthroughs in the realistic perception of virtual elements. Although VR/AR technology is advancing fast, enhanced sensor functions, long-term wearability, and seamless integration with other electronic components are still required for more natural interactions with the virtual world. Here, this report reviews the recent advances in multifunctional wearable sensors and integrated functional devices for VR/AR applications. Specified device designs, packaging strategies, and interactive physiological sensors are summarized based on their methodological approaches for sensory inputs and virtual feedback. In addition, limitations of the existing systems, key challenges, and future directions are discussed. It is envisioned that this progress report's outcomes will expand the insights on wearable functional sensors and device interfaces toward next-generation VR/AR technologies.     

Thermo-Haptic Materials and Devices for Wearable Virtual and Augmented Reality

The current generation of virtual and augmented realities (VR/AR) has substantially advanced in the past decade because of the rapid development of converging technologies in various engineering and scientific fields. However, the current VR/AR technologies rely mainly on visual and auditory senses to physically replicate the virtual environment, although tactile senses play a significant role in the daily life since a myriad of tactile information is received through physical touch. Of the tactile senses, thermal senses are of great importance to be reproduced in the VR/AR field, since heat is transferred constantly and further interact with the surrounding environment. To date, there has been a huge amount of research studies on functional materials, thermo-haptic devices, and wearable electronics that have all converged to form the fundamental groundwork for the development of wearable thermal VR/AR devices. In this progress report, a review on various physical mechanisms and research is provided that can potentially be applied in the next generation of thermal VR/AR technologies and discuss the essential challenges that need to be addressed.     

虚拟现实技术在医学领域的研究现状与进展

虚拟现实(VR)与其所衍生的增强现实(AR)和混合现实(MR)能把携带三维信息的虚拟场景与真实世界相互叠加,极大地提升用户感官世界的直观性、精准性、实时性。该技术的推广与应用,将会给医学领域带来变革式发展。本文剖析VR/AR/MR的概念并简述其发展历程,分别对虚拟现实和增强现实在医学领域的应用进行阐述,并通过微软产品HoloLens的特点分析基于混合现实的解决方案在医学领域的优势。最后对VR/AR/MR目前在医学领域所存在的不足进行归纳,并对未来的发展趋势进行展望。     

Skin Electronics: Next-Generation Device Platform for Virtual and Augmented Reality

Virtual reality (VR) and augmented reality (AR) are overcoming the physical limits of real-life using advances in devices and software. In particular, the recent restrictions in transportation from the coronavirus disease 2019 (COVID-19) pandemic are making people more interested in these virtual experiences. However, to minimize the differences between artificial and natural perception, more human-interactive and human-like devices are necessary. The skin is the largest organ of the human body and interacts with the environment as the site of interfacing and sensing. Recent progress in skin electronics has enabled the use of the skin as the mounting object of functional devices and the signal pathway bridging humans and computers, with opening its potential in future VR and AR applications. In this review, the current skin electronics are summarized as one of the most promising device solutions for future VR/AR devices, especially focusing on the recent materials and structures. After defining and explaining VR/AR systems and the components, the advantages of skin electronics for VR/AR applications are emphasized. Next, the detailed functionalities of skin electronic devices, including the input, output, energy devices, and integrated systems, are reviewed for future VR/AR applications.     

Wearable Soft Technologies for Haptic Sensing and Feedback

Virtual reality (VR) and augmented reality (AR) systems have garnered recent widespread attention due to increased accessibility, functionality, and affordability. These systems sense user inputs and typically provide haptic, audio, and visual feedback to blend interactive virtual environments with the real world for an enhanced or simulated reality experience. With applications ranging from immersive entertainment, to teleoperation, to physical therapy, further development of this technology has the potential for impact across multiple disciplines. However, VR/AR devices still face critical challenges that hinder integration into everyday life and additional applications; namely, the rigid and cumbersome form factor of current technology that is incompatible with the dynamic movements and pliable limbs of the human body. Recent advancements in the field of soft materials are uniquely suited to provide solutions to this challenge. Devices fabricated from flexible and elastic bio-compatible materials have significantly greater compatibility with the human body and could lead to a more natural VR/AR experience. This review reports state-of-the-art experimental studies in soft materials for wearable sensing and haptic feedback in VR/AR applications, explores emerging soft technologies for on-body devices, and identifies current challenges and future opportunities toward seamless integration of the virtual and physical world.     

Textile Electronics for VR/AR Applications

Textile electronics addresses fibers or fiber assemblies with electronic functions to generate, transmit, modulate, and detect electrons. Interactive textile electronic devices may provide suitable platforms for virtual reality (VR)/augmented reality (AR) applications because of their excellent performance and unique immersive features such as lightweight, handiness, flexibility, comfort, and low strain even under high deformations. This paper presents a systematic review of the literature on the state-of-the-art of interactive devices, fabrication technologies, system integration, promising applications, and challenges involved in textile-based VR/AR systems.     

Flexible Hybrid Sensor Systems with Feedback Functions

Skin-like wearable sensors are regarded as key technologies toward home-based healthcare, human–machine interfaces, robotics, prostheses, and enhanced augmented/virtual reality (AR/VR). Inspired by human somatosensory functions, artificial sensory feedback systems play vital roles in shaping interactions with complex environments and timely decision-making. This study presents an overview of recent advances in feedback-driven, closed-loop skin-inspired flexible sensor systems that make use of emerging functional nanomaterials and elaborate structures. Drawing on feedback solutions, four categories of sensor systems are highlighted, which include prosthesis- and AR/VR-based human–machine interfaces, smartphone-based approaches for point-of-care detection, and smart wearable displays for direct signal visualizations. Furthermore, the progress of machine learning on the reliable recognition of massive quantities of signals generated by flexible sensor networks is briefly discussed. The state-of-the-art hybrid sensor techniques, along with other emerging strategies, will enable total sensory feedback loop systems to be developed for next-generation electronic skins.     

Recent Advances and Opportunities of Active Materials for Haptic Technologies in Virtual and Augmented Reality

Virtual reality and augmented reality (VR/AR) are evolving. The market demands and innovation efforts call for a shift in the key VR/AR technologies and engaging people virtually. Tele-haptics with multimodal and bilateral interactions are emerging as the future of the VR/AR industry. By transmitting and receiving haptic sensations wirelessly, tele-haptics allow human-to-human interactions beyond the traditional VR/AR interactions. The core technologies for tele-haptics include multimodal tactile sensing and feedback based on highly advanced sensors and actuators. Recent developments of haptic innovations based on active materials show that active materials can significantly contribute to addressing the needs and challenges for the current and future VR/AR technologies. Thus, this paper intends to review the current status and opportunities of active material-based haptic technology with a focus on VR/AR applications. It first provides an overview of the current VR/AR applications of active materials for haptic sensing and actuation. It then highlights the state-of-the-art haptic interfaces that are relevant to the materials with an aim to provide perspectives on the role of active materials and their potential integration in haptic devices. This paper concludes with the perspective and outlook of immersive multimodal tele-haptic interaction technologies.     

Untethered Feel-Through Haptics Using 18-µm Thick Dielectric Elastomer Actuators

Head-mounted displays for virtual reality (VR) and augmented reality (AR) allow users to see highly realistic virtual worlds. The wearable haptics that enable feeling and touching these virtual objects are typically bulky, tethered, and provide only low fidelity feedback. A particularly challenging type of wearable human-machine interface is feel-through haptics: ultra-thin wearables so soft as to be mechanically imperceptible when turned off, yet generating sufficient force when actuated to make virtual objects feel tangible, or to change the perceived texture of a physical object. Here, 18 µm thick soft dielectric elastomer actuators (DEA), directly applied on the skin, reports rich vibrotactile feedback generation from 1 Hz to 500 Hz. Users correctly identifies different frequency and sequence patterns with success rates from 73 to 97% for devices applied on their fingertips. An untethered version weighing only 1.3 grams allowed blindfolded users to correctly identify letters by “seeing” them through their fingers. The silicone-based DEA membrane is mechanically transparent, enabling wearable haptics for the many applications where hand dexterity is critical. The feel-through DEA can be placed in array format anywhere on the body.     

A Metadata Design for Augmented Broadcasting and Testbed System Implementation

Augmented reality (AR) is a popular service in mobile devices, and many AR applications can be downloaded from app stores. As TV broadcasting has continued to integrate with the Internet, it has become an area in which the AR concept is able to reside, although in a simple form, such as an advertisement placed in the static region of a scene. There are some restrictions against TV broadcasting realizing AR since TVs are fixed devices and typically do not have GPS, geomagnetic, or acceleration sensors, which are standard equipment in mobile devices. However, the desire to experience AR on a large TV screen has triggered research and development for an ideal AR business model and service type. This paper introduces several use cases for augmented broadcasting services and also presents an augmented broadcasting metadata scheme designed for a broadcasting environment. We also verify some of the use cases and an augmented broadcasting metadata scheme in an implemented augmented broadcasting system based on a hybrid TV platform.     

基于增强现实的硬件维护系统研究与设计

传感器、多媒体、机器学习等技术的快速发展,促进了增强现实在多领域得到普及和应用。航空设备硬件运行维护也是一个重要领域,可以基于增强现实构建一个硬件维护系统,渲染一个虚拟的硬件运行环境,模拟实际的故障发生情况,加强对硬件设备的装配和维修,进一步提高航空设备硬件运行维护的水平。本文详细地描述了增强现实在硬件维护系统中的应用功能,分别是虚拟环境仿真功能、装配维护功能、3D模型设计功能,同时构建一个强大的硬件维护系统,该系统沉浸感强、娱乐性足,能够激发硬件维护人员的学习和应用兴趣,具有重要的作用和意义。     

虚拟现实全景图像显著性检测研究进展综述

随着虚拟现实处理技术的发展,虚拟现实全景图像的显著性检测成为近年来学术界和工业界关注的研究热点.本文分析虚拟现实全景图像的特性,综述虚拟现实全景图像显著性检测算法的研究进展.将已有的虚拟现实全景图像显著性检测算法进行分类、分析以及对比,本文总结了当前虚拟现实全景图像显著性检测面临的挑战,并对其发展趋势进行展望.     

VR技术助力虚拟实验教学平台发展的策略研究

虚拟现实已成为当今时代的热点话题,并渗透很多行业、领域,无形中给人们带来了较多方便与益处。文章主要研究虚拟现实技术在货币知识教学中的应用,通过VR技术开展中国货币发展史的虚拟教学研究,也让VR技术助力于虚拟实验教学平台的发展。     

Novel Hybrid Content Synchronization Scheme for Augmented Broadcasting Services

As a new hybrid broadcasting service, augmented broadcasting shows enhanced broadcasting content on a large TV screen, while augmented reality (AR) on a mobile device augments additional graphical content onto an input image from the device's own camera to provide useful and convenient information for users. A one‐sided broadcasting service using AR has already been attempted in virtual advertisements during sport broadcasts. However, because its augmentation is preprocessed before the video image is transmitted, the viewer at home may have no influence on this formation; and no interaction for the user is possible unless the viewer has a direct connection to the content provider. Augmented broadcasting technology enables viewers to watch mixed broadcasting content only when they want such service and to watch original broadcasting content when they do not. To realize an augmented broadcasting service, the most important issue is to resolve the hybrid content synchronization over heterogeneous broadcast and broadband networks. This paper proposes a novel hybrid content synchronization scheme for an augmented broadcasting service and presents its implementation and results in a terrestrial DTV environment.     

Organic Haptics: Intersection of Materials Chemistry and Tactile Perception

The goal of the field of haptics is to create technologies that manipulate the sense of touch. In virtual and augmented reality, haptic devices are for touch what loudspeakers and RGB displays are for hearing and vision. Haptic systems that utilize micromotors or other miniaturized mechanical devices (e.g., for vibration and pneumatic actuation) produce interesting effects, but are quite far from reproducing the feeling of real materials. They are especially deficient in recapitulating surface properties: fine texture, friction, viscoelasticity, tack, and softness. The central argument of this progress report is that in order to reproduce the feel of everyday objects, molecular control must be established over the properties of materials; ultimately, such control will enable the design of materials which can change these properties in real time. Stimuli‐responsive organic materials, such as polymers and composites, are a class of materials which can change their oxidation state, conductivity, shape, and rheological properties, and thus might be useful in future haptic technologies. Moreover, the use of such materials in research on tactile perception could help elucidate the limits of human tactile sensitivity. The work described represents the beginnings of this new area of inquiry, in which the defining approach is the marriage of materials science and psychology.     

Advanced graphics behind medical virtual reality: evolution ofalgorithms, hardware, and software interfaces

Applications of virtual reality (VR) and augmented reality (AR) in medicine require real-time visualization and modeling of large three-dimensional data sets. Consequently, these applications require powerful computation, extensive high-bandwidth memory, and fast communication links. In the past, the manufacturers of medical imaging equipment produced their own special-purpose proprietary hardware for image processing and solid graphics. Due to the developments in computer hardware in general and in graphics accelerators in particular, there is a trend toward replacing the proprietary hardware off-the-shelf (OTS) equipment. Computer graphics itself has advanced in its quest for realism. Generic algorithms such as shading, texture mapping, and volume rendering have been developed to meet the resultant ever increasing requirements. Advances in both the OTS CPU and graphics hardware have enabled real-time implementations of these algorithms, thereby facilitating many of the medical VR/AR applications used today. The development of graphics libraries such as OpenGL has also been an important factor. These libraries provide an underlying portable software platform that optimizes the utilization of the available graphics hardware. OpenGL has become a standard graphics application programming interface, particularly for graphics-intensive applications, and more and more OTS systems provide hardware implementations of OpenGL commands. The review paper follows the evolution of these technologies and examines their crucial role in enabling the appearance of the current VR/AR applications in medicine and provides a look at current trends and future possibilities     

Flex Ethernet Technology and Application in 5G Mobile Transport Network

With the advent of 5G era,the rise of cloud services,virtual reality/virtual reality(AR/VR),vehicle networking and other technologies has put forward new requirements for the bandwidth and delay of the bearer network.Traditional Ethernet technology cannot meet the new requirements very well.Flex Ethernet(FlexE)technology has emerged as the times require.This paper introduces the background,standardization process,functional principle,application mode and technical advantages of FlexE technology,and finally analyses its application prospects and shortcomings in 5G mobile transport network.     

The Potential of Haptics Technologies

In spite of the significant recent progress, the incorporation of haptics into virtual environments is still in its infancy due to limitations in the hardware, the cost of development, as well as the level of reality they provide. Nonetheless, we believe that the field will one day be one of the groundbreaking media of the future. It has its current holdups but the promise of the future is worth the wait. The technology is becoming cheaper and applications are becoming more forthcoming and apparent. If we can survive this infancy, it will promise to be an amazing revolution in the way we interact with computers and the virtual world. The researchers organize the rapidly increasing multidisciplinary research of haptics into four subareas: human haptics, machine haptics, computer haptics, and multimedia haptics