Textile Electronics for VR/AR Applications

Textile electronics addresses fibers or fiber assemblies with electronic functions to generate, transmit, modulate, and detect electrons. Interactive textile electronic devices may provide suitable platforms for virtual reality (VR)/augmented reality (AR) applications because of their excellent performance and unique immersive features such as lightweight, handiness, flexibility, comfort, and low strain even under high deformations. This paper presents a systematic review of the literature on the state-of-the-art of interactive devices, fabrication technologies, system integration, promising applications, and challenges involved in textile-based VR/AR systems.     

Recent Advances in Wearable Sensors and Integrated Functional Devices for Virtual and Augmented Reality Applications

The advancement in virtual reality/augmented reality (VR/AR) has been achieved by breakthroughs in the realistic perception of virtual elements. Although VR/AR technology is advancing fast, enhanced sensor functions, long-term wearability, and seamless integration with other electronic components are still required for more natural interactions with the virtual world. Here, this report reviews the recent advances in multifunctional wearable sensors and integrated functional devices for VR/AR applications. Specified device designs, packaging strategies, and interactive physiological sensors are summarized based on their methodological approaches for sensory inputs and virtual feedback. In addition, limitations of the existing systems, key challenges, and future directions are discussed. It is envisioned that this progress report's outcomes will expand the insights on wearable functional sensors and device interfaces toward next-generation VR/AR technologies.     

Wearable Soft Technologies for Haptic Sensing and Feedback

Virtual reality (VR) and augmented reality (AR) systems have garnered recent widespread attention due to increased accessibility, functionality, and affordability. These systems sense user inputs and typically provide haptic, audio, and visual feedback to blend interactive virtual environments with the real world for an enhanced or simulated reality experience. With applications ranging from immersive entertainment, to teleoperation, to physical therapy, further development of this technology has the potential for impact across multiple disciplines. However, VR/AR devices still face critical challenges that hinder integration into everyday life and additional applications; namely, the rigid and cumbersome form factor of current technology that is incompatible with the dynamic movements and pliable limbs of the human body. Recent advancements in the field of soft materials are uniquely suited to provide solutions to this challenge. Devices fabricated from flexible and elastic bio-compatible materials have significantly greater compatibility with the human body and could lead to a more natural VR/AR experience. This review reports state-of-the-art experimental studies in soft materials for wearable sensing and haptic feedback in VR/AR applications, explores emerging soft technologies for on-body devices, and identifies current challenges and future opportunities toward seamless integration of the virtual and physical world.     

Nanowire-Based Soft Wearable Human–Machine Interfaces for Future Virtual and Augmented Reality Applications

A virtual world has now become a reality as augmented reality (AR) and virtual reality (VR) technology become commercially available. Similar to how humans interact with the physical world, AR and VR systems rely on human–machine interface (HMI) sensors to interact with the virtual world. Currently, this is achieved via state of-the-art wearable visual and auditory tools that are rigid, bulky, and burdensome, thereby causing discomfort during practical application. To this end, a skin sensory interface has the potential to serve as the next-generation AR/VR technology because skin-like wearable sensors have advantages in that they can be ultrathin, ultra-soft, conformal, and imperceptible, which provides the ultimate comfort and immersive experience for users. In this progress report, nanowire-based soft wearable HMI sensors including acoustic, strain, pressure sensors, and physiological sensors are reviewed that may be adopted as skin sensory inputs in future AR/VR systems. Further, nanowire-based soft contact lenses, haptic force, and thermal and vibration actuators are covered as potential means of feedback for future AR/VR systems. Considering the possible effects of the virtual world on human health, skin-like wearable artery pulses, glucose, and lactate sensors are also described, which may enable imperceptible health monitoring during future AR/VR practices.     

Recent Advances and Opportunities of Active Materials for Haptic Technologies in Virtual and Augmented Reality

Virtual reality and augmented reality (VR/AR) are evolving. The market demands and innovation efforts call for a shift in the key VR/AR technologies and engaging people virtually. Tele-haptics with multimodal and bilateral interactions are emerging as the future of the VR/AR industry. By transmitting and receiving haptic sensations wirelessly, tele-haptics allow human-to-human interactions beyond the traditional VR/AR interactions. The core technologies for tele-haptics include multimodal tactile sensing and feedback based on highly advanced sensors and actuators. Recent developments of haptic innovations based on active materials show that active materials can significantly contribute to addressing the needs and challenges for the current and future VR/AR technologies. Thus, this paper intends to review the current status and opportunities of active material-based haptic technology with a focus on VR/AR applications. It first provides an overview of the current VR/AR applications of active materials for haptic sensing and actuation. It then highlights the state-of-the-art haptic interfaces that are relevant to the materials with an aim to provide perspectives on the role of active materials and their potential integration in haptic devices. This paper concludes with the perspective and outlook of immersive multimodal tele-haptic interaction technologies.     

虚拟现实技术在医学领域的研究现状与进展

虚拟现实(VR)与其所衍生的增强现实(AR)和混合现实(MR)能把携带三维信息的虚拟场景与真实世界相互叠加,极大地提升用户感官世界的直观性、精准性、实时性。该技术的推广与应用,将会给医学领域带来变革式发展。本文剖析VR/AR/MR的概念并简述其发展历程,分别对虚拟现实和增强现实在医学领域的应用进行阐述,并通过微软产品HoloLens的特点分析基于混合现实的解决方案在医学领域的优势。最后对VR/AR/MR目前在医学领域所存在的不足进行归纳,并对未来的发展趋势进行展望。     

Emerging Thermal Technology Enabled Augmented Reality

In the past decade, remarkable progress has been made in the domain of augmented reality/virtual reality (AR/VR). The need for realistic and immersive augmentation has propelled the development of haptics interfaces-enabled AR/VR. The haptics interfaces facilitate direct interaction and manipulation with both real and virtual objects, thus augmenting the perception and experiences of the users. The level of augmentation can be significantly improved by thermal stimulation or sensing, which facilitates a higher degree of object identification and discrimination. This review discusses the thermal technology-enabled augmented reality and summarizes the recent progress in the development of different thermal technology such as thermal haptics including thermo-resistive heater and Peltier devices, thermal sensors including resistive, pyroelectric, and thermoelectric sensors, which can be utilized to improve the realism of augmentation. The fundamental mechanism, design strategies, and the rational guidelines for the adoption of these technologies in AR/VR is explicitly discussed. The conclusion provides an outlook on the existing challenges and outlines the future roadmap for the realization of next-generation thermo-haptics enabled augmented reality.     

Elastomeric Haptic Devices for Virtual and Augmented Reality

Since the modern concepts for virtual and augmented reality are first introduced in the 1960's, the field has strived to develop technologies for immersive user experience in a fully or partially virtual environment. Despite the great progress in visual and auditory technologies, haptics has seen much slower technological advances. The challenge is because skin has densely packed mechanoreceptors distributed over a very large area with complex topography; devising an apparatus as targeted as an audio speaker or television for the localized sensory input of an ear canal or iris is more difficult. Furthermore, the soft and sensitive nature of the skin makes it difficult to apply solid state electronic solutions that can address large areas without causing discomfort. The maturing field of soft robotics offers potential solutions toward this challenge. In this article, the definition and history of virtual (VR) and augmented reality (AR) is first reviewed. Then an overview of haptic output and input technologies is presented, opportunities for soft robotics are identified, and mechanisms of intrinsically soft actuators and sensors are introduced. Finally, soft haptic output and input devices are reviewed with categorization by device forms, and examples of soft haptic devices in VR/AR environments are presented.     

Thermo-Haptic Materials and Devices for Wearable Virtual and Augmented Reality

The current generation of virtual and augmented realities (VR/AR) has substantially advanced in the past decade because of the rapid development of converging technologies in various engineering and scientific fields. However, the current VR/AR technologies rely mainly on visual and auditory senses to physically replicate the virtual environment, although tactile senses play a significant role in the daily life since a myriad of tactile information is received through physical touch. Of the tactile senses, thermal senses are of great importance to be reproduced in the VR/AR field, since heat is transferred constantly and further interact with the surrounding environment. To date, there has been a huge amount of research studies on functional materials, thermo-haptic devices, and wearable electronics that have all converged to form the fundamental groundwork for the development of wearable thermal VR/AR devices. In this progress report, a review on various physical mechanisms and research is provided that can potentially be applied in the next generation of thermal VR/AR technologies and discuss the essential challenges that need to be addressed.     

Advanced graphics behind medical virtual reality: evolution ofalgorithms, hardware, and software interfaces

Applications of virtual reality (VR) and augmented reality (AR) in medicine require real-time visualization and modeling of large three-dimensional data sets. Consequently, these applications require powerful computation, extensive high-bandwidth memory, and fast communication links. In the past, the manufacturers of medical imaging equipment produced their own special-purpose proprietary hardware for image processing and solid graphics. Due to the developments in computer hardware in general and in graphics accelerators in particular, there is a trend toward replacing the proprietary hardware off-the-shelf (OTS) equipment. Computer graphics itself has advanced in its quest for realism. Generic algorithms such as shading, texture mapping, and volume rendering have been developed to meet the resultant ever increasing requirements. Advances in both the OTS CPU and graphics hardware have enabled real-time implementations of these algorithms, thereby facilitating many of the medical VR/AR applications used today. The development of graphics libraries such as OpenGL has also been an important factor. These libraries provide an underlying portable software platform that optimizes the utilization of the available graphics hardware. OpenGL has become a standard graphics application programming interface, particularly for graphics-intensive applications, and more and more OTS systems provide hardware implementations of OpenGL commands. The review paper follows the evolution of these technologies and examines their crucial role in enabling the appearance of the current VR/AR applications in medicine and provides a look at current trends and future possibilities     

Recent Progress in Functional Dye-Doped Liquid Crystal Devices

As a typical representative of dopants, organic functional dyes have demonstrated their significant roles in novel smart liquid crystal (LC) devices, and dye-doped LCs have also been a source of inspiration for scientists to design and fabricate stimuli-gated materials or devices for envisioned applications in a wide range of areas. In this review, the focus on dichroic dyes, fluorescent dyes, and photothermal dyes, and the recent progress of the LC devices employing these dyes as dopants are overviewed. The review highlights the developments of the novel LC devices doped with these dyes. The structures, designs, and applications of these devices are outlined. The underlying principles of dichroic dyes, fluorescent dyes, and photothermal dyes which are utilized as functional dopants in LC devices are first introduced. Subsequently, the novel developments of functional dye-doped LC devices in the application fields of smart windows, attenuators for augmented reality (AR) systems, color-changeable textiles, dichroic color filters, dual-mode circular polarizers, chirality detectors, optical limiters, switchable luminescent solar concentrators, multiple information encryption, anti-counterfeiting, photo-addressed transparent displays, circularly polarized luminescence, tunable lasers, and light-driven soft actuators are discussed. Finally, the challenge and the strategies for the future improvement of dye-doped LC devices are also discussed.     

Immersive interactive technologies and virtual shopping experiences: Differences in consumer perceptions between augmented reality (AR) and virtual reality (VR)

Based on the concepts of the Reality-Virtuality (RV) continuum and the stimulus-organism-response (SOR) framework, this study investigated differences between AR and VR in their effects on vividness/interactivity, a sense of presence, users’ sensory brand app experience, attitude, and behavioral intention. For AR, participants downloaded an app to mobile phones. For VR, participants downloaded a VR app to smartphones attached to a Google Cardboard VR headset. Vividness and interactivity directly (or indirectly) impacted a sense of presence, sensory brand app experience, attitude towards technology, and behavioral intention. However, AR and VR differed in how vividness and interactivity influenced sensory brand app experience and attitude towards technology. Overall, a sense of presence was a significant mediator for 1) the relationship between vividness and sensory brand app experience and 2) the relationship between interactivity and attitude towards technology for VR but no such relationship was observed with AR. The relationships among sensory brand app experience, attitude, and behavioral intention were significant in both AR and VR settings. Employing real AR/VR shopping situations, the study provides for practitioners deep understandings of consumers’ actual perceptions of applications of AR/VR and offers practical insights as to the efficiency of AR and VR in improving consumer virtual shopping experiences.     

Virtual surgery in a (tele-)radiology framework

Presents telemedicine as an extension of a teleradiology framework through tools for virtual surgery. To classify the described methods and applications, the research field of virtual reality (VR) is broadly reviewed. Differences with respect to technical equipment, methodological requirements and areas of application are pointed out. VR, desktop VR and augmented reality are differentiated and discussed in some typical contexts of diagnostic support, surgical planning, therapeutic procedures, simulation and training. Visualization techniques are compared as a prerequisite for VR and assigned to distinct levels of immersion. The advantage of a hybrid visualization kernel is emphasized with respect to the desktop VR applications that are subsequently shown. Moreover, software design aspects are considered by outlining functional openness in the architecture of the host system. A teleradiology workstation was extended by dedicated tools for surgical planning through a plug-in mechanism. Examples of recent areas of application are introduced, such as liver tumor resection planning, diagnostic support in heart surgery, and craniofacial surgery planning. In the future, surgical planning systems will become more important. They will benefit from improvements in image acquisition and communication, new image processing approaches and techniques for data presentation. This will facilitate pre-operative planning and intra-operative applications     

Flexible Hybrid Sensor Systems with Feedback Functions

Skin-like wearable sensors are regarded as key technologies toward home-based healthcare, human–machine interfaces, robotics, prostheses, and enhanced augmented/virtual reality (AR/VR). Inspired by human somatosensory functions, artificial sensory feedback systems play vital roles in shaping interactions with complex environments and timely decision-making. This study presents an overview of recent advances in feedback-driven, closed-loop skin-inspired flexible sensor systems that make use of emerging functional nanomaterials and elaborate structures. Drawing on feedback solutions, four categories of sensor systems are highlighted, which include prosthesis- and AR/VR-based human–machine interfaces, smartphone-based approaches for point-of-care detection, and smart wearable displays for direct signal visualizations. Furthermore, the progress of machine learning on the reliable recognition of massive quantities of signals generated by flexible sensor networks is briefly discussed. The state-of-the-art hybrid sensor techniques, along with other emerging strategies, will enable total sensory feedback loop systems to be developed for next-generation electronic skins.     

Stretchable Skin‐Like Cooling/Heating Device for Reconstruction of Artificial Thermal Sensation in Virtual Reality

Along with visual and tactile sensations, thermal sensation by temperature feeling on the skin can provide rich physical information on the environment and objects. With a simple touch of objects, relative temperature can be sensed and even objects can be differentiated with different thermal properties without any visual cue. Thus, artificially reproducing accurate/controllable thermal sensation haptic signals on human epidermis will certainly be a major research area to reconstruct a more realistic virtual reality (VR) environment. In this study, for the first time, a skin‐like, highly soft and stretchable and bi‐functional (both cold and hot sensation) thermo‐haptic device is reported for wearable VR applications with a single device structure (not separate heater and cooler). The skin‐like thermo‐haptic (STH) device can actively cool down and heat up deformable skin surfaces with instantaneous and accurate adjustment of temperature based upon a feedback control algorithm to mimic desirable thermal sensation with 230% stretchability. As a proof‐of‐concept, the STH device is integrated with a finger‐motion tracking glove to provide artificial thermal sensation information to the skin in various situations such as touching cold beer bottles and hot coffee cups in virtual space. This new type of STH device can offer potential implications for next‐generation haptic devices to provide unique thermal information for a more realistic virtual‐world field and medical thermal treatment.     

Flexible energy storage devices for wearable bioelectronics

With the growing market of wearable devices for smart sensing and personalized healthcare applications,energy stor-age devices that ensure stable power supply and can be constructed in flexible platforms have attracted tremendous research in-terests.A variety of active materials and fabrication strategies of flexible energy storage devices have been intensively studied in recent years,especially for integrated self-powered systems and biosensing.A series of materials and applications for flex-ible energy storage devices have been studied in recent years.In this review,the commonly adopted fabrication methods of flex-ible energy storage devices are introduced.Besides,recent advances in integrating these energy devices into flexible self-powered systems are presented.Furthermore,the applications of flexible energy storage devices for biosensing are summar-ized.Finally,the prospects and challenges of the self-powered sensing system for wearable electronics are discussed.     

Multimodal 5-DOF Stretchable Electromagnetic Actuators toward Haptic Information Delivery

The rapid advancements in artificial intelligence, particularly in the domains of robotics, prosthetics, and virtual and augmented reality (VR/AR), have driven an escalating demand for intuitive and effective human–machine interactions. Consequently, haptic devices, being electronic displays for the sense of touch, have drawn increasing attention. More efforts are in demand to develop stretchable and lightweight haptic devices that can trigger multiple mechanical cutaneous receptors using a single device. This work presents a new 3-modal 5-DOF stretchable haptic interface that is enabled by electromagnetic actuators and high-fidelity multi-layer metal printing. The haptic device renders rich haptic sensations (i.e., normal force, vibration, angular force, skin dragging) in one device, allowing for the comprehensive delivery of tactile information through the excitation of multiple cutaneous receptors. Additionally, haptic devices are designed to be compact, lightweight, and skin-compatible. The skin-like softness and stretchability enable intimate skin contact, which is crucial for efficient haptic information delivery. This feature prevents the impediment to natural movements of the skin and ensures the functional integrity of the device during daily deformations of the skin. Finally, three proof-of-concept demonstrations illustrate the potential of the reported multimodal haptic devices for advanced haptic interactions across various domains.     

Three-dimensional input device with six degrees of freedom

Commodity priced PCs now come with powerful three-dimensional (3D) graphic capabilities that allow new applications to incorporate 3D interactivity. But, there are few low cost 3D input devices available for the desktop non-immersive virtual reality (VR) applications. Interactive systems that use a 3D world should ideally be complimented with a 3D input device. Such devices have previously been built mainly for immersive virtual reality systems and are not well suited for desktop systems. They are also expensive. Our device is intended to provide a low cost, intuitive full 3D input with six degrees of freedom. As well, it was to be used with the non-dominant hand, freeing the other hand for finer-motor skills to use a conventional mouse or the keyboard. We have built a prototype device that was inspired by the mechanics of the gimbal mount for a gyroscope. We used a field programmable gate array (FPGA) to provide rapid prototyping of the electronics from the sensors to the computer. Initial evaluation shows promise but also some problems together with suggestions for improvements.     

Cutting Edge Use of Conductive Patterns in Nanocellulose-Based Green Electronics

Green electronics made from degradable materials have recently attracted special attention, because electronic waste (e-waste) represents a serious threat to the environment and to human health worldwide. Among the novel materials used for sustainable technologies, nanocelluloses containing at least 1D in the nanoscale range (1–100 nm) have been widely exploited for various industrial applications owing to their inherent properties, such as biodegradability, mechanical strength, thermal stability, and optical transparency. This review highlights recent advances in research on the development of patterns for conductive material on nanocellulose substrates for use in high-performance green electronics. The advantages of nanocellulose substrates compared to conventional paper substrates for advanced green electronics are discussed. Importantly, this review emphasizes various fabrication strategies for producing conductive patterns on different types of nanocellulose-based substrates, such as cellulose nanofiber (CNF), (2,2,6,6-tetramethylpiperidin-1-yl)oxyl(TEMPO)-oxidized CNF, regenerated cellulose, and bacterial cellulose. In the latter part of this review, emerging engineering applications for green electronics such as circuits, transistors/antennas, sensors, energy storage systems, and electrochromic devices are further discussed.     

A Metadata Design for Augmented Broadcasting and Testbed System Implementation

Augmented reality (AR) is a popular service in mobile devices, and many AR applications can be downloaded from app stores. As TV broadcasting has continued to integrate with the Internet, it has become an area in which the AR concept is able to reside, although in a simple form, such as an advertisement placed in the static region of a scene. There are some restrictions against TV broadcasting realizing AR since TVs are fixed devices and typically do not have GPS, geomagnetic, or acceleration sensors, which are standard equipment in mobile devices. However, the desire to experience AR on a large TV screen has triggered research and development for an ideal AR business model and service type. This paper introduces several use cases for augmented broadcasting services and also presents an augmented broadcasting metadata scheme designed for a broadcasting environment. We also verify some of the use cases and an augmented broadcasting metadata scheme in an implemented augmented broadcasting system based on a hybrid TV platform.