Emerging Thermal Technology Enabled Augmented Reality

In the past decade, remarkable progress has been made in the domain of augmented reality/virtual reality (AR/VR). The need for realistic and immersive augmentation has propelled the development of haptics interfaces-enabled AR/VR. The haptics interfaces facilitate direct interaction and manipulation with both real and virtual objects, thus augmenting the perception and experiences of the users. The level of augmentation can be significantly improved by thermal stimulation or sensing, which facilitates a higher degree of object identification and discrimination. This review discusses the thermal technology-enabled augmented reality and summarizes the recent progress in the development of different thermal technology such as thermal haptics including thermo-resistive heater and Peltier devices, thermal sensors including resistive, pyroelectric, and thermoelectric sensors, which can be utilized to improve the realism of augmentation. The fundamental mechanism, design strategies, and the rational guidelines for the adoption of these technologies in AR/VR is explicitly discussed. The conclusion provides an outlook on the existing challenges and outlines the future roadmap for the realization of next-generation thermo-haptics enabled augmented reality.     

Integrated reproduction of human motion components by motion copying system

Currently, the development of leading‐edge technology for recording and loading human motion on the basis of haptic information is required in the fields of manufacturing and human support. Human movement is an assembly of motion components. Since human movements should be supported by a robot in real time, it is necessary to integrate the motion components that were saved earlier. Once such motion integration is realized, future technology for use in daily human life can be developed. This paper proposes the integrated reproduction of the decomposed components of human motion by using a motion copying system. This system is the key technology for the realization of the acquisition, saving, and reproduction of real‐world haptic information. By using the proposed method, it is possible not only to achieve expert skill acquisition, skill transfer to robots, and power assist for each motion component, but also to open up new areas of applications. © 2012 Wiley Periodicals, Inc. Electr Eng Jpn, 181(1): 28–35, 2012; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/eej.21263     

Translating visual art into tactile art to produce equivalent aesthetic experiences.

An argument is put forward to show that visual art may actually be translated into tactile objects that produce equivalent aesthetic experiences for blind or visually impaired people. This is shown by delineating the information required for such a translation in general, demonstrating that such a translation is already achievable from color vision to color vision and theoretically possible from spatial vision to spatial touch, and outlining that an analogous procedure could be applied to aesthetic experience. Limitations of such translations of visual works of art into tactile objects, and implications of such a procedure for artistic experience and education, are also mentioned. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Design of a slave arm of a surgical robot system to estimate the contact force at the tip of the employed instruments

Acquisition of the contact force at the instrument tip can enable better performance, e.g. transparency of the haptic feedback in the surgical robot systems. It is, however, difficult to measure the contact force directly due to technical limitations in attaching sensors to the tip of the instruments. This paper proposes a method to estimate the forces by installing the sensors away from the instrument tip. The proposed method employs specially designed mechanical parts of the slave robot, i.e. a slider cover plate for the z-axis translational force along the insertion direction, and docking clamps for the rotational pivot torques around the fulcrum point. Strain gauges are attached to specially designed places with enhancing shapes. The simulation results of the force estimation are presented to confirm the strain concentration area. The proposed method is validated with quantitative experimental results. Calibrated weights are determined upon the comparison of the strain value with a calibrated 6-axis force/torque sensor. The percentage error in the force calibration is about 5~8% calculated by the root mean square error (RMSE) of force-sensing performance. In addition, it can be computed by considering only the bending phase of each sensor although the hysteresis is observed from the calibration graph.     

The feasibility of a mixed reality surgical training environment

The Sheffield knee arthroscopy training system (SKATS) was originally a visual-based virtual environment without haptic feedback, but has been further developed as a mixed reality-training environment through the use of tactile augmentation (or passive haptics). The design of the new system is outlined and then tested. In the first experiment described, the effect of tactile augmentation on performance is considered by comparing novice performance using the original and mixed reality system. In the second experiment the mixed reality system is assessed in terms of construct validity by comparing the performance of users with differing levels of surgical expertise. The results are discussed in terms of the validity of a mixed reality environment for training knee arthroscopy.     

Real haptics and its applications

Real haptics has been an unsolved problem since its beginning in the 1940s. After much R&D, including the early works in the United States and other countries, bilateral control designed in the acceleration space is a solution for this issue. The quality of touch and tactile sensation is heavily dependent on the transparency between the master side and the slave side. Robust motion control, which is equivalent to acceleration control, gives high transparency. Real haptics is now a key concept in 21st century where the quality of life should be the basis of our society. Various examples are also given. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

触摸屏触感振动反馈设计研究

作为一种全新的人机交互技术,触摸屏已广泛应用于工业、医疗、通信、娱乐、教育等领域.触觉感受作为提升用户体验的有效途径,也逐渐引起了研究者的关注.文中对触摸屏触感振动反馈系统设计进行了研究,建立了触摸屏振动系统的力学模型,对触摸屏进行了振动模态仿真分析,对模型的正确性进行了实验验证,以指导触摸屏系统触感反馈功能的设计.     

Development of a whole‐body haptic sensor with multiple supporting points and its application to a manipulator

This paper proposes a whole‐body haptic sensing system that has multiple supporting points between the body frame and the end‐effector. The system consists of an end‐effector and multiple force sensors. Using this mechanism, the position of a contact force on the surface can be calculated without any sensor array. A haptic sensing system with a single supporting point structure has previously been developed by the present authors. However, the system has drawbacks such as low stiffness and low strength. Therefore, in this study, a mechanism with multiple supporting points was proposed and its performance was verified. In this paper, the basic concept of the mechanism is first introduced. Next, an evaluation of the proposed method, performed by conducting some experiments, is presented. © 2012 Wiley Periodicals, Inc. Electr Eng Jpn, 182(1): 48‐56, 2013; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/eej.22310     

Design and Analysis of Internet-Based Tele-Coordinated Multi-Robot Systems

The coordination of multi-robots is required in many scenarios for efficiency and task completion. Combined with teleoperation capabilities, coordinating robots provide a powerful tool. Add to this the Internet and now it is possible for multi-experts at multi-remote sites to control multi-robots in a coordinated fashion. For this to be feasible there are several hurdles to be crossed including Internet type delays, uncertainties in the environment and uncertainties in the object manipulated. In addition, there is a need to measure and control the quality of tele-coordination. To this end, the measure of force sensed by each robot is suggested and justified as a coordination index. It was proven that if n robots are event-transparent and event-synchronous then they can be teleoperated under random delay conditions to coordinate to any index value, which is feasible under no delay conditions. The design procedure that ensures a system can satisfy a small coordination index was presented and analyzed.In addition, the design and analysis of event-synchronous systems using Petri Nets is detailed. The Petri Net design methodology is presented for both event-synchronous single operator single robot teleoperation systems and event-synchronous multi-operator multi-robot teleoperation systems.The theory developed was tested by bilaterally tele-coordinating two mobile manipulators via the Internet. The experimental results confirmed the theoretical results presented.     

Soft Somatosensitive Actuators via Embedded 3D Printing

Humans possess manual dexterity, motor skills, and other physical abilities that rely on feedback provided by the somatosensory system. Herein, a method is reported for creating soft somatosensitive actuators (SSAs) via embedded 3D printing, which are innervated with multiple conductive features that simultaneously enable haptic, proprioceptive, and thermoceptive sensing. This novel manufacturing approach enables the seamless integration of multiple ionically conductive and fluidic features within elastomeric matrices to produce SSAs with the desired bioinspired sensing and actuation capabilities. Each printed sensor is composed of an ionically conductive gel that exhibits both long‐term stability and hysteresis‐free performance. As an exemplar, multiple SSAs are combined into a soft robotic gripper that provides proprioceptive and haptic feedback via embedded curvature, inflation, and contact sensors, including deep and fine touch contact sensors. The multimaterial manufacturing platform enables complex sensing motifs to be easily integrated into soft actuating systems, which is a necessary step toward closed‐loop feedback control of soft robots, machines, and haptic devices.