Force display using a hybrid haptic device composed of motors and brakes

In the virtual environment, force feedback to the human operator makes virtual experiences more realistic. However, the force feedback using active actuators such as motors can make the system active and sometimes unstable. To ensure the safe operation and enhance the haptic feeling, system stability should be guaranteed. Both active actuators such as motors and passive ones such as brakes are commonly used for haptic devices. Motors can generate a torque in any direction, but they can make the system active and thus, sometimes unstable during operation. On the other hand, brakes can generate a torque only against their rotation, but they dissipate energy during operation and this dissipation makes the system intrinsically stable. Consequently, motors and brakes are complementary to each other. In this research, a two degree-of-freedom (DOF) haptic device equipped with motors and brakes is designed, in which each DOF is actuated by a pair of motor and brake. Simultaneous operation of motors and brakes is analyzed. Models for some environments, virtual wall contact and frictional effect, are proposed. The results for the hybrid haptic system are compared with those for the active haptic system and the passivity based control system. The experimental results show that the hybrid haptic device is more suited to some applications than the other haptic systems.     

Design of a haptic arm exoskeleton for training and rehabilitation

A high-quality haptic interface is typically characterized by low apparent inertia and damping, high structural stiffness, minimal backlash, and absence of mechanical singularities in the workspace. In addition to these specifications, exoskeleton haptic interface design involves consideration of space and weight limitations, workspace requirements, and the kinematic constraints placed on the device by the human arm. These constraints impose conflicting design requirements on the engineer attempting to design an arm exoskeleton. In this paper, the authors present a detailed review of the requirements and constraints that are involved in the design of a high-quality haptic arm exoskeleton. In this context, the design of a five-degree-of-freedom haptic arm exoskeleton for training and rehabilitation in virtual environments is presented. The device is capable of providing kinesthetic feedback to the joints of the lower arm and wrist of the operator, and will be used in future work for robot-assisted rehabilitation and training. Motivation for such applications is based on findings that show robot-assisted physical therapy aids in the rehabilitation process following neurological injuries. As a training tool, the device provides a means to implement flexible, repeatable, and safe training methodologies.     

Haptic information in Internet-based teleoperation

Many tasks can be done easily by humans turn out to be very difficult to accomplish with a teleoperated robot. The main reason for this is the lack of tactile sensing, which cannot be replaced by visual feedback alone. Once haptic devices are developed, their potential in many fields is obvious. Especially, in teleoperation systems, where haptic feedback can increase the efficiency and even render some tasks feasible. This paper studies Internet-based teleoperation systems that include haptic feedback, concentrating on the control of such systems and their performance. The potential of this technology and its advantages are explored. In addition, key issues, such as stability, synchronization, and transparency are analyzed and studied. Specifically, an event-based planning and control of Internet-based teleoperation systems is presented with experimental results of several implemented system scenarios in micro- and macro-scales     

Stretchable Skin‐Like Cooling/Heating Device for Reconstruction of Artificial Thermal Sensation in Virtual Reality

Along with visual and tactile sensations, thermal sensation by temperature feeling on the skin can provide rich physical information on the environment and objects. With a simple touch of objects, relative temperature can be sensed and even objects can be differentiated with different thermal properties without any visual cue. Thus, artificially reproducing accurate/controllable thermal sensation haptic signals on human epidermis will certainly be a major research area to reconstruct a more realistic virtual reality (VR) environment. In this study, for the first time, a skin‐like, highly soft and stretchable and bi‐functional (both cold and hot sensation) thermo‐haptic device is reported for wearable VR applications with a single device structure (not separate heater and cooler). The skin‐like thermo‐haptic (STH) device can actively cool down and heat up deformable skin surfaces with instantaneous and accurate adjustment of temperature based upon a feedback control algorithm to mimic desirable thermal sensation with 230% stretchability. As a proof‐of‐concept, the STH device is integrated with a finger‐motion tracking glove to provide artificial thermal sensation information to the skin in various situations such as touching cold beer bottles and hot coffee cups in virtual space. This new type of STH device can offer potential implications for next‐generation haptic devices to provide unique thermal information for a more realistic virtual‐world field and medical thermal treatment.     

Haptic display with an interface device capable of continuous-time impedance display within a sampling period

Stability of a haptic interface is an important issue in virtual reality because an operator directly touches haptic interface devices. Stability is influenced by the sampling period and the discrete-time property of the control system. For decreasing the sampling system influence, this paper proposes a haptic device with an analog circuit which is placed between the computer and the haptic device. The circuit functions as springs and dampers. The control system can specify stiffness, damping coefficients, and their equilibrium. Since the impedance generated by the electric spring and damper can work continuously within the sampling period, it is effective for making the system more stable. We also develop a control method for displaying a static virtual object with the proposed device. Further, we discuss the effects of the proposed approach analytically, using passivity analysis for a 1-degree-of-freedom display system. Finally, some experimental results in a two-dimensional virtual environment are presented.     

A method for dynamics identification for haptic display of the operating feel in virtual environments

Realistic dynamics models are important for haptic display for virtual reality systems. Such dynamic models are desirably obtained via experimental identification. However, traditional dynamics identification methods normally require large sized training data sets, which maybe difficult to meet in many practical applications. To obtain the dynamics models, we present, in this paper, an identification method using support vector machines regression algorithm which is more effective than traditional methods for sparse training data. This method can be used as a generic learning machine or as a special learning technique that can make full use of the available knowledge about the dynamics structure. The experimental results show the application of our method for identifying friction models for haptic display.     

Friction modeling and compensation for haptic display based on support vector machine

As the presence of friction in a haptic display device seriously affects its performance, proper compensation of the frictional effects in such a device is of practical importance for advanced virtual reality applications where haptic display plays a critical role. This paper addresses the issue of friction modeling and compensation for haptic control system designs. A new method based on the Support Vector Machine (SVM) is developed in a controller design based on a two-port network to achieve accurate haptic display. The approximation model of friction is established offline through SVM learning and is used for online feed forward friction compensation. The advantages of this novel method are demonstrated through the experiments performed.     

Wearable Soft Technologies for Haptic Sensing and Feedback

Virtual reality (VR) and augmented reality (AR) systems have garnered recent widespread attention due to increased accessibility, functionality, and affordability. These systems sense user inputs and typically provide haptic, audio, and visual feedback to blend interactive virtual environments with the real world for an enhanced or simulated reality experience. With applications ranging from immersive entertainment, to teleoperation, to physical therapy, further development of this technology has the potential for impact across multiple disciplines. However, VR/AR devices still face critical challenges that hinder integration into everyday life and additional applications; namely, the rigid and cumbersome form factor of current technology that is incompatible with the dynamic movements and pliable limbs of the human body. Recent advancements in the field of soft materials are uniquely suited to provide solutions to this challenge. Devices fabricated from flexible and elastic bio-compatible materials have significantly greater compatibility with the human body and could lead to a more natural VR/AR experience. This review reports state-of-the-art experimental studies in soft materials for wearable sensing and haptic feedback in VR/AR applications, explores emerging soft technologies for on-body devices, and identifies current challenges and future opportunities toward seamless integration of the virtual and physical world.     

Haptic Perception,Mechanics, and Material Technologies for Virtual Reality

Emerging virtual and augmented reality technologies can transform human activities in myriad domains, lending tangible, embodied form to digital data, services, and information. Haptic technologies will play a critical role in enabling human to touch and interact with the contents of these virtual environments. The immense variety of skilled manual tasks that humans perform in real environments are only possible through the coordination of touch sensation, perception, and movement that together comprise the haptic modality. Consequently, many research groups are vigorously investigating haptic technologies for virtual reality. A longstanding research goal in this area has been to create haptic interfaces that allow their users to touch and feel plausibly realistic virtual objects. In this progress report, the perspective on this unresolved research challenge is shared, guided by the observation that no technologies can even approximately match the capabilities of the human sense of touch. Factors that have it challenging to engineer haptic technologies for virtual reality, including the extraordinary spatial and temporal tactile acuity of the skin, and the complex interplay between continuum mechanics, haptic perception, and interaction are identified. The perspective on how these challenges may be overcome through convergent research on haptic perception, mechanics, electronics, and material technologies is presented.     

Phantom-based multimodal interactions for medical education and training: the Munich knee joint simulator

Simulation environments based on virtual reality technologies can support medical education and training. In this paper, the novel approach of an "interactive phantom" is presented that allows a realistic display of haptic contact information typically generated when touching and moving human organs or segments. The key idea of the haptic interface is to attach passive phantom objects to a mechanical actuator. The phantoms look and feel as real anatomical objects. Additional visualization of internal anatomical and physiological information and sound generated during the interaction with the phantom yield a multimodal approach that can increase performance, didactic value, and immersion into the virtual environment. Compared to classical approaches, this multimodal display is convenient to use, provides realistic tactile properties, and can be partly adjusted to different, e.g., pathological properties. The interactive phantom is exemplified by a virtual human knee joint that can support orthopedic education, especially for the training of clinical knee joint evaluation. It is suggested that the technical principle can be transferred to many other fields of medical education and training such as obstetrics and dentistry.     

Performance evaluation of a tactile and kinesthetic finger feedback system for teleoperation

Teleoperation during a catastrophic event requires an interface that can perform under frequently changing circumstances caused by unpredictable and dangerous conditions. Thus, teleoperation interfaces are under active development to provide both visual and haptic feedback to the fingers. However, studies of teleoperation systems with finger haptic feedback based on force profiles are difficult to conduct because of interface limitations. Therefore, in this paper, we introduce an intuitive teleoperation interface, an anthropomorphic teleoperated robot, and a hand-wearable force-feedback system that provides various feedbacks to the fingers. We combined these systems to compare and evaluated the performance of tactile and kinesthetic finger feedback using two experiments: maintaining appropriate grip force for variably fragile objects and following a force trajectory that changed in real time. Ten subjects participated in the experiments. The results were analyzed using repeated measures analysis of variance. Feedback factors differed significantly. Provision of force feedback to the user’s finger was most effective in both teleoperation experiments.     

Design, fabrication, and evaluation of a new haptic device using aparallel mechanism

This paper presents design, fabrication, and evaluation of a new 6-DOF haptic device for interfacing with virtual reality by using a parallel mechanism. The mechanism is composed of three pantograph mechanisms that are driven by ground-fixed servomotors, three spherical joints between the top of the pantograph mechanisms and the connecting bars, and three revolute joints between the connecting bars and a mobile joystick handle. Forward and inverse kinematic analyses are performed and the Jacobian matrix is derived. Performance indexes such as global payload index, global conditioning index, translation and orientation workspaces, and sensitivity are evaluated to find optimal parameters in the design stage. The proposed haptic mechanism has better load capability than those of the pre-existing haptic mechanisms due to the fact that the motors are fixed at the base. It has also a wider orientation workspace mainly due to a RRR-type spherical joint. A control method is presented with gravity compensation and force feedback by a force/torque sensor to compensate for the effects of unmodeled dynamics such as friction and inertia. Also, the dynamic performance is evaluated for force characteristics. Virtual wall simulation with the developed haptic device is demonstrated     

Enabling 4-DoF hand guidance using a portable haptic device exerting tangential force on the user's finger pads

Body-grounded kinesthetic haptic devices can provide cues for movement in multiple degrees of freedom by exerting forces directly on users, as in dexterous robot teleoperation tasks. However, these haptic devices have limited workspaces, can destabilize a teleoperation control loop, and can be expensive. Portable haptic devices can approximate the sensations of a kinesthetic device by exploiting diverse human sense of touch principles without these shortcomings. Our goal is to analyze the feasibility of hand guidance (HG) using tangential force stimuli. Here we reveal and quantify users’ interpretation of simultaneous tactile stimulation (STS) applied to multiple finger pads of the same hand. We completed an extensive experiment on different users to reveal a maximum number of understandable cues which can be used as movement commands for HG. As expected, many tactile stimuli tested were meaningless for users, but a few could be clearly interpreted — we call these “intuitive movement cues”. For the experiment, we designed a device that can be held in the palm and exerts tactile stimuli to the user's finger pads on the thumb and index fingers, or the thumb and middle fingers. We performed two studies in which we identified the extent of salience of different movement cues. In particular, commands to redirect the hand position and orientation in four axes: moving forward/backward, wrist twisting right/left (rotate clockwise/counter-clockwise), moving right/left, and wrist tilting up/down (rotate upwards/downwards). The results revealed that this approach provided 7 intuitive directional movement cues for relative HG in 3D space. The proposed HG principle is promising for applications such as robotic surgery training, laparoscopic training, and needle insertion training, during which surgical trainees must learn dexterous hand movements involving motion paths. There are many applications for 3D movement guidance outside the medical domain that could benefit from this haptics technology, including training for precise manipulation and assembly tasks, augmented teleoperation, and communication during shared control in collaborative human-machine systems.     

Adaptive Haptic Feedback Steering Wheel for Driving Simulators

Controlling a virtual vehicle is a sensory-motor activity with a specific rendering methodology that depends on the hardware technology and the software in use. We propose a method that computes haptic feedback for the steering wheel. It is best suited for low-cost, fixed-base driving simulators but can be ported to any driving simulator platform. The goal of our method is twofold. 1) It provides an efficient yet simple algorithm to model the steering mechanism using a quadri-polar representation. 2) This model is used to compute the haptic feedback on top of which a tunable haptic augmentation is adjusted to overcome the lack of presence and the unavoidable simulation loop latencies. This algorithm helps the driver to laterally control the virtual vehicle. We also discuss the experimental results that demonstrate the usefulness of our haptic feedback method.     

Teleoperated touch feedback from the surfaces at the nanoscale: modeling and experiments

In this paper, a teleoperated nanoscale touching system is proposed, and continuum nanoscale contact mechanics models are introduced. The tele-nanorobotic system consists of a piezoresistive nanoprobe with a sharp tip as the nanorobot and force-topology sensor, a custom-made 1-degree-of-freedom haptic device for force-feedback, three-dimensional (3D) virtual reality (VR) graphics display of the nano world for visual feedback, and a force-reflecting servo type scaled teleoperation controller. Using this system, one-dimensional and 3D touching experiments and VR simulations are realized. Scaling of nano-forces is one of the major issues of the scaled teleoperation system since nanometer scale forces are dominated by surface forces instead of inertial forces as in the macro world. As the force scaling approach, a heuristic rule is introduced where nano-forces are linearly scaled with an experimentally determined scaling parameter. Simulation results and preliminary experiments of touching silicon and InAs quantum dot nanostructures show that adhesion forces at the nanoscale can be felt repeatedly at the operator's hand, and the proposed system enables the nanoscale surface topography and contact/noncontact nano-force feedback.     

Recent Advances and Opportunities of Active Materials for Haptic Technologies in Virtual and Augmented Reality

Virtual reality and augmented reality (VR/AR) are evolving. The market demands and innovation efforts call for a shift in the key VR/AR technologies and engaging people virtually. Tele-haptics with multimodal and bilateral interactions are emerging as the future of the VR/AR industry. By transmitting and receiving haptic sensations wirelessly, tele-haptics allow human-to-human interactions beyond the traditional VR/AR interactions. The core technologies for tele-haptics include multimodal tactile sensing and feedback based on highly advanced sensors and actuators. Recent developments of haptic innovations based on active materials show that active materials can significantly contribute to addressing the needs and challenges for the current and future VR/AR technologies. Thus, this paper intends to review the current status and opportunities of active material-based haptic technology with a focus on VR/AR applications. It first provides an overview of the current VR/AR applications of active materials for haptic sensing and actuation. It then highlights the state-of-the-art haptic interfaces that are relevant to the materials with an aim to provide perspectives on the role of active materials and their potential integration in haptic devices. This paper concludes with the perspective and outlook of immersive multimodal tele-haptic interaction technologies.     

Haptic joystick with hybrid actuator using air muscles and spherical MR-brake

In this research, a new 2-DOF hybrid actuator concept is explored as a powerful and compact alternative to conventional haptic actuators. The actuator combines a spherical MR-brake and three air muscles and is integrated into a joystick that can apply forces in two degrees-of-freedom. The air muscles are used to create high active forces in a compact volume and the brake compensates for the “spongy” feeling associated with air muscles. To decrease the overall size of the system an inertial measurement unit has been implemented as a position measurement solution. As high as 16 N of total force output could be achieved at the tip of the joystick. Also, up to 16 times improvement in the stable virtual wall stiffness was obtained when the MR-brake was used to compensate for force errors. Experiments with an impedance-based haptic controller with force-feedback gave satisfactory wall following performance. This device can be employed in applications including computer games, military or medical training applications, rehabilitation and in teleoperation of equipment where high force feedback in 2-DOF in a compact work volume may be desirable while interacting with rigid or elastic virtual objects.     

Reality-based models for vibration feedback in virtual environments

Reality-based modeling of vibrations has been used to enhance the haptic display of virtual environments for impact events such as tapping, although the bandwidths of many haptic displays make it difficult to accurately replicate the measured vibrations. We propose modifying reality-based vibration parameters through a series of perceptual experiments with a haptic display. We created a vibration feedback model, a decaying sinusoidal waveform, by measuring the acceleration of the stylus of a three degree-of-freedom haptic display as a human user tapped it on several real materials. A series of perceptual experiments, where human users rated the realism of various parameter combinations, were performed to further enhance the realism of the vibration display for impact events. The results provided different parameters than those derived strictly from acceleration data. Additional experiments verified the effectiveness of these modified model parameters by showing that users could differentiate between materials in a virtual environment     

Virtual environments for medical training: graphical and hapticsimulation of laparoscopic common bile duct exploration

We develop a computer-based training system to simulate laparoscopic procedures in virtual environments for medical training. The major hardware components of our system include a computer monitor to display visual interactions between 3D virtual models of organs and instruments together with a pair of force feedback devices interfaced with laparoscopic instruments to simulate haptic interactions. We simulate a surgical procedure that involves inserting a catheter into the cystic duct using a pair of laparoscopic forceps. This procedure is performed during laparoscopic cholecystectomy to search for gallstones in the common bile duct. Using the proposed system, the user can be trained to grasp and insert a flexible and freely moving catheter into the deformable cystic duct in virtual environments. The associated deformations are displayed on the computer screen and the reaction forces are fed back to the user through the force feedback devices. A hybrid modeling approach was developed to simulate the real-time visual and haptic interactions that take place between the forceps and the catheter, as well as the duct; and between the catheter and the duct     

Stereo‐Vision‐Based Human‐Computer Interaction with Tactile Stimulation

If a virtual object in a virtual environment represented by a stereo vision system could be touched by a user with some tactile feeling on his/her fingertip, the sense of reality would be heightened. To create a visual impression as if the user were directly pointing to a desired point on a virtual object with his/her own finger, we need to align virtual space coordinates and physical space coordinates. Also, if there is no tactile feeling when the user touches a virtual object, the virtual object would seem to be a ghost. Therefore, a haptic interface device is required to give some tactile sensation to the user. We have constructed such a human‐computer interaction system in the form of a simple virtual reality game using a stereo vision system, a vibro‐tactile device module, and two position/orientation sensors.