Finding the Latent Semantics of Haptic Interaction Research: A Systematic Literature Review of Haptic Interaction Using Content Analysis and Network Analysis

As interest in multimodal and tangible interfaces is increasing in the field of human–robot interaction and virtual reality, haptics has been researched across areas such as engineering, computer science, psychology, and neuroscience. The main objective of the study was to construct a comprehensive review of the current haptic‐related literature based on quantitative data derived from content analysis and network analysis. Using the results of content analysis and network analysis of 6,000 research articles on haptic interaction, the haptic‐related literature was classified into two categories: 1) studies on technologies providing haptic stimuli and 2) studies on the human perception of haptic stimuli. Emotions in haptic feedback and haptic perception characteristics of various body sites were identified as potential research topics for further investigation. Greater research effort on understanding human haptic sensation and perception using the proposed systematic approach could accelerate the development of haptic interaction technology.     

Virtual prototyping of mechanical systems with tool mediated haptic feedback

Haptic feedback usually involves two types of stimulation forces: forces that address the touch sense and forces that address the kinesthetic perception. Touch forces have a low intensity and a complex structure since they reflect contact phenomena where friction plays an important role. Therefore, they are quite difficult to simulate. Virtual prototyping with haptic feedback should ideally involve both types of forces, but the integration of the touch feeling makes the simulator very complex. In this paper, we present a novel concept for virtual prototyping in which the touch interaction is separated from the kinesthetic force feedback. This is possible using a prototype that has a real part undertaking the touch interaction and a virtual part that simulate feedback for the kinesthetic forces. In this way, a full haptic interaction with the virtual prototype is established by means of a device that provides a realistic simulation of the product. In order to illustrate the concept, several experiments have been carried out for the case of specific subsystems of a car, which are particularly involved in the driver–car interaction: steering system, clutch pedal and the gearshift. A user test is described in the last part as well as the conclusions of the research.     

Haptic Direct-Drive Robot Control Scheme in Virtual Reality

This paper explores the use of a 2-D (Direct-Drive Arm) manipulator for mechanism design applications based on virtual reality (VR). This article reviews the system include a user interface, a simulator, and a robot control scheme. The user interface is a combination of a virtual clay environment and human arm dynamics via robot effector handler. The model of the VR system is built based on a haptic interface device behavior that enables the operator to feel the actual force feedback from the virtual environment just as s/he would from the real environment. A primary stabilizing controller is used to develop a haptic interface device where realistic simulations of the dynamic interaction forces between a human operator and the simulated virtual object/mechanism are required. The stability and performance of the system are studied and analyzed based on the Nyquist stability criterion. Experiments on cutting virtual clay are used to validate the theoretical developments. It was shown that the experimental and theoretical results are in good agreement and that the designed controller is robust to constrained/unconstrained environment.     

视触觉融合的增强现实三维注册方法

增强现实技术是近年来人机交互领域的研究热点。在增强现实环境下加入触觉感知,可使用户在真实场景中看到并感知到虚拟对象。为了实现增强现实环境下与虚拟对象之间更加自然的交互,提出一种视触觉融合的三维注册方法。基于图像视觉技术获得三维注册矩阵;借助空间转换关系求解出触觉空间与图像空间的转换关系;结合两者与摄像头空间的关系实现视触觉融合的增强现实交互场景。为验证该方法的有效性,设计了一个基于视触觉增强现实的组装机器人项目。用户可触摸并移动真实环境中的机器人零件,还能在触摸时感受到反馈力,使交互更具真实感。     

一种基于物理意义的快速力反馈形变模型及实时力觉响应算法

虚拟物体在受力作用时的形变建模是虚拟环境中力/触觉人机交互的关键.文中提出了一种新的基于物理意义的形变建模方法,不仅计算速度快,满足力反馈的实时性要求,而且能够同时保证接触力和形变的计算具有较高的精度,适用于具有较大变形量的柔性物体的力反馈计算,满足精细作业对虚拟现实系统的要求.     

Influences of haptic communication on a shared manual task

With the advent of new haptic feedback devices, researchers are giving serious consideration to the incorporation of haptic communication in collaborative virtual environments. For instance, haptic interactions based tools can be used for medical and related education whereby students can train in minimal invasive surgery using virtual reality before approaching human subjects. To design virtual environments that support haptic communication, a deeper understanding of humans′ haptic interactions is required. In this paper, human′s haptic collaboration is investigated. A collaborative virtual environment was designed to support performing a shared manual task. To evaluate this system, 60 medical students participated to an experimental study. Participants were asked to perform in dyads a needle insertion task after a training period. Results show that compared to conventional training methods, a visual-haptic training improves user′s collaborative performance. In addition, we found that haptic interaction influences the partners′ verbal communication when sharing haptic information. This indicates that the haptic communication training changes the nature of the users′ mental representations. Finally, we found that haptic interactions increased the sense of copresence in the virtual environment: haptic communication facilitates users′ collaboration in a shared manual task within a shared virtual environment. Design implications for including haptic communication in virtual environments are outlined.     

A sequenced multimodal learning approach to support students' development of conceptual learning

Virtual learning environments can now be enriched not only with visual and auditory information, but also with tactile and kinesthetic feedback. However, the way to successfully integrate haptic feedback on a multimodal learning environment is still unclear. This study aims to provide guidelines on how visuohaptic simulations can be implemented effectively, thus the research question is: Under what conditions do visual and tactile information support students' development of conceptual learning of force‐related concepts? Participants comprised 170 undergraduate students of a Midwestern University enrolled in a physics for elementary education class. Four experiments were conducted using four different configurations of multimodal learning environments: Visual feedback only, haptic force feedback only, visual and haptic force feedback at the same time, and sequenced modality of haptic feedback first and visual feedback second. Our results suggest that haptic force feedback has the potential to enrich learning when compared with visual only environments. Also, haptic and visual modalities interact better when sequenced one after another rather than presented simultaneously. Finally, exposure to virtual learning environments enhanced by haptic forced feedback was a positive experience, but the ease of use and ease of interpretation was not so evident.     

Effect of Scaling on the Performance and Stability of Teleoperation Systems Interacting with Soft Environments

There is generally a tradeoff between stability and performance in haptic control systems. Teleoperation systems with haptic feedback are no exception. Scaling in these systems used in applications such as telemicrosurgical systems has further effects on the stability and performance. This paper focuses on those applications interacting with soft tissues and analyzes the effects of the scaling in an effort to increase the performance of these systems while maintaining the stability. Position tracking and kinesthetic perception are especially important in the tele-surgical systems and, hence, are used as the performance criteria. Quantitatively defined stability robustness, which is based on Llewellyn's absolute stability criterion, is used as a metric for stability analysis. Various choices of scaling factors, and human and environment impedances are then investigated. The proposed kinesthetic perception concept is validated using psychophysical experiments. Widely used bilateral control architectures such as the two-channel position–position, two-channel force–position and four-channel controls are specifically analyzed and evaluated using simulations and experiments with phantom soft tissues. Results also show that the force–position control architecture shows the best position tracking performance irrespective of the scaling factors while the four-channel controller shows the best kinesthetic perception capability.     

Human Interaction with Motion Planning Algorithm

This paper presents an interactive motion planning system to compute free collision motion in a numerical model. The system is based on interaction between a user and a motion planning algorithm. On one hand the user moves the object with an interactive device and on the other hand a motion planning algorithm searches a solution in the configuration space. The interaction aims at improving the guidance of an operator during a robot motion task in a virtual environment with the help of an automatic path planning algorithm. Existing works use a two-step decomposition which limits the interaction between the user and the ongoing process. We propose a modification of a classic motion planning method, the Rapidly-exploring Random Tree to build an Interactive-RRT. This method is based on exchanging pseudo-forces between the algorithm and the user, and on data gathering (labels) from the virtual scene. Examples are shown to illustrate the Interactive motion planning system with different interactive devices (space mouse and haptic arm). We analyze the influence of the user’s dexterity to find a solution depending on various parameters of the algorithm and we show how we can adapt these parameters to a user.     

Beyond the visuals: tactile augmentation and sensory enhancement in an arthroscopy simulator

This paper considers tactile augmentation, the addition of a physical object within a virtual environment (VE) to provide haptic feedback. The resulting mixed reality environment is limited in terms of the ease with which changes can be made to the haptic properties of objects within it. Therefore sensory enhancements or illusions that make use of visual cues to alter the perceived hardness of a physical object allowing variation in haptic properties are considered. Experimental work demonstrates that a single physical surface can be made to ‘feel’ both softer and harder than it is in reality by the accompanying visual information presented. The strong impact visual cues have on the overall perception of object hardness, indicates haptic accuracy may not be essential for a realistic virtual experience. The experimental results are related specifically to the development of a VE for surgical training; however, the conclusions drawn are broadly applicable to the simulation of touch and the understanding of haptic perception within VEs.     

Haptic rendering: introductory concepts

Haptic rendering allows users to "feel" virtual objects in a simulated environment. We survey current haptic systems and discuss some basic haptic-rendering algorithms. In the past decade we've seen an enormous increase in interest in the science of haptics. Haptics broadly refers to touch interactions (physical contact) that occur for the purpose of perception or manipulation of objects. These interactions can be between a human hand and a real object; a robot end-effector and a real object; a human hand and a simulated object (via haptic interface devices); or a variety of combinations of human and machine interactions with real, remote, or virtual objects. Rendering refers to the process by which desired sensory stimuli are imposed on the user to convey information about a virtual haptic object.     

Networked multiplayer cooperative interaction using decoupled motion control method in a shared virtual environment with haptic,visual and movement feedback

This paper focuses on multiplayer cooperative interaction in a shared haptic environment based on a local area network. Decoupled motion control, which allows one user to manipulate a haptic interface to control only one‐dimensional movement of an avatar, is presented as a new type haptic‐based cooperation among multiple users. Users respectively move an avatar along one coordinate axis so that the motion of the avatar is the synthesis of movements along all axes. It is different from previous haptic cooperation where all users can apply forces on an avatar along any direction to move it, the motion of which completely depends on the resultant force. A novel concept of movement feedback is put forward where one user can sense other users’ hand motions through his or her own haptic interface. The concept can also be explained wherein one person who is required to move a virtual object along only one axis can also feel the motions of the virtual object along other axes. Movement feedback, which is a feeling of motion, differs from force feedback, such as gravity, collision force and resistance. A spring‐damper force model is proposed for the computation of motion feedback to implement movement transmission among users through haptic devices. Experimental results validate that movement feedback is beneficial for performance enhancement of such kind of haptic‐based cooperation, and the effect of movement feedback in performance improvement is also evaluated by all subjects.Copyright © 2012 John Wiley & Sons, Ltd.     

Handling of Virtual Contact in Immersive Virtual Environments: Beyond Visuals

This paper addresses the issue of improving the perception of contact that users make with purely virtual objects in virtual environments. Because these objects have no physical component, the user's perceptual understanding of the material properties of the object, and of the nature of the contact, is hindered, often limited solely to visual feedback. Many techniques for providing haptic feedback to compensate for the lack of touch in virtual environments have been proposed. These systems have increased our understanding of the nature of how humans perceive contact. However, providing effective, general-purpose haptic feedback solutions has proven elusive. We propose a more-holistic approach, incorporating feedback to several modalities in concert. This paper describes a prototype system we have developed for delivering vibrotactile feedback to the user. The system provides a low-cost, distributed, portable solution for incorporating vibrotactile feedback into various types of systems. We discuss different parameters that can be manipulated to provide different sensations, propose ways in which this feedback can be combined with feedback of other modalities to create a better understanding of virtual contact, and describe possible applications.     

基于接触有限元模型的虚拟手指力建模研究

触觉反馈是虚拟现实应用中使人获得沉浸感的重要方式。随 显示技术的日趋成熟,触觉和力反馈的研究的焦点之一,如何度量虚拟手在接触物体时反馈的力的大小 是力反馈中的首要一步,对此,该文在研究了力反馈类型的基础上,给和指与虚拟物体间的接触数学模型,提出了基于接触有限元的接触数学模型,并有杉ＡＮＳＹＳ对霏 均匀有理Ｂ样条的手指表面和虚拟按钮间的接触变形和力分布情况进行了计算,得出了力反馈所需要的接触力大小。     

Building a virtual environment for endoscopic sinus surgery simulation

Advanced display technologies have made the virtual exploration of relatively complex models feasible in many applications. Unfortunately, only a few human interfaces allow natural interaction with the environment. Moreover, in surgical applications, such realistic interaction requires real-time rendering of volumetric data—placing an overwhelming performance burden on the system. We report on our advances towards developing a virtual reality system that provides intuitive interaction with complex volume data by employing real-time realistic volume rendering and convincing forece feedback (haptic) sensations. We describe our methods for real-time volume rendering, model deformation, interaction, and the haptic devices, and demonstrate the utilization of this system in the real-world application of Endoscopic Sinus Surgery (ESS) simulation.     

SPH haptic interaction with multiple-fluid simulation

Physics-based fluid interaction plays an important role in computer animation, with wide applications in virtual reality, computer games, digital entertainment, etc. For example, in virtual reality education and games, we often need fluid interactions like acting as an alchemist to create a potion by stirring fluid in a crucible. The traditional input devices such as a mouse and keyboard can basically input 2D information without feedback. In recent years, the continuous development of haptic device not only can achieve six degrees-of-freedom input, but also can calculate the force in virtual scenes and feedback to the user to make a better virtual experience. How to use haptic device in different kinds of virtual fluid scenarios to provide better experience is an important issue in the field of virtual reality. On the other hand, the researches on multiple-fluid interaction especially based on smoothed particle hydrodynamics (SPH) method are very lacking. Therefore, we study the key techniques of haptic interaction with SPH multiple-fluid to compensate this defect in computer graphics community. Different from the single-phase flow, interaction with multiple-fluid flow has difficulties in the realization of properties of different phases. After adding the multiple-fluid simulation, it is also important to keep haptic interaction real time. Our research is based on the mixture model. We guarantee the authenticity of multiple-fluid mixing effect while changing the drift velocity solver to improve efficiency. We employ a unified particle model to achieve rigid body–liquid coupling, and use FIR filter to smooth feedback force to the haptic device. Our novel multiple-fluid haptic simulation can provide an interactive experience for mixing liquid in virtual reality.     

Human-assisted virtual reality for a magnetic-haptic micromanipulation platform

This paper deals with the development of a virtual reality interface (VRI) for a magnetic-haptic micromanipulation platform (MHMP) (Mehrtash et?al. in IEEE/ASME Trans Mechatron 16(3):459–469, 2011). Our previously developed MHMP has shown a great deal of promise in non-contact micromanipulations. This micromanipulation platform concerns the integration of magnetic actuation technology and a bilateral macro–micro teleoperation. The MHMP has two separate stations: one magnetic microrobotic station and one haptic. The magnetic microrobotic station manipulates micro-sized objects based on the commands from the haptic station. The haptic station uses bilateral communication with the magnetic microrobotic station to allow a human operator the feeling of a micro-domain environment. In this paper, we report a VRI that enables human operators to improve their skills in using the MHMP, before carrying out an actual dexterous task. The VRI is made up of three main components: a haptic station, a simulation engine, and a display unit. The haptic station provides the operator with the force/torque information from virtual or remote environments, and is also used to recognize the operator’s hand motion command. Dynamical computation and control system modeling have been carried out on the simulation engine. Based on the real-time computation, this engine, as the heart of the system, provides force applied to the operator’s hand and the microrobot’s position for the haptic station and the display unit, respectively. The display unit employs 3D computer graphics to demonstrate the micromanipulation tasks and environments. The VRI is also developed in such a way that it can be separately used in parallel with the MHMP for the 3D visualization of a real task by providing multiple virtual viewports. This paper introduces the configuration of the proposed VRI, and reports the result of a preliminary experiment using micromanipulation investigation for validation.     

An evaluation of the virtual curvature with the StickGrip haptic device: a case study

Dynamic simulation of distance to the physical surface could promote the development of new inexpensive tools for blind and visually impaired users. The StickGrip is a haptic device comprised of the Wacom pen input device added with a motorized penholder. The goal of the research presented in this paper was to assess the accuracy and usefulness of the new pen-based interaction technique when the position and displacement of the penholder in relation to the pen tip provided haptic feedback to the user about the distance to the physical or virtual surface of interaction. The aim was to examine how accurately people are able (1) to align the randomly deformed virtual surfaces to the flat surface and (2) to adjust the number of surface samples having a randomly assigned curvature to the template having the given curvature and kept fixed. These questions were approached by measuring both the values of the adjusted parameters and the parameters of the human performance, such as a ratio between inspection time and control time spent by the participants to complete the matching task with the use of the StickGrip device. The test of the pen-based interaction technique was conducted in the absence of visual feedback when the subject could rely on the proprioception and kinesthetic sense. The results are expected to be useful for alternative visualization and interaction with complex topographic and mathematical surfaces, artwork, and modeling.     

Haptic force control based on impedance/admittance control aided by visual feedback

This paper demonstrates a haptic device for interaction with a virtual environment. The force control is added by visual feedback that makes the system more responsive and accurate. There are two popular control methods widely used in haptic controller design. First, is impedance control when user motion input is measured, and then, the reaction force is fed back to the operator. The alternative method is admittance control, when forces exerted by user are measured and motion is fed back to the user. Both, impedance and admittance control are also basic ways for interacting with a virtual environment. In this paper, several experiments were performed to evaluate the suitability of force-impedance control for haptic interface development. The difference between conventional application of impedance control in robot motion control and its application in haptic interface development is investigated. Open loop impedance control methodology is implemented for static case and a general-purpose robot under open loop impedance control was developed as a haptic device, while a closed loop model based impedance control was used for haptic controller design in both static and dynamic case. The factors that could affect to the performance of a haptic interface are also investigated experimentally using parametric studies. Experimental results for 1 DOF rotational motion and 2 DOF planar translational motion systems are presented. The results show that the impedance control aided by visual feedback broaden the applicability of the haptic device and makes the system more responsive and accurate.

虚拟现实的力触觉交互技术发展现状与趋势

虚拟现实通过模拟人的视觉、听觉、力触觉等,使人处于一种与真实世界非常逼真的虚拟世界中,来感受、体验和评价虚拟世界中的场景和设备。力触觉在虚拟现实环境中有其突出优越性,力触觉使得虚拟现实环境变得真实,是唯一的既可接受周围环境输入又可以对周围环境输出的感知通道,可极大增强可视化表达的效果。虚拟现实的力触觉交互技术包括力触觉再现技术和虚拟环境的力触觉建模。介绍了这两方面的发展现状、存在的问题和今后的发展趋势。