Toward realistic haptic rendering of surface textures

New sophisticated haptic-rendering algorithms let users experience virtual objects through touch. We systematically investigate the unrealistic behavior of virtual haptic textures. The emerging science of haptic rendering consists of delivering properties of physical objects through the sense of touch. Owing to the recent development of sophisticated haptic-rendering algorithms, users can now experience virtual objects through touch in many exciting applications, including surgical simulations, virtual prototyping, and data perceptualization. Haptics holds great promise to enrich the sensory attributes of virtual objects that these systems can produce. One area that has received increasing attention in the haptics community is haptic texture rendering, the goal of which is to introduce micro-geometry-scale features on object surfaces. Haptic objects rendered without textures usually feel smooth, and sometimes slippery. Appropriate haptic textures superimposed on haptic objects enhance an object's realism.     

Calibration and Evaluation for Visuo-haptic Collocation in Haptic Augmented Virtuality Systems

International Journal of Control, Automation and Systems - Haptic augmented virtuality systems can provide users with a highly immersive haptic experience by visuo-haptic collocation of physical...     

振动触觉纹理再现技术研究综述

纹理力触觉再现是通过特定的硬件装置模拟产生与物体纹理表面接触时的触感，使用户能感受到物体的粗糙度、软硬度等纹理特征信息。振动刺激作为再现物体触觉信息的一种刺激方式，在纹理触觉再现中被广泛运用，产生了不同的振动触觉表达装置和纹理触觉表达方法。从纹理触觉认知的角度，阐述了人对振动刺激的触觉感知生理学基础；介绍了纹理触觉再现的原理和方法；从振动与纹理特征的映射方法以及振动刺激方式两个方面分析了目前振动触觉纹理再现技术的发展现状；最后对相关研究的发展进行了总结展望。     

Enhancing personal communication with spatial haptics: Two scenario-based experiments on gestural interaction

Haptic gestures and sensations through the sense of touch are currently unavailable in remote communication. There are two main reasons for this: good quality haptic technology has not been widely available and knowledge on the use of this technology is limited. To address these challenges, we studied how users would like to, and managed to create spatial haptic information by gesturing. Two separate scenario-based experiments were carried out: an observation study without technological limitations, and a study on gesturing with a functional prototype with haptic actuators. The first study found three different use strategies for the device. The most common gestures were shaking, smoothing and tapping. Multimodality was requested to create the context for the communication and to aid the interpretation of haptic stimuli. The second study showed that users were able to utilize spatiality in haptic messages (e.g., forward–backward gesture for agreement). However, challenges remain in presenting more complex information via remote haptic communication. The results give guidance for communication activities that are usable in spatial haptic communication, and how to make it possible to enable this form of communication in reality.     

Haptic Cooperative Virtual Workspace: Architecture and evaluation

The Haptic Cooperative Virtual Workspace (HCVW), where users can simultaneously manipulate and haptically feel the same object, is beneficial and in some cases indispensable for training a team of surgeons, or in application areas in telerobotics and entertainment. In this paper we propose an architecture for the haptic cooperative workspace where the participants can kinesthetically interact, feel and push each other simultaneously while moving in the simulation. This involves the ability to manipulate the same virtual object at the same time. A set of experiments carried out to investigate the haptic cooperative workspace is reported. A new approach to quantitatively evaluate the cooperative haptic system is proposed, which can be extended to evaluate haptic systems in general.     

虚拟骨科手术中触觉交互建模方法综述

虚拟骨科手术可以让骨科医生使用多种虚拟骨科手术器械在三维重建的骨性解剖结构模型上进行骨科手术操作和 训练。力触觉交互在骨科手术中起着不容忽视的作用,如何在骨科手术模拟中实现真实快速的触觉交互,已成为近年来 虚拟骨科手术研究的热点。文章主要对现有的骨科手术模拟系统中用到的触觉交互模型进行了研究、分类和总结,分析 了现有的一些模型的特点和不足,并展望了骨科手术系统中未来力触觉交互的发展。     

A Survey of Haptic Rendering Techniques

Computer Graphics technologies have developed considerably over the past decades. Realistic virtual environments can be produced incorporating complex geometry for graphical objects and utilising hardware acceleration for per pixel effects. To enhance these environments, in terms of the immersive experience perceived by users, the human's sense of touch, or haptic system, can be exploited. To this end haptic feedback devices capable of exerting forces on the user are incorporated. The process of determining a reaction force for a given position of the haptic device is known as haptic rendering. For over a decade users have been able to interact with a virtual environment with a haptic device. This paper focuses on the haptic rendering algorithms which have been developed to compute forces as users manipulate the haptic device in the virtual environment.     

Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user’s sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application’s Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.     

Multi-modal virtual environments for education with haptic and olfactory feedback

It has been suggested that immersive virtual reality (VR) technology allows knowledge-building experiences and in this way provides an alternative educational process. Important key features of constructivist educational computer-based environments for science teaching and learning, include interaction, size, transduction and reification. Indeed, multi-sensory VR technology suits very well the needs of sciences that require a higher level of visualization and interaction. Haptics that refers to physical interactions with virtual environments (VEs) may be coupled with other sensory modalities such as vision and audition but are hardly ever associated with other feedback channels, such as olfactory feedback. A survey of theory and existing VEs including haptic or olfactory feedback, especially in the field of education is provided. Our multi-modal human-scale VE VIREPSE (virtual reality platform for simulation and experimentation) that provides haptic interaction using a string-based interface called SPIDAR (space interface device for artificial reality), olfactory and auditory feedbacks is described. An application that allows students experiencing the abstract concept of the Bohr atomic model and the quantization of the energy levels has been developed. Different configurations that support interaction, size and reification through the use of immersive and multi-modal (visual, haptic, auditory and olfactory) feedback are proposed for further evaluation. Haptic interaction is achieved using different techniques ranging from desktop pseudo-haptic feedback to human-scale haptic interaction. Olfactory information is provided using different fan-based olfactory displays (ODs). Significance of developing such multi-modal VEs for education is discussed.     

Interactive mixed reality white cane simulation for the training of the blind and the visually impaired

This paper presents a mixed reality tool developed for the training of the visually impaired based on haptic and auditory feedback. The proposed approach focuses on the development of a highly interactive and extensible Haptic Mixed Reality training system that allows visually impaired to navigate into real size Virtual Reality environments. The system is based on the use of the CyberGrasp™ haptic device. An efficient collision detection algorithm based on superquadrics is also integrated into the system so as to allow real time collision detection in complex environments. A set of evaluation tests is designed in order to identify the importance of haptic, auditory and multimodal feedback and to compare the MR cane against the existing Virtual Reality cane simulation system.     

Augmenting graphical user interfaces with haptic assistance for motion-impaired operators

Haptic assistance is an emerging field of research that is designed to improve human–computer interaction (HCI) by reducing error rates and targeting times through the use of force feedback. Haptic feedback has previously been investigated to assist motion-impaired computer users, however, limitations such as target distracters have hampered its integration with graphical user interfaces (GUIs). In this paper two new haptic assistive techniques are presented that utilise the 3DOF capabilities of the Phantom Omni. These are referred to as deformable haptic cones and deformable virtual switches. The assistance is designed specifically to enable motion-impaired operators to use existing GUIs more effectively. Experiment 1 investigates the performance benefits of the new haptic techniques when used in conjunction with the densely populated Windows on-screen keyboard (OSK). Experiment 2 utilises the ISO 9241-9 point-and-click task to investigate the effects of target size and shape. The results of the study prove that the newly proposed techniques improve interaction rates and can be integrated with existing software without many of the drawbacks of traditional haptic assistance. Deformable haptic cones and deformable virtual switches were shown to reduce the mean number of missed-clicks by at least 75% and reduce targeting times by at least 25%.     

Enhancing Realism of Wet Surfaces in Temporal Bone Surgical Simulation

We present techniques to improve visual realism in an interactive surgical simulation application: a mastoidectomy simulator that offers a training environment for medical residents as a complement to using a cadaver. As well as displaying the mastoid bone through volume rendering, the simulation allows users to experience haptic feedback and appropriate sound cues while controlling a virtual bone drill and suction/irrigation device. The techniques employed to improve realism consist of a fluid simulator and a shading model. The former allows for deformable boundaries based on volumetric bone data, while the latter gives a wet look to the rendered bone to emulate more closely the appearance of the bone in a surgical environment. The fluid rendering includes bleeding effects, meniscus rendering, and refraction. We incorporate a planar computational fluid dynamics simulation into our three-dimensional rendering to effect realistic blood diffusion. Maintaining real-time performance while drilling away bone in the simulation is critical for engagement with the system.     

An information-theoretic treatment of passive haptic media

Haptic rendering has been long considered as the process of estimating the force that stems from the interaction of a user and an object. Even if this approach follows the principles of natural haptic interaction, it places severe limitations in processing haptic media. This paper presents an information theoretic framework that aims to provide a new view of haptic rendering that can accommodate for open-loop synthetic haptic media, where interaction-based rendering is a special case. As a result, using the proposed information-theoretic approach, the haptic signal can be precomputed as a force field, stored and then filtered by taking into account device and perceptual capabilities of the receiver in order to lower the required bandwidth of the resulting stream, thus opening new possibilities for the representation and processing of haptic media.     

Simulating a syringe behavior using a pneumatic cylinder haptic interface

Mastering medical gestures, such as epidural needle insertion, requires much practice. Haptic interfaces can be efficient training solutions. Nevertheless, they must provide users with an accurate experience for an effective training. This article introduces a control framework that turns a pneumatic cylinder into a haptic interface able to simulate a syringe and more specifically the loss of resistance phenomenon encountered during an epidural needle insertion. To achieve this, the framework switches between two control laws (position and force tracking) according to the system state. Experiments involving two anesthetists confirm the practicality of this system for hands-on training or rehearsal purposes.     

Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations

We survey the haptics literature and identify conditions under which haptic interaction displays can enhance human perception and performance. Integrating haptic interactions in multimodal systems requires understanding user's sensory, perceptual, and cognitive abilities and limitations. Haptic design guidelines can aid developers of multimodal interactive systems. Haptic interaction relates to all aspects of touch and body movement and the application of these senses to computer interaction. This involves not only sensation and perception, but also motor and cognitive aspects of active movement (that is, self-initiated movement) for which detailed motor plans are created, stored in memory, and compared to receptor feedback from the muscles, joints, and skin.     

The road to surgical simulation and surgical navigation

The recent advantage of the power of graphic workstations has made it possible to handle 3D human structures in an interactive way. Real-time imaging of medical 3D or 4D images can be used not only for diagnosis, but also for various novel medical treatments. By elaborating on the history of the establishment of our laboratory, which focuses on medical virtual reality, we describe our experience of developing surgery simulation and surgery navigation systems according to our research results. In the case of surgical simulation, we mention two kinds of virtual surgery simulators that produce the haptic sensation of surgical maneuvers in the user’s fingers. Regarding surgical navigation systems, we explain the necessity of the augmented reality function for the encouragement of the ability of robotic surgery and its trial for clinical case.     

Identifying the effectiveness of using three different haptic devices for providing non-visual access to the web

Haptic technologies are often used to improve access to the structural content of graphical user interfaces, thereby augmenting the interaction process for blind users. While haptic design guidelines offer valuable assistance when developing non-visual interfaces, the recommendations presented are often tailored to the feedback produced via one particular haptic input/output device. A blind user is therefore restricted to interacting with a device which may be unfamiliar to him/her, rather than selecting from the range of commercially available products. This paper reviews devices available on the first and second-hand markets, and describes an exploratory study undertaken with 12 blindfolded sighted participants to determine the effectiveness of three devices for non-visual web interaction. The force-feedback devices chosen for the study, ranged in the number of translations and rotations that the user was able to perform when interacting with them. Results have indicated that the Novint Falcon could be used to target items faster in the first task presented, compared with the other devices. However, participants agreed that the force-feedback mouse was most comfortable to use when interacting with the interface. Findings have highlighted the benefits which low cost haptic input/output devices can offer to the non-visual browsing process, and any changes which may need to be made to accommodate their deficiencies. The study has also highlighted the need for web designers to integrate appropriate haptic feedback on their web sites to cater for the strengths and weaknesses of various devices, in order to provide universally accessible sites and online applications.     

Haptic rendering: introductory concepts

Haptic rendering allows users to "feel" virtual objects in a simulated environment. We survey current haptic systems and discuss some basic haptic-rendering algorithms. In the past decade we've seen an enormous increase in interest in the science of haptics. Haptics broadly refers to touch interactions (physical contact) that occur for the purpose of perception or manipulation of objects. These interactions can be between a human hand and a real object; a robot end-effector and a real object; a human hand and a simulated object (via haptic interface devices); or a variety of combinations of human and machine interactions with real, remote, or virtual objects. Rendering refers to the process by which desired sensory stimuli are imposed on the user to convey information about a virtual haptic object.     

Haptic palpation for medical simulation in virtual environments

Palpation is a physical examination technique where objects, e.g., organs or body parts, are touched with fingers to determine their size, shape, consistency and location. Many medical procedures utilize palpation as a supplementary interaction technique and it can be therefore considered as an essential basic method. However, palpation is mostly neglected in medical training simulators, with the exception of very specialized simulators that solely focus on palpation, e.g., for manual cancer detection. In this article we propose a novel approach to enable haptic palpation interaction for virtual reality-based medical simulators. The main contribution is an extensive user study conducted with a large group of medical experts. To provide a plausible simulation framework for this user study, we contribute a novel and detailed interaction algorithm for palpation with tissue dragging, which utilizes a multi-object force algorithm to support multiple layers of anatomy and a pulse force algorithm for simulation of an arterial pulse. Furthermore, we propose a modification for an off–the–shelf haptic device by adding a lightweight palpation pad to support a more realistic finger grip configuration for palpation tasks. The user study itself has been conducted on a medical training simulator prototype with a specific procedure from regional anesthesia, which strongly depends on palpation. The prototype utilizes a co-rotational finite-element approach for soft tissue simulation and provides bimanual interaction by combining the aforementioned techniques with needle insertion for the other hand. The results of the user study suggest reasonable face validity of the simulator prototype and in particular validate medical plausibility of the proposed palpation interaction algorithm.