The influence of combined visual and tactile information on finger and eye movements during shape tracing.

Adaptation experiments in shape tracing were conducted to investigate finger and eye movements in various conditions of visual and tactile information. Maximum velocity, mean velocity, maximum acceleration and reacceleration point were calculated from finger movements. Number of eye fixations and lead time of eye fixation to finger position were calculated from eye movements. The results showed that for the finger movement the values of the indices studied were higher in the combined visual and tactile condition than in the visual only condition. The number of eye fixations decreased when subjects repeated the tracing and was more marked in the combined visual and tactile condition than in the visual only condition. The results suggest that finger movements become faster and use of vision is reduced when both visual and tactile information are given.     

The influence of combined visual and tactile information on finger and eye movements during shape tracing

Abstract

Adaptation experiments in shape tracing were conducted to investigate finger and eye movements in various conditions of visual and tactile information. Maximum velocity, mean velocity, maximum acceleration and reacceleration point were calculated from finger movements. Number of eye fixations and lead time of eye fixation to finger position were calculated from eye movements. The results showed that for the finger movement the values of the indices studied were higher in the combined visual and tactile condition than in the visual only condition. The number of eye fixations decreased when subjects repeated the tracing and was more marked in the combined visual and tactile condition than in the visual only condition. The results suggest that finger movements become faster and use of vision is reduced when both visual and tactile information are given.     

Haptic display of micro surface undulation based on discrete mechanical stimuli to whole fingers

Humans can perceive a wide and small surface undulation that is hundreds micrometers in height and hundreds millimeters in width by scanning the surface with their whole fingers and palm in the distal and proximal directions. We developed a wearable haptic device that presents a surface undulation to the hand. The device is composed of nine independent stimulator units that control the heights of nine finger pads of the index finger, the middle finger, and the ring finger (three units on each finger) according to the virtual surface. Three experiments are carried out to evaluate the haptic perception by the haptic device. A first experiment shows that the perceived dimensions are diminished as compared to the dimensions applied by the haptic device. On the basis of this result, the applied dimensions are calibrated to match the virtual surface undulation to the real surface undulation. A second experiment shows that the shape of gently-curved surfaces can be estimated with the haptic display. A third experiment shows that the discrimination threshold is not different between the virtual surface undulation and the real surface undulation. These experimental results show the applicability of the haptic device as a haptic interface.     

Design and Control of Five-Fingered Haptic Interface Opposite to Human Hand

This paper presents the design and control of a newly developed five-fingered haptic interface robot named HIRO II. The developed haptic interface can present force and tactile feeling to the five fingertips of the human hand. Its mechanism consists of a 6 degree of freedom (DOF) arm and a 15 DOF hand. The interface is placed opposite the human hand, which ensures safety and freedom of movement, but this arrangement leads to difficulty in designing and controlling the haptic interface, which should accurately track the fingertip positions of the operator. A design concept and optimum haptic finger layout, which maximizes the design performance index is presented. The design performance index consists of the product space between the operator's finger and the hapic finger, and the opposability of the thumb and fingers. Moreover, in order to reduce the feeling of uneasiness in the operator, a mixed control method consisting of a finger-force control and an arm position control intended to maximize the control performance index, which consists of the hand manipulability measure and the norm of the arm-joint angle vector is proposed. The experimental results demonstrate the high potential of the multifingered haptic interface robot HIRO II⁺ utilizing the mixed control method.     

Tactile display with tangential and normal skin displacement for robot-assisted surgery

This paper proposes a tactile display providing both shear and normal feedback to the fingertip for generating three-axis tactile feedback during teleoperation of a surgical robot. The display is composed of five balloons actuated by controlling the pneumatic pressure. The implemented display is 18?mm?×?18?mm?×?15?mm. This size is suitable for mounting the display onto the master controls of a surgical robot. The maximum normal and shear displacements are 2 and 1.3?mm, respectively. The proposed tactile display may provide perceivable stimuli to a human finger pad in all five directions: normal, distal, proximal, radial, and ulnar. This paper also reports on the results of psychophysical measurement of the minimum perceivable movement of the developed tactile display for each of the five directions.     

High performance haptic device for force rendering in textile exploration

The HAPTEX system aims to develop a new multi-sensory environment for the immersive exploration of textiles. HAPTEX is based on a multi-layer/multi-thread architecture that optimizes the computational speed and integrates three different types of sensory feedback: vision, tactile and haptic. Such kind of environment is suitable for demanding VR applications such as the online marketing of novel textiles or garments, however it requires the design of a high performance multi-point haptic interface. The present work focuses on the haptic device design and describes how demanding requirements can be met by integrating on a high performance device a force sensor to achieve closed loop control. The methodology for the dimensioning of a motion based explicit force control on the basis of the dynamic parameters will be discussed and the specific implementation for the HAPTEX system presented.     

Improvising through the senses: a performance approach with the indirect use of technology

This article explores and proposes new ways of performing in a technology-mediated environment. We present a case study that examines feedback loop relationships between a dancer and a pianist. Rather than using data from sensor technologies to directly control and affect musical parameters, we captured data from a dancer’s arm movements and mapped them onto a bespoke device that stimulates the pianist’s tactile sense through vibrations. The pianist identifies and interprets the tactile sensory experience, with his improvised performance responding to the changes in haptic information received. Our system presents a new way of technology-mediated performer interaction through tactile feedback channels, enabling the user to establish new creative pathways. We present a classification of vibrotactile interaction as means of communication, and we conclude how users experience multi-point vibrotactile feedback as one holistic experience rather than a collection of discrete feedback points.     

Synergistic visual/haptic rendering modes for scientific visualization

Our approach uses a visual/haptic interface to display scientific data both graphically and haptically. The haptic interface provides a natural means of interacting with the data through direct tactile sensing and manipulation of the data display. Our experience with this interface suggests that users understand the data more clearly when the haptic component acts as a synergistic companion to the visual display. That is, rather than replace the visual display of data or display disparate data haptically, the haptic component reinforces and clarifies visual information via compatible haptic data rendering.     

Effects of Laptop Touchpad Texturing on User Performance

This research assessed user performance with different laptop touchpad textures. In specific, the study measured discrete movement task time and accuracy. It was hypothesized that texturing would increase task times but improve accuracy by providing users with tactile references. A variable representing the frictional potential of pads was introduced into an established model of discrete movement performance (Fitts’ Law) in an attempt to accurately model user performance under experimental task conditions. Results revealed touchpad texturing to degrade task performance. However, accuracy in pointing tasks was not significantly affected. Results also revealed that the expanded form of Fitts’ Law, including a parameter for representing the frictional potential of pad texturing, was more predictive of actual movement times than the original form of the Law. Results from the study increase understanding of the effects of touchpad texture on human motor control behavior and provide some guidance for future pad design.     

Remote Tactile Transmission with Time Delay for Robotic Master–Slave Systems

This study develops a method to compensate for the communication time delay for tactile transmission systems. For transmitting tactile information from remote sites, the communication time delay degrades the validity of feedback. However, so far time delay compensation methods for tactile transmissions have yet to be proposed. For visual or force feedback systems, local models of remote environments were adopted for compensating the communication delay. The local models cancel the perceived time delay in sensory feedback signals by synchronizing them with the users' operating movements. The objectives of this study are to extend the idea of the local model to tactile feedback systems and develop a system that delivers tactile roughness of textures from remote environments to the users of the system. The local model for tactile roughness is designed to reproduce the characteristic cutaneous deformations, including vibratory frequencies and amplitudes, similar to those that occur when a human finger scans rough textures. Physical properties in the local model are updated in real-time by a tactile sensor installed on the slave-side robot. Experiments to deliver the perceived roughness of textures were performed using the developed system. The results showed that the developed system can deliver the perceived roughness of textures. When the communication time delay was simulated, it was confirmed that the developed system eliminated the time delay perceived by the operators. This study concludes that the developed local model is effective for remote tactile transmissions.     

基于虚拟现实触觉感知接口技术的研究与进展

基于虚拟现实触觉感知接口技术（Haptic Interface Technique）涉及的学科 面很宽，包括机器人学、虚拟现实技术、仿生学、传感技术及互联网通讯技术等．触觉感知接口是基于虚拟现实机器人遥操作系统的重要组成部分．从广义角度出发，触觉感知（Haptic Sensing）应包括力/力矩觉（Force/Torque Sensing）和接触觉（Tactile or Touch Sensing）等主要感觉功能．这里将着重介绍触觉感知接口的研究意义与结构、用于触觉感知接口的特种驱动机构．最后，讨论触觉感知接口技术今后的研究与发展趋势．     

The impact of haptic augmentation on middle school students’ conceptions of the animal cell

Of the five sensory channels—sight, sound, taste, smell, and touch, it is only our sense of touch that enables us to modify and manipulate the world around us. This article reports the preliminary findings of a systematic study investigating the efficacy of adding haptic feedback to a desktop virtual reality program for use in middle school science instruction. Current technology allows for the simulation of tactile and kinesthetic sensations via sophisticated haptic devices and a computer interface. This research, conducted with 80 middle school students, examined the cognitive and affective impact of this technology on students’ understandings of the structure and function of an animal cell. The results of this work offer valuable insights into the theoretical and practical considerations involved in the development and implementation of haptically augmented virtual reality instructional programs.     

Ubi-Pen: A Haptic Interface with Texture and Vibrotactile Display

The Ubi-Pen is a pen-like haptic interface incorporating a compact tactile display and a vibrating module. It can represent tactile patterns and provide feedback with the click of a button. It's also applicable to combined force- and tactile-feedback display.     

Slingshot 3D: A synchronous haptic-audio-video game

Recent advances in multimedia and human computer interaction technologies have paved the way for rich contents across multiple media such as haptics, audio, and video. This paper introduces a multimodal game named the Slingshot 3D Game: an interactive and synchronous haptic-audio-video shooter game over the Internet network. The game incorporates two types of haptic feedback: tactile feedback using a haptic jacket and kinesthetic feedback using the Novint Falcon haptic interface. Furthermore, the game utilizes a depth camera to track the player's (upper) body movements and detect collisions between the player's body and the shot projectile. To promote availability and cost, the game uses the Internet network as the communication medium between the players, by utilizing the Admux communication framework [10]. The game design and implementation are detailed in this paper. Both the player performance analysis and the user satisfaction analysis have shown that the incorporation of synchronous haptic-video multimedia has enhanced the perception of player presence and the overall quality of performance.     

Wearable artificial skin layer for the reconstruction of touched geometry by morphological computation

ABSTRACT

This paper proposes a wearable haptic sensor for the reconstruction of the surface geometry of an object to be touched. The proposed haptic sensor is a thin rubber artificial skin layer that is formed around the user's finger or other body part and contains a small embedded strain gauge for measuring large deformations. The sensor can be easily fabricated by rubber dipping. First, it was demonstrated that the proposed sensor is not only able to statically detect the curvature of the touched surface but can also measure deformations due to rapid light tapping with robustness against motion noise. As an illustrative demonstration of morphological computation in haptics, it was then demonstrated that it is possible to reconstruct the geometry of tiny undulations in the surface of the touched object from the information obtained by the proposed sensor.     

Effects of feedback type and modality on motor skill learning and retention

ABSTRACT

Two types of feedback, including knowledge of results (KR) and knowledge of performance (KP) are typically delivered as part of motor training using different sensory modalities. Twenty-four right-handed individuals performed a computer-based psychomotor training task using a contemporary haptic interface. Results revealed that KP was superior to KR for overall task performance retention, whereas KR was good for training specific aspects of performance. A combination of KP and KR through the same modality did not produce an additive positive learning effect for task performance. Auditory and haptic feedback improved retention over visual, but only with the use of multiple information cues. Design guidelines are provided for training systems for motor skill development and/or rehabilitation.     

Surface exploration using laparoscopic surgical instruments: The perception of surface roughness

During laparoscopic surgery video images are used to guide the movements of the hand and instruments, and objects in the operating field often obscure these images. Thus, surgeons often rely heavily on tactile information (sense of touch) to help guide their movements. It is important to understand how tactile perception is affected when using laparoscopic instruments, since many surgical judgements are based on how a tissue ‘feels’ to the surgeon, particularly in situations where visual inputs are degraded. Twelve naïve participants used either their index finger or a laparoscopic instrument to explore sandpaper surfaces of various grits (60, 100, 150 and 220). These movements were generated with either vision or no vision. Participants were asked to estimate the roughness of the surfaces they explored. The normal and tangential forces of either the finger or instrument on the sandpaper surfaces were measured. Results showed that participants were able to judge the roughness of the sandpaper surfaces when using both the finger and the instrument. However, post hoc comparisons showed that perceptual judgements of surface texture were altered in the no vision condition compared to the vision condition. This was also the case when using the instrument, compared to the judgements provided when exploring with the finger. This highlights the importance of the completeness of the video images during laparoscopic surgery. More normal and tangential force was used when exploring the surfaces with the finger as opposed to the instrument. This was probably an attempt to increase the contact area of the fingertip to maximize tactile input. With the instrument, texture was probably sensed through vibrations of the instrument in the hand. Applications of the findings lie in the field of laparoscopic surgery simulation techniques and tactile perception.     

基于混合驱动的柔顺性触觉接口装置

针对目前柔顺性触觉接口设备存在的容易失真的问题,提出了一种基于电机和磁流变液混合驱动的柔顺性触觉接口装置。在简单介绍了磁流变液的基础上,讨论了基于磁流变液被动驱动器的结构和原理,该驱动器采用多转子设计思路以增大力输出范围。利用磁流变液驱动器能够模拟肌体组织的黏滞性,电机能够模拟肌体组织的弹性,将驱动器与电机串联实现肌体组织的柔顺性再现,同时利用电机补偿驱动器非有益阻尼力,增强装置的保真效果。在此基础上设计了柔顺性触觉接口装置,对装置模拟自由空间、不同的变形程度柔顺性物体受力进行了介绍,分析了装置的控制方法,最后加工了触觉装置原型,开展了不同柔顺度虚拟肌体组织柔顺性再现实验,实验结果验证了所设计装置及控制方法的有效性。     

Surface exploration using laparoscopic surgical instruments: the perception of surface roughness

During laparoscopic surgery video images are used to guide the movements of the hand and instruments, and objects in the operating field often obscure these images. Thus, surgeons often rely heavily on tactile information (sense of touch) to help guide their movements. It is important to understand how tactile perception is affected when using laparoscopic instruments, since many surgical judgements are based on how a tissue 'feels' to the surgeon, particularly in situations where visual inputs are degraded. Twelve na?ve participants used either their index finger or a laparoscopic instrument to explore sandpaper surfaces of various grits (60, 100, 150 and 220). These movements were generated with either vision or no vision. Participants were asked to estimate the roughness of the surfaces they explored. The normal and tangential forces of either the finger or instrument on the sandpaper surfaces were measured. Results showed that participants were able to judge the roughness of the sandpaper surfaces when using both the finger and the instrument. However, post hoc comparisons showed that perceptual judgements of surface texture were altered in the no vision condition compared to the vision condition. This was also the case when using the instrument, compared to the judgements provided when exploring with the finger. This highlights the importance of the completeness of the video images during laparoscopic surgery. More normal and tangential force was used when exploring the surfaces with the finger as opposed to the instrument. This was probably an attempt to increase the contact area of the fingertip to maximize tactile input. With the instrument, texture was probably sensed through vibrations of the instrument in the hand. Applications of the findings lie in the field of laparoscopic surgery simulation techniques and tactile perception.     

Presentation of Surface Height Profiles Based on Frequency Modulation at Constant Amplitude Using Vibrotactile Elements

This study attempted to observe what effects the frequency modulation of vibration elements produce in representing a tactile shape. Tactile shapes were modulated based on frequency difference at constant amplitude through a tactile feedback array of 30 (5 × 6) pins, which stimulated the finger pad. Experiment I showed that participants feel height changes when modulating frequency. In Experiment II, the participants were asked to discriminate three basic tactile shape patterns, which were generated with different frequencies at constant amplitude. Experiment II proved that spatial height information can be represented by modulating temporal information. In Experiment III, the frequency modulation method was applied to the tactile mouse system. Dynamic frequency modulation at passive touch can be used to transmit tactile height pattern information to the user of the mouse pointer for more practical application. The results showed that the participants were able to discern eight predefined shapes with an accuracy of 98.4% upon passive touch.