Surface exploration using laparoscopic surgical instruments: the perception of surface roughness

During laparoscopic surgery video images are used to guide the movements of the hand and instruments, and objects in the operating field often obscure these images. Thus, surgeons often rely heavily on tactile information (sense of touch) to help guide their movements. It is important to understand how tactile perception is affected when using laparoscopic instruments, since many surgical judgements are based on how a tissue 'feels' to the surgeon, particularly in situations where visual inputs are degraded. Twelve na?ve participants used either their index finger or a laparoscopic instrument to explore sandpaper surfaces of various grits (60, 100, 150 and 220). These movements were generated with either vision or no vision. Participants were asked to estimate the roughness of the surfaces they explored. The normal and tangential forces of either the finger or instrument on the sandpaper surfaces were measured. Results showed that participants were able to judge the roughness of the sandpaper surfaces when using both the finger and the instrument. However, post hoc comparisons showed that perceptual judgements of surface texture were altered in the no vision condition compared to the vision condition. This was also the case when using the instrument, compared to the judgements provided when exploring with the finger. This highlights the importance of the completeness of the video images during laparoscopic surgery. More normal and tangential force was used when exploring the surfaces with the finger as opposed to the instrument. This was probably an attempt to increase the contact area of the fingertip to maximize tactile input. With the instrument, texture was probably sensed through vibrations of the instrument in the hand. Applications of the findings lie in the field of laparoscopic surgery simulation techniques and tactile perception.     

Surface exploration using laparoscopic surgical instruments: The perception of surface roughness

During laparoscopic surgery video images are used to guide the movements of the hand and instruments, and objects in the operating field often obscure these images. Thus, surgeons often rely heavily on tactile information (sense of touch) to help guide their movements. It is important to understand how tactile perception is affected when using laparoscopic instruments, since many surgical judgements are based on how a tissue ‘feels’ to the surgeon, particularly in situations where visual inputs are degraded. Twelve naïve participants used either their index finger or a laparoscopic instrument to explore sandpaper surfaces of various grits (60, 100, 150 and 220). These movements were generated with either vision or no vision. Participants were asked to estimate the roughness of the surfaces they explored. The normal and tangential forces of either the finger or instrument on the sandpaper surfaces were measured. Results showed that participants were able to judge the roughness of the sandpaper surfaces when using both the finger and the instrument. However, post hoc comparisons showed that perceptual judgements of surface texture were altered in the no vision condition compared to the vision condition. This was also the case when using the instrument, compared to the judgements provided when exploring with the finger. This highlights the importance of the completeness of the video images during laparoscopic surgery. More normal and tangential force was used when exploring the surfaces with the finger as opposed to the instrument. This was probably an attempt to increase the contact area of the fingertip to maximize tactile input. With the instrument, texture was probably sensed through vibrations of the instrument in the hand. Applications of the findings lie in the field of laparoscopic surgery simulation techniques and tactile perception.     

Factors Involved in Tactile Texture Perception Through Probes

An understanding of texture perception by robotic systems can be gained by examining human texture perception through a probe. As with texture perception in direct touch with the finger, texture perception by indirect touch by means of a probe is multidimensional, comprising rough, hard and sticky texture continua. In this study we describe variability among individual subjects in probe-mediated texture perception, and compare similarities and differences of texture perception between direct touch and indirect touch. The results show variability among subjects, as individual subjects may choose to rely on different texture dimensions, and examine the texture surface using different scanning velocities and contact forces. Despite the fact that the subjects use different scanning velocities and forces, they discriminate textures reliably and make subjective magnitude estimates consistently along a texture continuum when indirectly exploring texture surfaces with a probe. These data will be potentially useful in designing robotic sensory systems and understanding of sensory feedback, which is essential to teleoperations.     

Toward quality texture display: vibrotactile stimuli to modify material roughness sensations

For industrial purposes such as product design, texture displays should deliver a quality sense of touch to users. We have developed a vibrotactile texture display that uses real materials such as fabric, wood, and leather to enable the presentation of quality textures to users. By applying two types of vibrotactile stimuli to users’ finger pads through the materials, their fine and macro roughness sensations can be selectively modified while maintaining their original perceptual characteristics. This approach is effective for different types of textures such as paper, wood, leather, and cloth unless they possess strong damping properties that may attenuate the vibratory stimuli applied through them.     

Measurement-based modeling of contact forces and textures for haptic rendering

Haptic texture represents the fine-grained attributes of an object's surface and is related to physical characteristics such as roughness and stiffness. We introduce an interactive and mobile scanning system for the acquisition and synthesis of haptic textures that consists of a visually tracked handheld touch probe. The most novel aspect of our work is an estimation method for the contact stiffness of an object based solely on the acceleration and forces measured during stroking of its surface with the handheld probe. We establish an experimental relationship between the estimated stiffness and the contact stiffness observed during compression. We also measure the height-displacement profile of an object's surface enabling us to generate haptic textures. We show an example of mapping the textures on to a coarse surface mesh obtained with an image-based technique, but the textures may also be combined with coarse surface meshes obtained by manual modeling.     

Effects of vision and friction on haptic perception

OBJECTIVE: Two experiments were conducted to examine the effects of vision and masking friction on contact perception and compliance differentiation thresholds in a simulated tissue-probing task. BACKGROUND: In minimally invasive surgery, the surgeon receives limited haptic feedback because of the current design of the instrumentation and relies on visual feedback to judge the amount of force applied to the tissues. It is suggested that friction forces inherent in the instruments contribute to errors in surgeons' haptic perception. This paper investigated the psychophysics of contact detection and cross-modal sensory processing in the context of minimally invasive surgery. METHOD: A within-subjects repeated measures design was used, with friction, vision, tissue softness, and order of presentation as independent factors, and applied force, detection time, error, and confidence as dependent measures. Eight participants took part in each experiment, with data recorded by a custom force-sensing system. RESULTS: In both detection and differentiation tasks, higher thresholds, longer detection times, and more errors were observed when vision was not available. The effect was more pronounced when haptic feedback was masked by friction forces in the surgical device (p < .05). CONCLUSION: Visual and haptic feedback were equally important for tissue compliance differentiation. APPLICATION: A frictionless endoscopic instrument can be designed to restore critical haptic information to surgeons without having to create haptic feedback artificially.     

Using perceptual relation of regularity and anisotropy in the texture with independent component model for defect detection

This paper addresses the raw textile defect detection problem using independent components approach with insights from human vision system. Human vision system is known to have specialized receptive fields that respond to certain type of input signals. Orientation-selective bar cells and grating cells are examples of receptive fields in the primary visual cortex that are selective to periodic- and aperiodic-patterns, respectively. Regularity and anisotropy are two high-level features of texture perception, and we can say that disruption in regularity and/or orientation field of the texture pattern causes structural defects. In our research, we observed that independent components extracted from texture images give bar or grating cell like results depending on the structure of the texture. For those textures having lower regularity and dominant local anisotropy (orientation or directionality), independent components look similar to bar cells whereas textures with high regularity and lower anisotropy have independent components acting like grating cells. Thus, we will expect different bar or grating cell like independent components to respond to defective and defect-free regions. With this motivation, statistical analysis of the structure of the texture by means of independent components and then extraction of the disturbance in the structure can be a promising approach to understand perception of local disorder of texture in human vision system. In this paper, we will show how to detect regions of structural defects in raw textile data that have certain regularity and local orientation characteristics with the application of independent component analysis (ICA), and we will present results on real textile images with detailed discussions.     

A Wavelet-Based Approach for Analyzing Industrial Stochastic Textures With Applications

Several continuous manufacturing processes use stochastic texture images for quality control and monitoring. Large amounts of pictorial data are acquired, providing important information about both the materials produced and the manufacturing processes involved. However, it is often difficult to measure objectively the similarity among industrial stochastic images or to discriminate between texture images of stochastic materials with distinct properties. Nowadays, the degree of discrimination required by industrial processes often goes beyond the limits of human visual perception. This paper proposes to model this specific class of textures as colored noise and presents a new approach for multiresolution stochastic texture representation and discrimination in industry (e.g., nonwoven textiles and paper). The wavelet transform is used to represent stochastic texture images in multiple resolutions and to describe them using local orientation and density variability as features. Based on this representation, a multiresolution distance measure for stochastic textures is proposed, and industrial applications of the method and experimental results are reported. The conclusions include ideas for future work     

A compact planar distributed tactile display and effects of frequency on texture judgment

This paper describes the development of a planar distributed tactile display and the evaluation of the results of its effectiveness for displaying textures. The tactile display is composed of a 6 × 5 pin array actuated by 30 piezoelectric bimorphs. The distance between each pin's centers is 1.8 mm. Vertical excursion of each pin is controlled over a 0–0.7 mm range. Perceptual experiments were conducted to evaluate the performance of the system under three conditions: active touch, passive touch with vibration and passive touch without vibration. The experimental results showed that vibrational stimuli helped subjects discriminate tactile patterns. Measurements of the error rate during discrimination tasks were used to find an optimal vibration frequency for stimuli presented at a constant sensation level (32 SLdB above threshold). The experiment was repeated, this time holding the energy transferred mechanically to the fingertip tissue constant. At low frequencies, we found that the passive stimulation allowed subjects to discriminate just as well as active touch of static stimuli did. The results suggested new possibilities for displaying texture using passive touch, constant energy and spatially varied vibration frequency.     

Texture models and image measures for texture discrimination

The task of texture segmentation is to identify image curves that separate different textures. To segment textured images, one must first be able to discriminate textures. A segmentation algorithm performs texture-discrimination tests at densely spaced image positions, then interprets the results to localize edges. This article focuses on the first stage, texture discrimination.We distinguish between perceptual and physical texture differences: the former differences are those perceived by humans, while the latter, on which we concentrate, are those defined by differences in the processes that create the texture in the scene. Physical texture discrimination requires computing image texture measures that allow the inference of physical differences in texture processes, which in turn requires modeling texture in the scene. We use a simple texture model that describes textures by distributions of shape, position, and color of substructures. From this model, a set of image texture measures is derived that allows reliable texture discrimination. These measures are distributions of overall substructure length, width, and orientation; edge length and orientation; and differences in averaged color. Distributions are estimated without explicitly isolating image substructures. Tests of statistical significance are used to compare texture measures.A forced-choice method for evaluating texture measures is described. The proposed measures provide empirical discrimination accuracy of 84 to 100% on a large set of natural textures. By comparison, Laws' texture measures provide less than 50% accuracy when used with the same texture-edge detector. Finally, the measures can distinguish textures differing in second-order statistics, although those statistics are not explicitly measured.The author was with the Robotics Laboratory, Computer Science Department, Stanford University, Stanford, California 94305. He is now with the Institut National de Recherche en Informatique et en Automatique (INRIA), Sophia-Antipolis, 2004 Route des Lucioles, 06565 Valbonne Cedex, France.     

Reconstruction of surface texture based on spatial information measured with a pen-type texture sensor

Surface texture is one of the important properties for the human to identify objects by touch. Effective reconstructions of textures are necessary for realistic interactions between the human and environment via human–computer interfaces. This paper presents a systematic approach for sensing and reconstructing periodic surface textures. Three significant issues are discussed: a pen-type texture sensor that measures the spatial information based on the measurements of contact forces; an algorithm for the reconstruction of periodic textures based on the obtained spatial information; and the method of incremental scanning to identify the polar spectrum of a surface by limited number of scans. The concept of polar spectrum is introduced to describe the spatial properties of the surface, that is, the relation between spatial frequencies and the direction of measurement. The pattern of polar spectrum is used to facilitate surface reconstructions. Experimental results based on the spatial information obtained with a laser displacement sensor and the pen-type texture sensor demonstrate the effectiveness of the proposed methods for the measurement and reconstruction of periodic textures.     

织物纹理的简单视神经细胞感受野的选择特性

研究了织物纹理的简单视神经细胞感受野的选择特性。分别选择具有明显方向性和周期性以及周期性和方向性不显著的两类代表织物纹理为研究对象,采用基于独立分量分析的视觉模型估计出简单视觉细胞的感受野。通过对感受野兴奋区域重心、方向角和面积的分析,以有效表示感受野在位置、方向和空间频率的选择特性。基于织物纹理与自然纹理两类纹理的简单细胞感受野选择性比较实验显示,相比于自然纹理,织物纹理的简单细胞感受野单元中的兴奋区域在方向、位置和空间频率上具有更加显著的选择特性。实验结果表明,提出的简单视觉细胞感受野选择特性研究方法在一定程度上揭示了初级视皮层对织物纹理图像处理的神经机制,从视觉感知层面反映了织物纹理的视觉特性。     

振动触觉纹理再现技术研究综述

纹理力触觉再现是通过特定的硬件装置模拟产生与物体纹理表面接触时的触感，使用户能感受到物体的粗糙度、软硬度等纹理特征信息。振动刺激作为再现物体触觉信息的一种刺激方式，在纹理触觉再现中被广泛运用，产生了不同的振动触觉表达装置和纹理触觉表达方法。从纹理触觉认知的角度，阐述了人对振动刺激的触觉感知生理学基础；介绍了纹理触觉再现的原理和方法；从振动与纹理特征的映射方法以及振动刺激方式两个方面分析了目前振动触觉纹理再现技术的发展现状；最后对相关研究的发展进行了总结展望。     

Stereometric pattern recognition by artificial touch

An artificial touch perception system has been created in order to study a method of processing information obtainable through tactile exploration of three-dimensional forms. The results can be useful for several purposes.The first part of the work concerns the project of the tactile explorator. For this purpose we used a kind of artificial limb like a finger with a certain number of touch sensitive transducers distributed along the surface of the finger tip. The information received by touching the object with the finger, is successively utilized as the input of the control servosystem which moves the finger point-by-point along the object surface in order to proceed with the exploration. It must be noticed that, from a philosophical point-of-view, the parallel approach with more fingers touching simultaneously the object in several points, is equal to sequential touching of these points by one moving finger.The second part describes the use of the propositional calculus in logical classification of the objects, as a method of three-dimensional pattern recognition.Elaboration of the input data obtained by tactile exploration, and computation of characteristic geometrical features of three-dimensional forms, has been performed on computer.     

A comparison of forearm and thumb muscle electromyographic responses to the use of laparoscopic instruments with either a finger grasp or a palm grasp

Laparoscopic techniques allow for less-invasive treatment of common surgical problems. Laparoscopic instruments are different from standard surgical instruments and generally incorporate a pistol-grip handle configuration with rings for the fingers. This handle configuration has been reported as being uncomfortable, leading to finger compression neuropathies in some cases. As an alternative, the surgeon can choose to grasp laparoscopic instruments using a more powerful palm grip during grasping motions. This study evaluates the hypothesis that the use of the palm grip requires less muscle tension than the finger-grip when grasping with laparoscopic instruments. Nine general surgeons used an Autosuture laparoscopic grasper with a ringed pistol-grip handle held in both a finger-in-ring (F) or palm (P) hand grip position to grasp and close two spring-loaded metal plates. The same task was performed with a surgical haemostat clamp (H) for comparison. Each subject performed the grasping task in a random sequence for the three instrument configurations at two grasping forces levels (0.7 and 4.2 N), and with the instrument at three angles to the subjects' sagittal plane (0 degree, 45 degrees and 90 degrees). Surface electromyographic (EMG) signals were acquired from the flexor carpi ulnaris (FCU), flexor digitorum profundus (FDP), flexor digitorum superficialis (FDS), extensor carpi ulnaris (ECU), extensor digitorum comunis (EDC) and the thenar compartment (TH). The peak root mean squared (RMS) EMG voltage was averaged for five repetitions at each instrument, force and angle condition. Statistical analysis was carried out by repeated measures ANOVA. The muscle EMG RMS amplitude while using the palm grip was decreased in the FDS, TH and EDC, was unchanged in the ECU and FCU, and was slightly higher in the FDP when compared with the finger grip. These differences were most prominent at 90 degrees to the sagittal plane where the subjects' wrists neared maximal flexion. It is concluded that the palm grip is more powerful than the finger grip when grasping with laparoscopic instruments, particularly at angles perpendicular to the surgeon's sagittal plane.     

Electrostatic tactile display with thin film slider and its application to tactile telepresentation systems

A new electrostatic tactile display is proposed to realize compact tactile display devices that can be incorporated with virtual reality systems. The tactile display of this study consists of a thin conductive film slider with stator electrodes that excite electrostatic forces. Users of the device experience tactile texture sensations by moving the slider with their fingers. The display operates by applying two-phase cyclic voltage patterns to the electrodes. The display is incorporated into a tactile telepresentation system to realize explorations of remote surface textures with real-time tactile feedback. In the system, a PVDF tactile sensor and a DSP controller automatically generate voltage patterns to present surface texture sensations through the tactile display. A sensor, in synchronization with finger motion on the tactile display, scans a texture sample and outputs information about the sample surface. The information is processed by a DSP and fed back to the tactile display in real time. The tactile telepresentation system was evaluated in texture discrimination tests and demonstrated a 79 percent correct answer ratio. A transparent electrostatic tactile display is also reported in which the tactile display is combined with an LCD to realize a visual-tactile integrated display system.     

Two-finger stiffness discrimination of a virtual object with haptic sensation enhancement via the stochastic resonance effect

We investigated the stiffness discrimination ability of two fingers with stochastic resonance. Haptic perception at the fingertip is known to improve when vibrotactile noise propagates to the fingertip, and this phenomenon is called stochastic resonance. The improvement of a fingertip’s haptic sensation depends on the intensity of the noise propagating to the fingertip. Therefore, to improve the haptic sensation at multiple fingertips, we do not need to attach noise sources, such as vibrators, to multiple fingertips. Even if only one vibrator is used, the haptic sensation at multiple fingertips may be improved by propagating noise to the fingertips. In this study, we focus on stiffness discrimination as a task using multiple fingers, in which the thumb and index finger are used to touch a virtual object and perceive its stiffness. Subsequently, we demonstrate that the stiffness perception is improved by propagating sufficiently intense noise to the thumb and index finger using a vibrator. Furthermore, this study considers the same task in a real-world situation to investigate the relationship between the stochastic resonance effect and the stiffness discrimination task. The findings indicate the possibility of improving the haptic sensation at multiple fingertips using one vibrator.     

Phantom-based haptic interaction with virtual objects

In 1993, haptic interaction with computers took a significant step forward with the development of the Phantom haptic interface. This simple device has spawned a new field analogous to computer graphics-computer haptics-defined as the discipline concerned with the techniques and processes associated with generating and displaying synthesized haptic stimuli to the human user. Inspired by the authors' previous work in interpreting robot touch sensor information and study of human touch perception, the Phantom interface permits users to feel the forces of interaction they would encounter while touching objects with the end of a stylus or the tip of their finger. The resulting sensations prove startling, and many first-time users are quite surprised at the compelling sense of physical presence they encounter when touching virtual objects. To appreciate why the Phantom system succeeded where others failed, you need to understand the nature and functioning of the human haptic system.     

Induction operators for a computational colour–texture representation

The aim of this paper is to outline a perceptual approach to a computational colour–texture representation based on some colour induction phenomena. The extension of classical grey level methods for texture processing to the RGB channels of the corresponding colour texture is not the best solution to simulate human perception. Chromatic induction mechanisms of the human visual system, that has been widely studied in psychophysics, play an important role when looking at scenes where the spatial frequency is high as it occurs on texture images. Besides others, chromatic induction includes two complementary effects: chromatic assimilation and chromatic contrast. While the former has been measured by Wandell and Zhang [A spatial extension of CIELAB for digital colour image reproduction, in: SID, 1996] and extended to computer vision by Petrou et al. [Perceptual smoothing and segmentation of colour textures, in: 5th European Conference on Computer Vision, Freiburg, Germany, 1998, pp. 623] as a perceptual blurring, some aspects on the last one still remain to be measured, but it has to be a computational operator that simulates the contrast induction phenomenon performing a perceptual sharpening that preserves the structural properties of the texture. Applying both, the perceptual sharpening and the perceptual blurring, we propose to build a tower of images as an induction front-end that can be the basis of a perceptual representation of colour–textures.