SmartTouch: electric skin to touch the untouchable

Augmented haptics lets users touch surface information of any modality. SmartTouch uses optical sensors to gather information and electrical stimulation to translate it into tactile display. Augmented reality is an engineer's approach to this dream. In AR, sensors capture artificial information from the world, and existing sensing channels display it. Hence, we virtually acquire the sensor's physical ability as our own. Augmented haptics, the result of applying AR to haptics, would allow a person to touch the untouchable. Our system, SmartTouch, uses a tactile display and a sensor. When the sensor contacts an object, an electrical stimulation translates the acquired information into a tactile sensation, such as a vibration or pressure, through the tactile display. Thus, an individual not only makes physical contact with an object, but also touches the surface information of any modality, even those that are typically untouchable.     

Development of a sensor system for collecting tactile information

This paper presents the development of a sensor system for collecting tactile information. An active sensing system using the piezoelectric effect and the pyroelectric effect of a PVDF (Polyvinylidene fluoride) film is proposed. The active sensing is designed with human motions for tactile perception in mind. First, as the pretest, the distinction examination of six fabrics with different textures is carried out through human tactile perception. Next, the proposed sensor system is assembled. The sensor is composed of a PVDF film and a soft rubber. The surface of the sensor can be heated through temperature control. The sensor is attached on the tip of a robot finger driven by a piezoelectric bimorph strip and the root of the finger is mounted on a linear slider. Two kinds of active sensing are introduced. First, the heated sensor is contacted with an object and pyroelectric output signals are collected in order to obtain the information on tactile warmth. Next, the heated sensor is slid over the object and piezoelectric output signals are collected in order to obtain the information on feelings of vibration. Through the discussion about each sensing, three indexes representing features of the collected data are extracted and proposed as the sensor outputs for the evaluation of tactile sensation. The measurement using the sensor system is done on the samples used in the distinction examination. Comparison with the results shows that the sensor system extracts features on feelings of vibration and warmth.     

MTDriver:一种改进的多点触摸桌面工具包

目前多点触摸桌面广泛采用计算机视觉技术实现.触摸信息是通过手指反射的红外线在红外摄像机下成像,然后对红外相机得到灰度图像进行手指区域提取,跟踪,校正得到.由于桌面表面红外光照射不均衡,环境噪声干扰等因素,目前存在的多点触摸工具包在检测和跟踪方面效果较差,也没有考虑手指运动和摄像机畸变的影响.本文提出基于图像局部极值点检测手指触摸区域,结合手指运动和相邻帧信息进行手指跟踪,实现摄像机畸变校正的多点触摸桌面系统工具包MTDriver.实验结果表明,MTDriver 跟踪识别准确,效率高,鲁棒性强,具有实用性.     

An intelligent tactile sensor-an on-line hierarchical object and seam analyzer

This paper describes an on-line computer supported tactile sensor that can recognize complex industrial objects, identify a seam location, and track a three-dimensional seam trajectory for an arc welding robot. A one finger tactile sensor with two degrees of freedom is used. The supporting microcomputer system organizes and reduces the useful sensor data from digitized tactile information to a compact symbolic representation which is matched to a knowledge network. A beam search algorithm is used. The hierarchical organization provides that simple features are first detected on the object, and more complex features are examined later. Pruning of the search tree is based on a likelihood function.     

Surface exploration using laparoscopic surgical instruments: The perception of surface roughness

During laparoscopic surgery video images are used to guide the movements of the hand and instruments, and objects in the operating field often obscure these images. Thus, surgeons often rely heavily on tactile information (sense of touch) to help guide their movements. It is important to understand how tactile perception is affected when using laparoscopic instruments, since many surgical judgements are based on how a tissue ‘feels’ to the surgeon, particularly in situations where visual inputs are degraded. Twelve naïve participants used either their index finger or a laparoscopic instrument to explore sandpaper surfaces of various grits (60, 100, 150 and 220). These movements were generated with either vision or no vision. Participants were asked to estimate the roughness of the surfaces they explored. The normal and tangential forces of either the finger or instrument on the sandpaper surfaces were measured. Results showed that participants were able to judge the roughness of the sandpaper surfaces when using both the finger and the instrument. However, post hoc comparisons showed that perceptual judgements of surface texture were altered in the no vision condition compared to the vision condition. This was also the case when using the instrument, compared to the judgements provided when exploring with the finger. This highlights the importance of the completeness of the video images during laparoscopic surgery. More normal and tangential force was used when exploring the surfaces with the finger as opposed to the instrument. This was probably an attempt to increase the contact area of the fingertip to maximize tactile input. With the instrument, texture was probably sensed through vibrations of the instrument in the hand. Applications of the findings lie in the field of laparoscopic surgery simulation techniques and tactile perception.     

Surface exploration using laparoscopic surgical instruments: the perception of surface roughness

During laparoscopic surgery video images are used to guide the movements of the hand and instruments, and objects in the operating field often obscure these images. Thus, surgeons often rely heavily on tactile information (sense of touch) to help guide their movements. It is important to understand how tactile perception is affected when using laparoscopic instruments, since many surgical judgements are based on how a tissue 'feels' to the surgeon, particularly in situations where visual inputs are degraded. Twelve na?ve participants used either their index finger or a laparoscopic instrument to explore sandpaper surfaces of various grits (60, 100, 150 and 220). These movements were generated with either vision or no vision. Participants were asked to estimate the roughness of the surfaces they explored. The normal and tangential forces of either the finger or instrument on the sandpaper surfaces were measured. Results showed that participants were able to judge the roughness of the sandpaper surfaces when using both the finger and the instrument. However, post hoc comparisons showed that perceptual judgements of surface texture were altered in the no vision condition compared to the vision condition. This was also the case when using the instrument, compared to the judgements provided when exploring with the finger. This highlights the importance of the completeness of the video images during laparoscopic surgery. More normal and tangential force was used when exploring the surfaces with the finger as opposed to the instrument. This was probably an attempt to increase the contact area of the fingertip to maximize tactile input. With the instrument, texture was probably sensed through vibrations of the instrument in the hand. Applications of the findings lie in the field of laparoscopic surgery simulation techniques and tactile perception.     

A pattern classification by dynamic tactile sense information processing

This paper describes pattern classification with an artificial tactile sense. In this method, an object's shape is determined by touching, groping and grasping it with an artificial hand with tactile sense.

A simplified experiment classifying cylinders and square pillars was performed by an artificial hand with on-off switches instead of pressure sensitive elements. Highly reliable results were obtained. In addition, results of a surface groping experiment are given.     

Presentation of Surface Height Profiles Based on Frequency Modulation at Constant Amplitude Using Vibrotactile Elements

This study attempted to observe what effects the frequency modulation of vibration elements produce in representing a tactile shape. Tactile shapes were modulated based on frequency difference at constant amplitude through a tactile feedback array of 30 (5 × 6) pins, which stimulated the finger pad. Experiment I showed that participants feel height changes when modulating frequency. In Experiment II, the participants were asked to discriminate three basic tactile shape patterns, which were generated with different frequencies at constant amplitude. Experiment II proved that spatial height information can be represented by modulating temporal information. In Experiment III, the frequency modulation method was applied to the tactile mouse system. Dynamic frequency modulation at passive touch can be used to transmit tactile height pattern information to the user of the mouse pointer for more practical application. The results showed that the participants were able to discern eight predefined shapes with an accuracy of 98.4% upon passive touch.     

Human shape recognition performance for 3D tactile display

The paper describes the relationship between the pin-matrix density of a tactile display and the recognition performance of displayed 3D shapes. Three types of pin-matrix tactile display, that generate 3D shapes, were used for the experiment. The pitch of pins was 2 mm, 3 mm, 5 mm each. We assumed that surfaces, edges, and vertices were primitive 3D shape information, so tested shapes were classified into these three categories. We assumed two types of finger touching mode: 1) fingertip-only, allowed full use of spatial shape information given to the fingertip; and 2) allowed tracing of the object. Recognition time and the classified error rate were measured. We obtained results on the relationship between pin pitch and recognition performance data. Regression curves for pin pitch and recognition time were plotted. A significance test of recognition time versus pin pitch was done. The error rate of identification versus pin pitch was described. Our results provide basic knowledge for developing tactile presentation devices     

振动触觉纹理再现技术研究综述

纹理力触觉再现是通过特定的硬件装置模拟产生与物体纹理表面接触时的触感，使用户能感受到物体的粗糙度、软硬度等纹理特征信息。振动刺激作为再现物体触觉信息的一种刺激方式，在纹理触觉再现中被广泛运用，产生了不同的振动触觉表达装置和纹理触觉表达方法。从纹理触觉认知的角度，阐述了人对振动刺激的触觉感知生理学基础；介绍了纹理触觉再现的原理和方法；从振动与纹理特征的映射方法以及振动刺激方式两个方面分析了目前振动触觉纹理再现技术的发展现状；最后对相关研究的发展进行了总结展望。     

Extracting data from human manipulation of objects towards improving autonomous robotic grasping

Humans excel in manipulation tasks, a basic skill for our survival and a key feature in our manmade world of artefacts and devices. In this work, we study how humans manipulate simple daily objects, and construct a probabilistic representation model for the tasks and objects useful for autonomous grasping and manipulation by robotic hands. Human demonstrations of predefined object manipulation tasks are recorded from both the human hand and object points of view. The multimodal data acquisition system records human gaze, hand and fingers 6D pose, finger flexure, tactile forces distributed on the inside of the hand, colour images and stereo depth map, and also object 6D pose and object tactile forces using instrumented objects. From the acquired data, relevant features are detected concerning motion patterns, tactile forces and hand-object states. This will enable modelling a class of tasks from sets of repeated demonstrations of the same task, so that a generalised probabilistic representation is derived to be used for task planning in artificial systems. An object centred probabilistic volumetric model is proposed to fuse the multimodal data and map contact regions, gaze, and tactile forces during stable grasps. This model is refined by segmenting the volume into components approximated by superquadrics, and overlaying the contact points used taking into account the task context. Results show that the features extracted are sufficient to distinguish key patterns that characterise each stage of the manipulation tasks, ranging from simple object displacement, where the same grasp is employed during manipulation (homogeneous manipulation) to more complex interactions such as object reorientation, fine positioning, and sequential in-hand rotation (dexterous manipulation). The framework presented retains the relevant data from human demonstrations, concerning both the manipulation and object characteristics, to be used by future grasp planning in artificial systems performing autonomous grasping.     

Computation of Independent Contact Regions for Grasping 3-D Objects

Precision grasp synthesis has received a lot of attention in past few last years. However, real mechanical hands can hardly assure that the fingers will precisely touch the object at the computed contact points. The concept of independent contact regions (ICRs) was introduced to provide robustness to finger positioning errors during an object grasping: A finger contact anywhere inside each of these regions assures a force-closure grasp, despite the exact contact position. This paper presents an efficient algorithm to compute ICRs with any number of frictionless or frictional contacts on the surface of any 3-D object. The proposed approach generates the independent regions by growing them around the contact points of a given starting grasp. A two-phase approach is provided to find a locally optimal force-closure grasp that serves as the starting grasp, considering as grasp quality measure the largest perturbation wrench that the grasp can resist, independently of the perturbation direction. The proposed method can also be applied to compute ICRs when several contacts are fixed beforehand. The approach has been implemented, and application examples are included to illustrate its performance.      

Three-Dimensional Robotic Vision Using Ultrasonic Sensors

This article describes a three-dimensional artificial vision system for robotic applications using an ultrasonic sensor array. The array is placed on the robot grip so that it is possible to detect the presence of an object, to direct the robot tool towards it, and to locate the object position. It will provide visual information about the object's surface by means of superficial scanning and it permits the object shape reconstruction. The developed system uses an approximation of the ultrasonic radiation and reception beam shape for calculating the first contact points with the object's surface. On the other hand, the position of the array's sensors has been selected in order to provide the sensorial head with other useful capabilities, such as edge detection and edge tracking. Furthermore, the article shows the structure of the sensorial head for avoiding successive rebounds between the head and the object surface, and for eliminating the mechanical vibrations among sensors.     

Providing force feedback in virtual environments

An integral part of successfully manipulating objects is the sensation of touch or force. Experiments with telerobots (robots controlled at a distance) show that the sensation of force and contact improves the efficiency and accuracy of such tasks. Many believe that the same can be said of tasks in a virtual environment. Unfortunately, it is not possible to actually grasp a virtual object in the same manner as you would a real object because virtual objects are defined in the computer, while the user exists in the real world. Thus, there must be some intermediate device that provides the user with the effects of touch, either through the virtual environment itself or in a physical model of the object, which then communicates information to the virtual environment and displays the virtual object to the user. At the University of Tokyo, we are experimenting with surface display, a method that allows users to directly manipulate an object in a virtual environment by touching a physical model of the object's surface as presented by an intermediate device outside the computer. We have created a prototype of such a device that measures the force exerted on the surface. We have also implemented control and calculation methods, and have evaluated both the device and the methods in experiments with users. These experiments have validated the concepts underlying our work, and we are continuing to investigate implementation issues     

Exploration of local surface geometry with minimum number of contact points and surface normal information

In this paper, we propose a method of exploring the surface geometry of an unknown object by touch. The method is based on the idea that a three-dimensional surface geometry can be reconstructed from two principal curvatures of the object which are estimated from three concurrent curves. First, the process to minimize the number of contact points is addressed for the approximation of an arbitrary curve, which uses normal vectors at the contact points. Then, an algorithm for reconstructing a three-dimensional local surface from four contact points, two of which can be used to compute a normal curvature, is presented. Lastly, our method is applied to cylindrical, spherical and planar objects in simulation and experiments for validation.     

An automated tactile sensing strategy for planar object recognitionand localization

The planning problem associated with tactile exploration for object recognition and localization is addressed. Given that an object has been sensed and is one of a number of modeled objects, and given that the data obtained so far are insufficient for recognition and/or localization, the methods developed determin the paths along which a point contact sensor must be directed in order to obtain further highly diagnostic measurements. Three families of sensor paths are found. The first is the family of paths for which recognition and localization are guaranteed. The second guarantees only that something will be learned. The third represents paths to avoid because nothing new will be learned. The methods are based on a small but powerful set of geometric ideas and are developed for two-dimensional, planar-faced objects. They are conceptually easily generalized to handle three-dimensional objects, including objects with through holes     

一种虚拟环境温度触觉再现仿真系统设计

分析了温度触觉感知特性,提出了手指和物体接触时各自表面的温度变化过程,结合单热源温度触觉再现装置设计了一种基于三维力反馈手控器的虚拟环境温度触觉再现仿真系统。手控器可控制虚拟手的动作,对虚拟物体进行温度和力的感知。仿真系统中能再现手指与不同热属性物体接触时的温度变化和力变化,且视觉效果逼真,体现了温度触觉再现的实时性和有效性。     

Artificial Whiskers Suitable for Array Implementation: Accounting for Lateral Slip and Surface Friction

The exquisite tactile sensing ability of biological whiskers has recently led to increasing interest in constructing robotic versions with similar capabilities. Tactile extraction of three-dimensional (3-D) object shape poses several unique challenges that have only begun to be addressed. The present study develops a method for estimating the contact location of a robotic whisker rotating against an object based on small changes in moment at the whisker base. Importantly, the method accounts for lateral slip as well as surface friction, making it particularly well suited for implementation on an array of robotic whiskers. Array implementation would permit simultaneous extraction of multiple contact points and enable highly parallel, efficient 3-D object feature extraction. A simple, scalable array design is suggested to fulfill this approach.      

一种基于阵列式触觉传感器的主动触觉搜索方法及仿真

本文在一种二指节手指模型的基础上，提出了一种新的主动触觉搜索策略和方法，综合利用阵列式触觉传感铭的输出图象来不断改变机械手的姿态，从而依次有序地搜索到三维物体的特征点，并据此生成物体的三维图形。此外还讨论了上述方法的计算机仿真实现。     

Multi-oriented touching text character segmentation in graphical documents using dynamic programming

The touching character segmentation problem becomes complex when touching strings are multi-oriented. Moreover in graphical documents sometimes characters in a single-touching string have different orientations. Segmentation of such complex touching is more challenging. In this paper, we present a scheme towards the segmentation of English multi-oriented touching strings into individual characters. When two or more characters touch, they generate a big cavity region in the background portion. Based on the convex hull information, at first, we use this background information to find some initial points for segmentation of a touching string into possible primitives (a primitive consists of a single character or part of a character). Next, the primitives are merged to get optimum segmentation. A dynamic programming algorithm is applied for this purpose using the total likelihood of characters as the objective function. A SVM classifier is used to find the likelihood of a character. To consider multi-oriented touching strings the features used in the SVM are invariant to character orientation. Experiments were performed in different databases of real and synthetic touching characters and the results show that the method is efficient in segmenting touching characters of arbitrary orientations and sizes.