Recent progress in tactile sensing and sensors for robotic manipulation: can we turn tactile sensing into vision?1

This paper surveys recently published literature on tactile sensing in robotic manipulation to understand effective strategies for using tactile sensing and the issues involved in tactile sensing. It consists of a brief review of existing tactile sensors for robotic grippers and hands, review of modalities available from tactile sensing, review of the applications of tactile sensing in robotic manipulations, and discussion of the issues of tactile sensing and an approach to make tactile sensors more useful. We emphasize vision-based tactile sensing because of its potential to be a good tactile sensor for robots.     

Development of cutaneo-motor coordination in an autonomous robotic system

The capability of autonomously discovering relations between perceptual data and motor actions is crucial for the development of robust adaptive robotic systems intended to operate in an unknown environment. In the case of robotic tactile perception, a proper interaction between contact sensing and motor control is the basic step toward the execution of complex motor procedures such as grasping and manipulation.In this paper the autonomous development of cutaneo-motor coordination is investigated in the case of a robotic finger mounted on a robotic manipulator, for a particular class of micromovements. A neural network architecture linking changes in the sensed tactile pattern with the motor actions performed is described and experimental results are analyzed. Examples of application of the developed sensory-motor coordination in the generation of motor control procedures for the estimate of surface curvature are considered.     

Robust object manipulation for tactile-based blind grasping

Tactile-based blind grasping addresses realistic robotic grasping in which the hand only has access to proprioceptive and tactile sensors. The robotic hand has no prior knowledge of the object/grasp properties, such as object weight, inertia, and shape. There exists no manipulation controller that rigorously guarantees object manipulation in such a setting. Here, a robust control law is proposed for object manipulation in tactile-based blind grasping. The analysis ensures semi-global asymptotic and exponential stability in the presence of model uncertainties and external disturbances that are neglected in related work. Simulation and hardware results validate the effectiveness of the proposed approach.     

Minimal representation multisensor fusion using differentialevolution

Fusion of information from multiple sensors is required for planning and control of robotic systems in complex environments. The minimal representation approach is based on an information measure as a universal yardstick for fusion and provides a framework for integrating information from a variety of sources. In this paper, we describe the principles of minimal representation multisensor fusion and evaluate a differential evolution approach to the search for solutions. Experiments in robot manipulation using both tactile and visual sensing demonstrate that this algorithm is effective in finding useful and practical solutions to this problem for real systems. Comparison of this differential evolution algorithm with more traditional genetic algorithms shows distinct advantages in both accuracy and efficiency     

Reliable non-prehensile door opening through the combination of vision, tactile and force feedback

Whereas vision and force feedback—either at the wrist or at the joint level—for robotic manipulation purposes has received considerable attention in the literature, the benefits that tactile sensors can provide when combined with vision and force have been rarely explored. In fact, there are some situations in which vision and force feedback cannot guarantee robust manipulation. Vision is frequently subject to calibration errors, occlusions and outliers, whereas force feedback can only provide useful information on those directions that are constrained by the environment. In tasks where the visual feedback contains errors, and the contact configuration does not constrain all the Cartesian degrees of freedom, vision and force sensors are not sufficient to guarantee a successful execution. Many of the tasks performed in our daily life that do not require a firm grasp belong to this category. Therefore, it is important to develop strategies for robustly dealing with these situations. In this article, a new framework for combining tactile information with vision and force feedback is proposed and validated with the task of opening a sliding door. Results show how the vision-tactile-force approach outperforms vision-force and force-alone, in the sense that it allows to correct the vision errors at the same time that a suitable contact configuration is guaranteed.     

Comparison of contact sensor localization abilities during manipulation

This paper presents an experimental comparison of tactile array versus force-torque sensing for localizing contact during manipulation. The manipulation tasks involved rotating objects and translating objects using a planar two fingered manipulator. A pin and a box were selected as limiting cases of point and line contact against a cylindrical robot finger tip. Practical implementations of the two sensor types are compared theoretically and experimentally, and three different localization algorithms for the tactile array sensor are considered. Force-torque contact sensing results suffered from difficulties in calibration, transient forces, and low grasp force. Tactile array sensing was immune to these problems and the effect of shear loading was only noticeable for a simple centroid algorithm. The results show that with care, both of these sensing schemes can determine the contact location within a millimeter during real manipulation tasks.     

An active sensing strategy for contact location without tactile sensors using robot geometry and kinematics

In this study, we develop new techniques to sense contact locations and control robots in contact situations in order to enable articulated robotic systems to perform manipulations and grasping actions. Active sensing approaches are investigated by utilizing robot kinematics and geometry to improve upon existing sensing methods for contact. Compliant motion control is used so that a robot can actively search for and localize the desired contact location. Robot control in a contact situation is improved by the precise estimation of the contact location. From this viewpoint, we investigate a new control strategy to accommodate the proposed sensing techniques in contact situations. The proposed estimation algorithm and the control strategy both work complementarily. Then, we verify the proposed algorithm through experiments using 7-DOF hardware and a simulation environment. The two major contributions of the proposed active sensing strategy are the estimation algorithm for contact location without any tactile sensors, and the control strategy complementing the proposed estimation algorithm.     

Extracting data from human manipulation of objects towards improving autonomous robotic grasping

Humans excel in manipulation tasks, a basic skill for our survival and a key feature in our manmade world of artefacts and devices. In this work, we study how humans manipulate simple daily objects, and construct a probabilistic representation model for the tasks and objects useful for autonomous grasping and manipulation by robotic hands. Human demonstrations of predefined object manipulation tasks are recorded from both the human hand and object points of view. The multimodal data acquisition system records human gaze, hand and fingers 6D pose, finger flexure, tactile forces distributed on the inside of the hand, colour images and stereo depth map, and also object 6D pose and object tactile forces using instrumented objects. From the acquired data, relevant features are detected concerning motion patterns, tactile forces and hand-object states. This will enable modelling a class of tasks from sets of repeated demonstrations of the same task, so that a generalised probabilistic representation is derived to be used for task planning in artificial systems. An object centred probabilistic volumetric model is proposed to fuse the multimodal data and map contact regions, gaze, and tactile forces during stable grasps. This model is refined by segmenting the volume into components approximated by superquadrics, and overlaying the contact points used taking into account the task context. Results show that the features extracted are sufficient to distinguish key patterns that characterise each stage of the manipulation tasks, ranging from simple object displacement, where the same grasp is employed during manipulation (homogeneous manipulation) to more complex interactions such as object reorientation, fine positioning, and sequential in-hand rotation (dexterous manipulation). The framework presented retains the relevant data from human demonstrations, concerning both the manipulation and object characteristics, to be used by future grasp planning in artificial systems performing autonomous grasping.     

Whole-finger manipulation with a two-fingered robot hand

This paper describes a method for whole-finger rolling manipulation using a two-fingered robot hand. 'Whole-finger' refers to the use of the complete phalangeal surface during the manipulation. An example of whole-finger manipulation by the human hand is the rolling of a pen between two fingers. The proposed method is based on a two-dimensional model for modelling an object manipulation and is derived from a study of the movement of the contact line between both fingers. Also, the method uses tactile sensor information to estimate the contact point position together with the local curvature of the object. This whole-finger dexterous manipulation is demonstrated on a prototype two-fingered hand. This 5 d.o.f. hand consists of a tendon driven index and thumb, and is equipped with force and tactile sensors. The dimensions and performance of this device are 'human-sized'. A hybrid force-position control scheme is used. The hierarchical control structure is implemented on a dual transputer system. This paper first describes the kinematic model used for whole-finger manipulation. In the second part, the main emphasis is put on the mechanical design and on the transputer-based control system.     

On-line exploration of an unknown surface by a robotic probe

A control strategy is developed for a robotic probe with tactile sensing to explore an unknown surface without losing contact. Digital computer simulations are performed of a three-link, planar manipulator exploring an ellipsoidal surface in order to test the control strategy. The equations of motion are written and linear, time-varying, state-variable feedback is applied to stabilize and decouple the system. A tactile sensor is simulated to supply the normal force between the robot and the surface. From the magnitude and direction of this force, the desired trajectory is determined. An inverse plant and force feedback are implemented to provide the required input torques to the robot's actuators.     

Towards autonomous robotic servicing: Using an integrated hand-arm-eye system for manipulating unknown objects

Executing complex robotic tasks including dexterous grasping and manipulation requires a combination of dexterous robots, intelligent sensors and adequate object information processing. In this paper, vision has been integrated into a highly redundant robotic system consisting of a tiltable camera and a three-fingered dexterous gripper both mounted on a puma-type robot arm. In order to condense the image data of the robot working space acquired from the mobile camera, contour image processing is used for offline grasp and motion planning as well as for online supervision of manipulation tasks. The performance of the desired robot and object motions is controlled by a visual feedback system coordinating motions of hand, arm and eye according to the specific requirements of the respective situation. Experiences and results based on several experiments in the field of service robotics show the possibilities and limits of integrating vision and tactile sensors into a dexterous hand-arm-eye system being able to assist humans in industrial or servicing environments.     

The design and control of a robot finger for tactile sensing

This article describes the design and control of a lightweight robot finger intended for tactile sensing research. The finger is a three-link planar chain with the joints actuated through cables by two motors. Kinematic coupling of the three joints provides two degrees of freedom for finger tip manipulation, and a curling action of the finger for enclosing an object. Hall effect sensors in each joint provide position feedback, and strain gage sensors on each cable provide tension information. To minimize weight and power consumption, a high speed low torque motor together with a 172:1 speed reducer is used as the actuator. A force control loop around the motor speed reducer system reduces the effect of the friction inherent in the speed reducer. Flat mounting plates are provided on each link for special purpose grasping surfaces and sensors.     

Evaluation of a grey scale tactile array sensor pad for robotic applications

Since sensory feedback is an important part of robot control and the acquisition, manipulation, and recognition of objects, incorporating a sense of touch into a robotic system can greatly enhance the performance of that system. This article describes the evaluation of a recently developed low-resolution tactile array sensor pad system for use in robotic applications. Computer algorithms are developed which acquire data from the sensor pad and display the data on a CRT screen. Vision algorithms are implemented in order to extract the necessary information from the tactile data which will aid in the acquisition, manipulation, and recognition of objects. An object's pose is estimated by calculating its center of gravity (position) and principal axis (orientation). Recognizing an object and distinguishing between different objects is accomplished by implementing algorithms which estimate an object's perimeter (shape) and area (size). This work demonstrates that a low-resolution tactile array sensor is capable of providing the information that is required for many robotic applications in which objects must be acquired, manipulated, and recognized. Such a system provides a low-cost alternative to more conventional vision-based systems.     

面向ERT大面积触觉传感的自适应优化成像方法

在动态的非结构化环境中,有效地感知物理接触对于智能机器人安全交互至关重要。为了能够检测各种潜在的物理交互,需要在机器人表面部署大面积触觉传感器。目前,现有的大面积触觉传感器主要是通过传感阵列方式实现的,但是大规模部署传感元件在实际应用中存在巨大挑战。电阻层析成像(Electrical Resistance Tomography, ERT)技术作为一种连续传感方式,有望克服传统触觉传感阵列的一些限制。为此,利用ERT设计了一款新型的大面积触觉传感器。在此基础上,提出了一种基于自适应感兴趣区(Region of Interest, ROI)的图像重构算法,将图像重构限制在交互区域内,以提高传感器的空间分辨率。为了验证提出成像算法的有效性,通过仿真与物理实验对其进行了全面评估。实验验证了该算法可以有效提高触觉传感器在交互区域的空间分辨率,使其具有较高的测量精度。实验结果表明,该传感器的平均定位误差为0.823 cm,能够准确地识别8种不同交互模式,其精度高达98.6%。这一研究工作表明,该传感器为机器人具身触觉传感的实现提供了一个新的解决方案。     

Exoskeletal Force-Sensing End-Effectors With Embedded Optical Fiber-Bragg-Grating Sensors

Force sensing is an essential requirement for dexterous robot manipulation. We describe composite robot end-effectors that incorporate optical fibers for accurate force sensing and estimation of contact locations. The design is inspired by the sensors in arthropod exoskeletons that allow them to detect contacts and loads on their limbs. In this paper, we present a fabrication process that allows us to create hollow multimaterial structures with embedded fibers and the results of experiments to characterize the sensors and controlling contact forces in a system involving an industrial robot and a two-fingered dexterous hand. We also briefly describe the optical-interrogation method used to measure multiple sensors along a single fiber at kilohertz rates for closed-loop force control.      

Artificial Whiskers Suitable for Array Implementation: Accounting for Lateral Slip and Surface Friction

The exquisite tactile sensing ability of biological whiskers has recently led to increasing interest in constructing robotic versions with similar capabilities. Tactile extraction of three-dimensional (3-D) object shape poses several unique challenges that have only begun to be addressed. The present study develops a method for estimating the contact location of a robotic whisker rotating against an object based on small changes in moment at the whisker base. Importantly, the method accounts for lateral slip as well as surface friction, making it particularly well suited for implementation on an array of robotic whiskers. Array implementation would permit simultaneous extraction of multiple contact points and enable highly parallel, efficient 3-D object feature extraction. A simple, scalable array design is suggested to fulfill this approach.      

Feature-based tactile object recognition

Tactile sensing offers powerful capabilities for robotic perception. Through the use of array-force sensors, precisely located surface information about objects in the workspace is available wherever the robot arm may reach. In order to use this information to identify objects and their placement, interpretation processes should employ proprioceptive information and should use tactile image features which reflect object characteristics. A technique is described for the generation of constraints on object identity and placement such that information from multiple sensor contacts may cooperate towards interpretation.