A bin picking system based on depth from defocus

It is generally accepted that to develop versatile bin-picking systems capable of grasping and manipulation operations, accurate 3-D information is required. To accomplish this goal, we have developed a fast and precise range sensor based on active depth from defocus (DFD). This sensor is used in conjunction with a three-component vision system, which is able to recognize and evaluate the attitude of 3-D objects. The first component performs scene segmentation using an edge-based approach. Since edges are used to detect the object boundaries, a key issue consists of improving the quality of edge detection. The second component attempts to recognize the object placed on the top of the object pile using a model-driven approach in which the segmented surfaces are compared with those stored in the model database. Finally, the attitude of the recognized object is evaluated using an eigenimage approach augmented with range data analysis. The full bin-picking system will be outlined, and a number of experimental results will be examined. Received: 2 December 2000 / Accepted: 9 September 2001 Correspondence to: O. Ghita     

Stereo vision based automation for a bin-picking solution

As flexibility becomes an important factor in factory automation, the bin-picking system, where a robot performs pick-and-place tasks for randomly piled parts in a bin through measuring the 3D pose of an object by a 3D vision sensor, has been actively studied. However, conventional bin-picking systems that are employed for particular tasks are limited by such things as the FOV (Field of View), the shape of landmark features, and computation time. This paper proposes a general-purpose stereo vision based bin-picking system. To detect the workpiece to be picked, a geometric pattern matching (GPM) method with respect to the 2D image with a wide FOV is applied. The accurate 3D pose of a selected workpiece among the pick-up candidates is acquired by measuring the 3D positions of three features in the workpiece using the stereo camera. In order to improve the 3D position estimation performance, the GPM method is also used instead of the stereo matching method. The multiple pattern registration and ellipse fitting techniques are additionally applied to increase the reliability. The grasp position of a workpiece without collision is determined using the pose of the object and the bin information. By using these methods a practical bin-picking strategy is established to operate robustly with minimum help from the human workers in the factory. Through experiments on commercial industrial workpieces and industrial robot, we validated that the proposed vision system accurately measures the 3D pose of part and the robot successfully manipulates the workpiece among randomly stacked parts.     

A sensor for dynamic tactile information with applications in human–robot interaction and object exploration

We present a novel tactile sensor, which is applied for dextrous grasping with a simple robot gripper. The hardware novelty consists of an array of capacitive sensors, which couple to the object by means of little brushes of fibers. These sensor elements are very sensitive (with a threshold of about 5 mN) but robust enough not to be damaged during grasping. They yield two types of dynamical tactile information corresponding roughly to two types of tactile sensor in the human skin. The complete sensor consists of a foil-based static force sensor, which yields the total force and the center of the two-dimensional force distribution and is surrounded by an array of the dynamical sensor elements. One such sensor has been mounted on each of the two gripper jaws of our humanoid robot and equipped with the necessary read-out electronics and a CAN bus interface. We describe applications to guiding a robot arm on a desired trajectory with negligible force, reflective grip improvement, and tactile exploration of objects to create a shape representation and find stable grips, which are applied autonomously on the basis of visual recognition.     

基于PVDF的三维力机器人触觉传感器的设计

当机器手抓取时物体受力信息检测是抓取过程顺利进行的基础,检测三维方向上的力便可充分反映物体的受力信息。当前用于抓取过程中三维力检测的触觉传感器还存在着一些不足,基于此,论文拟设计一种基于PVDF的三维力机器人触觉传感器。论文展示了传感器的结构设计,建立了压电薄膜及传感头结构的数学模型,设计了调理电路并对传感器进行测试和验证,结果表明该传感器能有效检测机器手抓取过程中的三维力信息。     

Robust grasping under object pose uncertainty

This paper presents a decision-theoretic approach to problems that require accurate placement of a robot relative to an object of known shape, such as grasping for assembly or tool use. The decision process is applied to a robot hand with tactile sensors, to localize the object on a table and ultimately achieve a target placement by selecting among a parameterized set of grasping and information-gathering trajectories. The process is demonstrated in simulation and on a real robot. This work has been previously presented in Hsiao et al. (Workshop on Algorithmic Foundations of Robotics (WAFR), 2008; Robotics Science and Systems (RSS), 2010) and Hsiao (Relatively robust grasping, Ph.D. thesis, Massachusetts Institute of Technology, 2009).     

3-D imaging systems and high-speed processing for robot control

Bin picking by a robot in real time requires the performance of a series of tasks that are beyond the capabilities of commercially available state-of-the-art robotic systems. In this paper, a laser-ranging sensor for real-time robot control is described. This sensor is incorporated into a robot system that has been applied to the bin-picking or random-parts problem. This system contains new technological components that have been developed recently at the Environmental Research Institute of Michigan (ERIM). These components (the 3-D imaging scanner and a recirculating cellular-array pipeline processor) make generalized real-time robot vision a practical and viable technology. This paper describes these components and their implementation in a typical real-time robot vision system application.     

Robust object manipulation for tactile-based blind grasping

Tactile-based blind grasping addresses realistic robotic grasping in which the hand only has access to proprioceptive and tactile sensors. The robotic hand has no prior knowledge of the object/grasp properties, such as object weight, inertia, and shape. There exists no manipulation controller that rigorously guarantees object manipulation in such a setting. Here, a robust control law is proposed for object manipulation in tactile-based blind grasping. The analysis ensures semi-global asymptotic and exponential stability in the presence of model uncertainties and external disturbances that are neglected in related work. Simulation and hardware results validate the effectiveness of the proposed approach.     

Current research in robotics and automation-an intelligent graspingsystem

A research project is described that focuses on building a comprehensive grasping environment capable of performing tasks such as locating moving objects and picking them up, manipulating man-made objects such as tools, and recognizing unknown objects through touch. In addition, an integrated programming environment is being designed that will allow grasping and grasping primitives within an overall robotic control and programming system that includes dextrous hands, vision sensors, and multiple-degree-of-freedom manipulators. A system overview is given, and the applications are discussed     

Design and Developmental Metrics of a ‘Skin-Like’ Multi-Input Quasi-Compliant Robotic Gripper Sensor Using Tactile Matrix

Slip-resistant robust grasping of objects during remote manipulation remains one of the major open issues in robotics. Finer measurement of tangential force and slippage need to be considered for the task planning and control of robotic gripper in operation. Design and development of such a multi-sensory tactile array is reported in this paper, which is aimed for direct use in an instrumented jaw intelligent robot gripper for potentially hazardous radioactive environments. A new design has been reported in the paper, wherein sensing members of the prototype follow a combination of beam (bending) and truss-type (axial deformation) behavior under external loadings. Various characteristics of the sensor, viz. condition number, static and dynamic stiffness, sensitivity and repeatability have been evaluated, based on the results from field trials of the prototype. Besides the comparatively larger prototype, a miniaturized version of the sensor has also been developed and tested for object grasping in real-time.     

Towards autonomous robotic servicing: Using an integrated hand-arm-eye system for manipulating unknown objects

Executing complex robotic tasks including dexterous grasping and manipulation requires a combination of dexterous robots, intelligent sensors and adequate object information processing. In this paper, vision has been integrated into a highly redundant robotic system consisting of a tiltable camera and a three-fingered dexterous gripper both mounted on a puma-type robot arm. In order to condense the image data of the robot working space acquired from the mobile camera, contour image processing is used for offline grasp and motion planning as well as for online supervision of manipulation tasks. The performance of the desired robot and object motions is controlled by a visual feedback system coordinating motions of hand, arm and eye according to the specific requirements of the respective situation. Experiences and results based on several experiments in the field of service robotics show the possibilities and limits of integrating vision and tactile sensors into a dexterous hand-arm-eye system being able to assist humans in industrial or servicing environments.     

Control of an object with parallel surfaces by a pair of finger robots without object sensing

This paper proposes a method for controlling an object with parallel surfaces in a horizontal plane by a pair of finger robots. The control method can achieve stable grasping, relative orientation control, and relative position control of the grasped object. The control inputs require neither any object parameters nor any object sensing, such as tactile sensors, force sensors, or visual sensors. The control inputs are also quite simple and do not need to solve either inverse kinematics or inverse dynamics. The stability of the closed-loop system is proved, and simulation and experimental results validate the control method.     

Discriminability-based evaluation of transmission capability of tactile transmission systems

Tactile transmission systems deliver tactile information such as texture roughness to operators of robotic systems. Such systems are typically composed of tactile sensors that sense the physical characteristics of textures and tactile displays that present tactile stimuli to operators. One problem associated with tactile transmission systems is that when the system has a bottleneck, it is difficult to identify whether the tactile sensor, tactile display, or perceptual ability of the user is the cause because they have different performance criteria. To solve this problem, this study established an evaluation method that uses the discriminability index as an evaluation criterion. The method lets tactile sensors, displays, and human tactile perception be assessed in terms of the ability to transmit physical quantities; the same criterion is used for all three possible causes so that their abilities can be directly compared. The developed method was applied to a tactile-roughness transmission system (Okamoto et?al. 2009), and its tactile sensor was identified as the bottleneck of the system.     

Detection of incipient object slippage by skin-like sensing andneural network processing

Detection of incipient slippage is of great importance in robotics for the control of grasping and manipulation tasks. Together with fine-form reconstruction and primitive recognition, it has to be the main feature of an artificial tactile system. The system presented here is based on a neural network used to detect incipient slippage and on a skin-like sensor sensible to normal and shear stresses. Normal and shear stresses components inside the sensor are the input data of the neural net. An important feature of the system is that the a priori knowledge of the friction coefficient between the sensor and the object being manipulated is not needed. To validate the method we worked on both simulated and experimental data. In the first case, the finite element method is used to solve the direct problem of elastic contact in its full nonlinearity by resorting to the lowest number of approximations regarding the real problem. Simulation has shown that the network learns and is robust to noise. Then an experimental test was carried out. Experimental results show that, in a simple case, the method is able to detect the insipiency of slippage between an object and the sensor.     

Grasping objects localized from uncertain point cloud data

Robotic grasping is very sensitive to how accurate is the pose estimation of the object to grasp. Even a small error in the estimated pose may cause the planned grasp to fail. Several methods for robust grasp planning exploit the object geometry or tactile sensor feedback. However, object pose range estimation introduces specific uncertainties that can also be exploited to choose more robust grasps. We present a grasp planning method that explicitly considers the uncertainties on the visually-estimated object pose. We assume a known shape (e.g. primitive shape or triangle mesh), observed as a–possibly sparse–point cloud. The measured points are usually not uniformly distributed over the surface as the object is seen from a particular viewpoint; additionally this non-uniformity can be the result of heterogeneous textures over the object surface, when using stereo-vision algorithms based on robust feature-point matching. Consequently the pose estimation may be more accurate in some directions and contain unavoidable ambiguities.The proposed grasp planner is based on a particle filter to estimate the object probability distribution as a discrete set. We show that, for grasping, some ambiguities are less unfavorable so the distribution can be used to select robust grasps. Some experiments are presented with the humanoid robot iCub and its stereo cameras.     

Design of an optical soft sensor for measuring fingertip force and contact recognition

Among the various ways to estimate user intention in hand exoskeletons, a contact force measurement is definitely the most straightforward and intuitive method. A force sensor, located at the center of a fingertip usually, hinders the tactile sensation of the user by blocking the contact between an object and the fingertip. To overcome this problem, a soft force sensor with horse shoe shape is utilized to measure the contact force and provide the tactile sensation to the user. This work presents the mechanical design, implementation and evaluation of a soft fingertip force sensor. To maximize tactile sensation of the user, we adopted a horse shoe shape structure to leave the finger pad exposed. An optical sensing mechanism was selected for its relatively fast response compared to other soft sensors. The whole sensor system has a soft exterior providing flexibility and a user-friendly interface. To evaluate the sensor’s performance, we carried out sensor optimization process and calibration experiment with a customized test bed. Then, we investigated both static and dynamic response and observed the mechanical behavior and light intensity changes caused by the cross sectional shape and base/agent ratio of PDMS. Lastly, we applied the proposed sensor to the glove type fingertip force monitoring system. The sensor estimates the index finger tip force with high accuracy (R ² = 0:96) within 5N range.     

SmartTouch: electric skin to touch the untouchable

Augmented haptics lets users touch surface information of any modality. SmartTouch uses optical sensors to gather information and electrical stimulation to translate it into tactile display. Augmented reality is an engineer's approach to this dream. In AR, sensors capture artificial information from the world, and existing sensing channels display it. Hence, we virtually acquire the sensor's physical ability as our own. Augmented haptics, the result of applying AR to haptics, would allow a person to touch the untouchable. Our system, SmartTouch, uses a tactile display and a sensor. When the sensor contacts an object, an electrical stimulation translates the acquired information into a tactile sensation, such as a vibration or pressure, through the tactile display. Thus, an individual not only makes physical contact with an object, but also touches the surface information of any modality, even those that are typically untouchable.     

Development of multi-fingered dexterous hand for grasping manipulation

In this paper, a developed multi-fingered dexterous hand with flexible tactile skin is described. The dexterous hand has 5-fingers with 6-DOFs and each finger is equipped with a small harmonic drive gear and a fine high-power mini actuator. To achieve the goal of grasping with high accuracy, each fingertip is covered with the tactile array sensors for determination of the force between the finger and the grasped object. Some preliminary experiments are conducted to illustrate the performance of the grasping of the developed dexterous hand.     

虚拟触觉传感器的仿真模型研究

本文根据真实触觉传感器的原理,对虚拟触觉传感器的仿真模型进行了研究,以便 在机器人手抓拾取操作中揭示与材料特性相关的触觉信息瞬态特性,进而为在虚拟环境中从 事基于传感信息的机器人柔性操作、精密装配操作等提供良好的研究平台．     

Development of soft and distributed tactile sensors and the application to a humanoid robot

This paper describes a comprehensive tactile sensor system which can cover wide areas of full-body robots. Based on design criteria which are introduced from requirements, we develop two types of tactile sensor elements. One is a multi-valued touch sensor which has multi-level pressure thresholds. It is capable of covering wide areas of robot surfaces. The other is made of soft, conductive gel, which has the advantage of compliance compared with other sheet-type tactile sensors. With these two types sensors, we develop the tactile sensor system on the full-body robot 'H4'. Details of the sensor system on the robot and some experiments using tactile information are described.