Towards informative sensor-based grasp planning

This paper proposes a probabilistic framework for sensor-based grasping and describes how information about object attributes, such as position and orientation, can be updated using on-line sensor information gained during grasping. This allows learning about the target object even with a failed grasp, leading to replanning with improved performance at each successive attempt. Two grasp planning approaches utilizing the framework are proposed. Firstly, an approach maximizing the expected posterior stability of a grasp is suggested. Secondly, the approach is extended to use an entropy-based explorative procedure, which allows gathering more information when the current belief about the grasp stability does not allow robust grasping. In the framework, both object and grasp attributes as well as the stability of the grasp and on-line sensor information are represented by probabilistic models. Experiments show that the probabilistic treatment of grasping allows improving the probability of success in a series of grasping attempts. Moreover, experimental results on a real platform using the basic stability maximizing approach not only validate the proposed probabilistic framework but also show that under large initial uncertainties, explorative actions help to achieve successful grasps faster.     

Shape Morphing-Based Control of Robotic Visual Servoing

We present an approach for controlling robotic interactions with objects, using synthetic images generated by morphing shapes. In particular, we attempt the problem of positioning an eye-in-hand robotic system with respect to objects in the workspace for grasping and manipulation. In our formulation, the grasp position (and consequently the approach trajectory of the manipulator), varies with each object. The proposed solution to the problem consists of two parts. First, based on a model-based object recognition framework, images of the objects taken at the desired grasp pose are stored in a database. The recognition and identification of the grasp position for an unknown input object (selected from the family of recognizable objects) occurs by morphing its contour to the templates in the database and using the virtual energy spent during the morph as a dissimilarity measure. In the second step, the images synthesized during the morph are used to guide the eye-in-hand system and execute the grasp. The proposed method requires minimal calibration of the system. Furthermore, it conjoins techniques from shape recognition, computer graphics, and vision-based robot control in a unified engineering amework. Potential applications range from recognition and positioning with respect to partially-occluded or deformable objects to planning robotic grasping based on human demonstration.     

Multi-fingered grasp synthesis based on the object dynamic properties

The grasping and manipulation of objects, especially when they are heavy with respect to the hand power capability, requires the synthesis of grasp configurations that explicitly take into account the dynamic properties of the object. Specifically, suitable grasp configurations reducing gravitational and inertial effects during object manipulation, and minimizing and equally distributing the grasping forces among all the available fingers, must be computed. A new method for fast synthesis of multi-fingered grasp configurations is proposed in this paper. In particular, to reduce the computational complexity, all the regions of the object surface favoring the synthesis of minimal inertia grasps are evaluated first. Then, a reduced number of discrete grasping regions are selected on the basis of the fingertip size, model uncertainty, and surface curvature. Finally, an exhaustive search of the optimal grasp configurations with respect to the grasp quality is performed. Several case studies and comparisons with other methods are proposed to demonstrate the effectiveness of the proposed approach.     

Primitive Object Grasping for Finger Motion Synthesis

We developed a new framework to generate hand and finger grasping motions. The proposed framework provides online adaptation to the position and orientation of objects and can generate grasping motions even when the object shape differs from that used during motion capture. This is achieved by using a mesh model, which we call primitive object grasping (POG), to represent the object grasping motion. The POG model uses a mesh deformation algorithm that keeps the original shape of the mesh while adapting to varying constraints. These characteristics are beneficial for finger grasping motion synthesis that satisfies constraints for mimicking the motion capture sequence and the grasping points reflecting the shape of the object. We verify the adaptability of the proposed motion synthesizer according to its position/orientation and shape variations of different objects by using motion capture sequences for grasping primitive objects, namely, a sphere, a cylinder, and a box. In addition, a different grasp strategy called a three‐finger grasp is synthesized to validate the generality of the POG‐based synthesis framework.     

Synergy-based affordance learning for robotic grasping

In this paper, we present an affordance learning system for robotic grasping. The system involves three important aspects: the affordance memory, synergy-based exploration, and a grasping control strategy using local sensor feedback. The affordance memory is modeled with a modified growing neural gas network that allows affordances to be learned quickly from a small dataset of human grasping and object features. After being trained offline, the affordance memory is used in the system to generate online motor commands for reaching and grasping control of the robot. When grasping new objects, the system can explore various grasp postures efficiently in the low dimensional synergy space because the synergies automatically avoid abnormal postures that are more likely to lead to failed grasps. Experimental results demonstrated that the affordance memory can generalize to grasp new objects and predict the effect of the grasp (i.e., the tactile patterns).     

基于三级卷积神经网络的物体抓取检测

借鉴人类抓取物体的特点,提出一种三级串联卷积神经网络用于物体抓取框的检测,实现了对未知物体的高准确度抓取．在所提出的三级串联卷积神经网络中：第1级用于物体的初步定位,为下一级卷积神经网络搜索抓取框确定位置;第2级用于获取预选抓取框,以较小的网络获取较少的特征,从而快速地找出物体的可用抓取框,剔除不可用的抓取框;第3级用于重新评判预选抓取框,以较大的网络获取较多的特征,从而准确地评估每个预选抓取框,获取最佳抓取框．测试结果表明,与单一卷积神经网络相比,三级网络获得抓取框的正确率提高了6.1%,最终在实际Youbot机器人上实现了高准确度的抓取操作．     

A Probabilistic Framework for Uncertainty-Aware High-Accuracy Precision Grasping of Unknown Objects

Grasping is a fundamental skill for robots which work for manipulation tasks. Grasping of unknown objects remains a big challenge. Precision grasping of unknown objects is even harder. Due to imperfection of sensor measurements and lack of prior knowledge of objects, robots have to handle the uncertainty effectively. In previous work (Chen and Wichert 2015), we use a probabilistic framework to tackle precision grasping of model-based objects. In this paper, we extend the probabilistic framework to tackle the problem of precision grasping of unknown objects. We first propose an object model called probabilistic signed distance function (p-SDF) to represent unknown object surface. p-SDF models measurement uncertainty explicitly and allows measurement from multiple sensors to be fused in real time. Based on the surface representation, we propose a model to evaluate the likelihood of grasp success for antipodal grasps. This model uses four heuristics to model the condition of force closure and perceptual uncertainty. A two step simulated annealing approach is further proposed to search and optimize a precision grasp. We use the object representation as a bridge to unify grasp synthesis and grasp execution. Our grasp execution is performed in a closed-loop, so that robots can actively reduce the uncertainty and react to external perturbations during a grasping process. We perform extensive grasping experiments using real world challenging objects and demonstrate that our method achieves high robustness and accuracy in grasping unknown objects.     

Appropriate lengths between phalanges of multijointed fingers for stable grasping

An appropriate arrangement of finger joints is very important since the stability of grasping an object greatly depends on that arrangement. Multijointed fingers can grasp an object with many points of contact each of which is pressed against the object as if wrapping up that object. The amount of the wrapped up area and the form of the finger when an object is grasped are therefore important factors for determining the stability of grasping. We propose the wrapping factor to be used for the evaluation of the stability of grasping by using these factors. We consider 28 models for the finger having three joints, and perform a simulation of their ability to grasp various shapes stably. Based on the simulation results, an appropriate arrangement of lengths between phalanges for a multijointed finger is presented.     

Static grip selection for robot-based automated assembly systems

In this article we present an algorithmic approach to determine the suitable grasp of an object in an automated assembly environment. The algorithm is based on the available object surfaces and the initial and final task constraints and gripper characteristics. If the imposed task and gripper constraints do not allow a possible grasp, intermediate motions may need to be made to reorient the part. Once a set of possible grasps which statisfy task and gripper constraints are found, the stability of each grasp is analyzed using screw theory. An optimal grasp is one which minimizes the grasping forces over the possible set of grasps. Results utilizing our methodology are presented. Our method can be interfaced with CAD database such as a solid modelling system based on boundary representation for automatic selection of grasping configurations.     

杂乱场景下小物体抓取检测研究

目的 杂乱场景下的物体抓取姿态检测是智能机器人的一项基本技能。尽管六自由度抓取学习取得了进展,但先前的方法在采样和学习中忽略了物体尺寸差异,导致在小物体上抓取表现较差。方法 提出了一种物体掩码辅助采样方法,在所有物体上采样相同的点以平衡抓取分布,解决了采样点分布不均匀问题。此外,学习时采用多尺度学习策略,在物体部分点云上使用多尺度圆柱分组以提升局部几何表示能力,解决了由物体尺度差异导致的学习抓取操作参数困难问题。通过设计一个端到端的抓取网络,嵌入了提出的采样和学习方法,能够有效提升物体抓取检测性能。结果 在大型基准数据集GraspNet-1Billion上进行评估,本文方法取得对比方法中的最优性能,其中在小物体上的抓取指标平均提升了7%,大量的真实机器人实验也表明该方法具有抓取未知物体的良好泛化性能。结论 本文聚焦于小物体上的抓取,提出了一种掩码辅助采样方法嵌入到提出的端到端学习网络中,并引入了多尺度分组学习策略提高物体的局部几何表示,能够有效提升在小尺寸物体上的抓取质量,并在所有物体上的抓取评估结果都超过了对比方法。     

Grasping objects localized from uncertain point cloud data

Robotic grasping is very sensitive to how accurate is the pose estimation of the object to grasp. Even a small error in the estimated pose may cause the planned grasp to fail. Several methods for robust grasp planning exploit the object geometry or tactile sensor feedback. However, object pose range estimation introduces specific uncertainties that can also be exploited to choose more robust grasps. We present a grasp planning method that explicitly considers the uncertainties on the visually-estimated object pose. We assume a known shape (e.g. primitive shape or triangle mesh), observed as a–possibly sparse–point cloud. The measured points are usually not uniformly distributed over the surface as the object is seen from a particular viewpoint; additionally this non-uniformity can be the result of heterogeneous textures over the object surface, when using stereo-vision algorithms based on robust feature-point matching. Consequently the pose estimation may be more accurate in some directions and contain unavoidable ambiguities.The proposed grasp planner is based on a particle filter to estimate the object probability distribution as a discrete set. We show that, for grasping, some ambiguities are less unfavorable so the distribution can be used to select robust grasps. Some experiments are presented with the humanoid robot iCub and its stereo cameras.     

Visual programming by demonstration of grasping skills in the context of a mobile service robot using 1D-topology based self-organizing-maps

An approach to the task of Programming by demonstration (PbD) of grasping skills is introduced, where a mobile service robot is taught by a human instructor how to grasp a specific object. In contrast to other approaches the instructor demonstrates the grasping action several times to the robot to increase reconstruction performance. Only the robot’s stereoscopic vision system is used to track the instructor’s hand. The developed tracking algorithm is designed to not need artificial markers, data gloves or being restricted to fixed or difficult to calibrate sensor installations while at the same time being real-time capable on a mobile service robot with limited resources. Due to the instructor’s repeated demonstrations and his low repeating accuracy, every time a grasp is demonstrated the instructor performs it differently. To compensate for these variations and also to compensate for tracking errors, the use of a Self-Organizing-Map (SOM) with a one-dimensional topology is proposed. This SOM is used to generalize over differently demonstrated grasping actions and to reconstruct the intended approach trajectory of the instructor’s hand while grasping an object. The approach is implemented and evaluated on the service robot TASER using synthetically generated data as well as real world data.     

A strategy for fast grasping of unknown objects using partial shape information from range sensors

In this paper, we present a strategy for fast grasping of unknown objects based on the partial shape information from range sensors for a mobile robot with a parallel-jaw gripper. The proposed method can realize fast grasping of an unknown object without needing complete information of the object or learning from grasping experience. Information regarding the shape of the object is acquired by a 2D range sensor installed on the robot at an inclined angle to the ground. Features for determining the maximal contact area are extracted directly from the partial shape information of the unknown object to determine the candidate grasping points. Note that since the shape and mass are unknown before grasping, a successful and stable grasp cannot be in fact guaranteed. Thus, after performing a grasping trial, the mobile robot uses the 2D range sensor to judge whether the object can be lifted. If a grasping trial fails, the mobile robot will quickly find other candidate grasping points for another trial until a successful and stable grasp is realized. The proposed approach has been tested in experiments, which found that a mobile robot with a parallel-jaw gripper can successfully grasp a wide variety of objects using the proposed algorithm. The results illustrate the validity of the proposed algorithm in term of the grasping time.     

Toeard a formulation of the human grasping quality sense

This article examines the grasping quality problem and suggests a formulation that can capture the major mechanisms that generate the human grasping quality sense. Here, the quality of a grasp is examined from the gripper, object, and grasping-configuration perspectives. These perspectives are incorporated into the formulation by selecting an appropriate objective function that is minimized subject to constraints that represent the geometry, friction, and force-balance conditions of a grasp. The quality of a grasp is depicted as a polar plot that demonstrates the dependency of grasping quality on the external-loading direction. The suggested grasping quality measures possess characteristics that are in agreement with basic human intuition. In particular this article proves that the three proposed quality measures are improved when the number of contact points is increased. In summary, the suggested grasping quality formulation captures some of the physical mechanisms that characterize a human grasp, and therefore it may lead to a powerful mathematical model of the human grasping quality sense. © 1995 John Wiley & Sons, Inc.     

Stability index and contact configuration planning for multifingered grasp

It is necessary to plan the contact configuration to guarantee a stable grasp. This article discusses the grasping stability of multifingered robot hands. The fingers are assumed to be point contacts with friction. A stability index for evaluating a grasp, which is proportional to the ellipsoidal volume in the grasping task space, is proposed. The invariance of the index is proved under an object linear coordinate transformation and under a change of the torque origin. The similar invariance of the index is also proved under a change of the dimensional unit. The optimal grasping of an object by a multifingered robot hand can be obtained using the stability index to plan the grasp configurations. The index is applicable to plan adaptable fixtures as well. A nonlinear programming method to plan configurations is addressed. Several examples are given using the index to evaluate a grasp, in which the obtained optimal grasping is consistent with what human beings expect. The sensibility of the optimal grasping is analyzed in these examples. © 1998 John Wiley & Sons, Inc.     

A New Methodology for Human Grasp Prediction

Grasping is an essential requirement for digital human models (DHMs). It is a complex process and thus a challenging problem for DHMs, involving a skeletal structure with many degrees-of-freedom (DOFs), cognition, and interaction between the human and objects in the environment. Furthermore, grasp planning involves not only finding the shape of the hand and the position and orientation of the wrist but also the posture of the upper body required for producing realistic grasping simulations. In this paper, a new methodology is developed for grasping prediction by combining a shape-matching method and an optimization-based posture prediction technique. We use shape matching to pick a hand shape from a database of stored grasps, then position the hand around the object. The posture prediction algorithm then calculates the optimal posture for the whole upper body necessary to execute the grasp. The proposed algorithm is tested on a variety of objects in a 3-D environment. The results are realistic and suggest that the new method is more suitable for grasp planning than conventional methods. This improved performance is particularly apparent when the nature of the grasped objects is not known a priori , and when a complex high-DOF hand model is necessary.      

Demonstration-based learning and control for automatic grasping

We present a method for automatic grasp generation based on object shape primitives in a Programming by Demonstration framework. The system first recognizes the grasp performed by a demonstrator as well as the object it is applied on and then generates a suitable grasping strategy on the robot. We start by presenting how to model and learn grasps and map them to robot hands. We continue by performing dynamic simulation of the grasp execution with a focus on grasping objects whose pose is not perfectly known.     

Manipulation of unmodeled objects using intelligent grasping schemes

We investigate "intelligent" grasping schemes using a fuzzy logic rule base expert system. We use a vision system, robot arm and mechanical hand to locate and manipulate unmodeled, randomly placed objects of various sizes and shapes. In the pregrasp stage, we use vision data to provide a nonlinear mapping from object characteristics to hand configuration. In the postgrasp stage, we use hand data to ascertain the security of the grasp. Computational geometry is used to gauge the quality of the grasp and to quantify and validate the choice of hand configurations generated by the fuzzy logic expert system. The system is implemented within a low-cost virtual collaborative environment.     

On the grasping stability and optimality under external perturbations

There are two types of grasping analysis in robotics research: find the grasping force distributions among the grasping fingers when given the contact points and find a good set of the contact points when given the shape of the object. Each kind of problem is associated with optimality and stability analysis. In this article, we investigate the grasping stability and optimality issues under the influence of external perturbations. A rotation‐displacement geometry model is used in computing the changes of grasping forces under external perturbations. Using these results, we present the concept of perturbation closure, which plays the central role in our analysis. A method for finding the local minimal perturbation resisting forces required for non‐slip contacts is developed based on this concept. A grasp so determined is guaranteed to be stable if the external perturbations do not exceed the threshold. Based on this property, we develop a quantitative measurement that can be used to evaluate the performance of different grasping configurations. One can use this measurement to determine the best grasping configuration from a set of perturbation resisting grasps. This actually gives a method which enables the optimal grasping configuration to be found. Both two‐dimensional and three‐dimensional cases are discussed in detail for determining the perturbation closure, the local minimal perturbation resisting force, and the perturbation resisting grasp. Examples are given at the end of the article to illustrate our idea. ©1999 John Wiley & Sons, Inc.     

A new strategy combining empirical and analytical approaches for grasping unknown 3D objects

This paper proposes a novel strategy for grasping 3D unknown objects in accordance with their corresponding task. We define the handle or the natural grasping component of an object as the part chosen by humans to pick up this object. When humans reach out to grasp an object, it is generally in the aim of accomplishing a task. Thus, the chosen grasp is quite related to the object task. Our approach learns to identify object handles by imitating humans. In this paper, a new sufficient condition for computing force-closure grasps on the obtained handle is also proposed. Several experiments were conducted to test the ability of the algorithm to generalize to new objects. They also show the adaptability of our strategy to the hand kinematics.