Learning cooperative grasping with the graph representation of a state-action space

In this paper we present a method for two robot manipulators to learn cooperative tasks. If a single robot is unable to grasp an object in a certain orientation, it can only continue with the help of other robots. The grasping can be realized by a sequence of cooperative operations that re-orient the object. Several sequences are needed to handle the different situations in which an object is not graspable for the robot. It is shown that a distributed learning method based on a Markov decision process is able to learn the sequences for the involved robots, a master robot that needs to grasp and a helping robot that supports him with the re-orientation. A novel state-action graph is used to store the reinforcement values of the learning process. Further an example of aggregate assembly shows the generality of this approach.     

Learning shape classes

This paper is a summary of a new approach to learning shape concepts. In this system, a shape is represented by conjunctions of local shape properties. Conjunctions of local properties are consistent and unique for distinct shapes and are robust enough to represent shape in the presence of occlusion. A new learning method, called property based learning, is developed and used to learn conjunctions of local properties. Unlike other classification methods based on distances or similarities, classification performance does not degrade linearly as the number of classes increases and classification can be done correctly with only partial information of instances. Property based learning is an incremental learning method that selects properties crucial for classification. Two experiments are reported. In the first experiment with tool shapes, this shape learning system is used to classify shapes in the presence of view point changes, local movements such as moving handles of pliers, and occlusion. In the second experiment with hand gestures, the system can classify different gestures regardless of the movement in joints, fingers, and palms     

A 3D shape segmentation approach for robot grasping by parts

Neuro-psychological findings have shown that human perception of objects is based on part decomposition. Most objects are made of multiple parts which are likely to be the entities actually involved in grasp affordances. Therefore, automatic object recognition and robot grasping should take advantage from 3D shape segmentation. This paper presents an approach toward planning robot grasps across similar objects by part correspondence. The novelty of the method lies in the topological decomposition of objects that enables high-level semantic grasp planning.In particular, given a 3D model of an object, the representation is initially segmented by computing its Reeb graph. Then, automatic object recognition and part annotation are performed by applying a shape retrieval algorithm. After the recognition phase, queries are accepted for planning grasps on individual parts of the object. Finally, a robot grasp planner is invoked for finding stable grasps on the selected part of the object. Grasps are evaluated according to a widely used quality measure. Experiments performed in a simulated environment on a reasonably large dataset show the potential of topological segmentation to highlight candidate parts suitable for grasping.     

Learning object,grasping and manipulation activities using hierarchical HMMs

This article presents a probabilistic algorithm for representing and learning complex manipulation activities performed by humans in everyday life. The work builds on the multi-level Hierarchical Hidden Markov Model (HHMM) framework which allows decomposition of longer-term complex manipulation activities into layers of abstraction whereby the building blocks can be represented by simpler action modules called action primitives. This way, human task knowledge can be synthesised in a compact, effective representation suitable, for instance, to be subsequently transferred to a robot for imitation. The main contribution is the use of a robust framework capable of dealing with the uncertainty or incomplete data inherent to these activities, and the ability to represent behaviours at multiple levels of abstraction for enhanced task generalisation. Activity data from 3D video sequencing of human manipulation of different objects handled in everyday life is used for evaluation. A comparison with a mixed generative-discriminative hybrid model HHMM/SVM (support vector machine) is also presented to add rigour in highlighting the benefit of the proposed approach against comparable state of the art techniques.     

Learning visually guided grasping: a test case in sensorimotorlearning

We present a general scheme for learning sensorimotor tasks, which allows rapid online learning and generalization of the learned knowledge to unfamiliar objects. The scheme consists of two modules, the first generating candidate actions and the second estimating their quality. Both modules work in an alternating fashion until an action which is expected to provide satisfactory performance is generated, at which point the system executes the action. We developed a method for off-line selection of heuristic strategies and quality predicting features, based on statistical analysis. The usefulness of the scheme was demonstrated in the context of learning visually guided grasping. We consider a system that coordinates a parallel-jaw gripper and a fixed camera. The system learns to estimate grasp quality by learning a function from relevant visual features to the quality. An experimental setup using an AdeptOne manipulator was developed to test the scheme     

Affine shape representation from motion through reference points

This paper shows how an affine representation of spatial configuration is obtained from a pair of projection views. Calibration of cameras and knowledge of the camera's motion are not necessary; however, some preselected reference points and their correspondences are needed. Projective and affine geometry invariants are trickily manipulated to do the affine reconstruction. The method is thus geometrically constructive. When it is compared with the solution proposed in 1989 by J.J. Koenderink and A.J. Van Doorn (Affine Structure from Motion, Technical Report, Utrect University), the method provides a viewpoint-independent affine representation under parallel projections. Further, we investigate the central-projection case in which, with three additional special reference points, the same affine reconstruction can be done. We also discuss some important applications of this viewpoint independence of shape representation.     

3D shape reconstruction by using vanishing points

This paper investigates the quantitative reconstruction of the 3D structure of a scene from a line drawing, by using the geometrical constraints provided by the location of vanishing points. The additional information on vanishing points allows the design of an algorithm which has several advantages with respect to the usual approach based on a reduction to linear programming (Sugihara, 1982). These advantages range from a lower computational complexity to error tolerance and exact reconstruction of the 3D-geometry of the objects. These features make the algorithm a useful tool for the quantitative analysis of real-world images, which is useful for several tasks from scene understanding to automatic vehicle guidance     

Force-closed grasping with two hands

Two-handed grasping of rigid objects in two-dimensional space is studied. The hands considered in this article are either flat-surface palms or grippers with two angular-motion fingers. Presented in this article is a condition that establishes the existence of force-closed grasping without the knowledge of the shape of the grasped object and of the exact contact locations on the palms or fingers. Further, an algorithm is developed that determines force-closed grasping based on the position and orientation of the two hands.     

Compliant grasping with passive forces

Because friction is central to robotic grasp, developing an accurate and tractable model of contact compliance, particularly in the tangential direction, and predicting the passive force closure are crucial to robotic grasping and contact analysis. This paper analyzes the existence of the uncontrollable grasping forces (i.e., passive contact forces) in enveloping grasp or fixturing, and formulates a physical model of compliant enveloping grasp. First, we develop a locally elastic contact model to describe the nonlinear coupling between the contact force with friction and elastic deformation at the individual contact. Further, a set of “compatibility” equations is given so that the elastic deformations among all contacts in the grasping system result in a consistent set of displacements of the object. Then, combining the force equilibrium, the locally elastic contact model, and the “compatibility” conditions, we formulate the natural compliant model of the enveloping grasp system where the passive compliance in joints of fingers is considered, and investigate the stability of the compliant grasp system. The crux of judging passive force closure is to predict the passive contact forces in the grasping system, which is formulated into a nonlinear least square in this paper. Using the globally convergent Levenberg‐Marquardt method, we predict contact forces and estimate the passive force closure in the enveloping grasps. Finally, a numerical example is given to verify the proposed compliant enveloping grasp model and the prediction method of passive force closure. © 2005 Wiley Periodicals, Inc.     

A general shape context framework for object identification

A general shape context framework is proposed for object/image retrieval in occluded and cluttered environment with hundreds of models as the potential matches of an input. The approach is general since it does not require separation of input objects from complex background. It works by first extracting consistent and structurally unique local neighborhood information from inputs or models, and then voting on the optimal matches. Its performance degrades gracefully with respect to the amount of structural information that is being occluded or lost. The local neighborhood information applicable to the system can be shape, color, texture feature, etc. Currently, we employ shape information only. The mechanism of voting is based on a novel hyper cube based indexing structure, and driven by dynamic programming. The proposed concepts have been tested on database with thousands of images. Very encouraging results have been obtained.     

Learning in context: enhancing machine learning with context-based reasoning

This article describes how an experiment to train an agent to perform a task, which had originally failed, was made successful by incorporating a contextual structure that decomposed the tasks into contexts through Context-based Reasoning. The task involved a simulation of a crane that was used by a human operator to move boxes from arbitrary locations throughout a wide area to a designated drop off location in the environment. Initial attempts to teach an agent how to perform the task through observation in a context-free manner yielded poor performance. However, when the task to be learned was decomposed into separate contexts and the agents learned each context independently, the performance improved significantly. The paper describes the process that enabled the improvements achieved and discusses the tests and results that demonstrated the improvement.     

Automatic recognition of object size and shape via user-dependent measurements of the grasping hand

An investigation is conducted on the feasibility of using the posture of the hand during prehension in order to identify geometric properties of grasped objects such as size and shape. A recent study of Paulson et al. (2011) already demonstrated the successful use of hand posture for discriminating between several actions in an office setting. Inspired by their approach and following closely the results in motor planning and control from psychology (Makenzie and Iberall, 1994), we adopt a more cautious and punctilious approach in order to understand the opportunities that hand posture brings for recognizing properties of target objects. We present results from an experiment designed in order to investigate recognition of object properties during grasping in two different conditions: object translation (involving firm grasps) and object exploration (which includes a large variety of different hand and finger configurations). We show that object size and shape can be recognized with up to 98% accuracy during translation and up to 95% and 91% accuracies during exploration by employing user-dependent training. In contrast, experiments show less accuracy (up to 60%) for user-independent training for all tested classification techniques. We also point out the variability of individual grasping postures resulted during object exploration and the need for using classifiers trained with a large set of examples. The results of this work can benefit psychologists and researchers interested in human studies and motor control by providing more insights on grasping measurements, pattern recognition practitioners by reporting recognition results of new algorithms, as well as designers of interactive systems that work on gesture-based interfaces by providing them with design guidelines issued from our experiment.     

Learning graph prototypes for shape recognition

This paper presents some new approaches for computing graph prototypes in the context of the design of a structural nearest prototype classifier. Four kinds of prototypes are investigated and compared: set median graphs, generalized median graphs, set discriminative graphs and generalized discriminative graphs. They differ according to (i) the graph space where they are searched for and (ii) the objective function which is used for their computation. The first criterion allows to distinguish set prototypes which are selected in the initial graph training set from generalized prototypes which are generated in an infinite set of graphs. The second criterion allows to distinguish median graphs which minimize the sum of distances to all input graphs of a given class from discriminative graphs, which are computed using classification performance as criterion, taking into account the inter-class distribution. For each kind of prototype, the proposed approach allows to identify one or many prototypes per class, in order to manage the trade-off between the classification accuracy and the classification time.Each graph prototype generation/selection is performed through a genetic algorithm which can be specialized to each case by setting the appropriate encoding scheme, fitness and genetic operators.An experimental study performed on several graph databases shows the superiority of the generation approach over the selection one. On the other hand, discriminative prototypes outperform the generative ones. Moreover, we show that the classification rates are improved while the number of prototypes increases. Finally, we show that discriminative prototypes give better results than the median graph based classifier.     

基于H-EMD 的形状上下文特征形状匹配方法

为了快速有效利用推土机距离(EMD)模型计算直方图间的交叉相似度,提出一种基于直方图的EMD(H-EMD)模型.将原始模型对应的线性规划问题中变量数目进行约减,降低了直方图相似度计算的复杂度.利用H-EMD模型计算形状上下文特征间的相似度,进而对基于形状上下文形状匹配方法进行改进.通过对不同的数据仿真结果进行比较,H-EMD模型在匹配时间上更具优势,同时,改进的形状匹配方法能有效实现形状识别和检索.     

A strategy for fast grasping of unknown objects using partial shape information from range sensors

In this paper, we present a strategy for fast grasping of unknown objects based on the partial shape information from range sensors for a mobile robot with a parallel-jaw gripper. The proposed method can realize fast grasping of an unknown object without needing complete information of the object or learning from grasping experience. Information regarding the shape of the object is acquired by a 2D range sensor installed on the robot at an inclined angle to the ground. Features for determining the maximal contact area are extracted directly from the partial shape information of the unknown object to determine the candidate grasping points. Note that since the shape and mass are unknown before grasping, a successful and stable grasp cannot be in fact guaranteed. Thus, after performing a grasping trial, the mobile robot uses the 2D range sensor to judge whether the object can be lifted. If a grasping trial fails, the mobile robot will quickly find other candidate grasping points for another trial until a successful and stable grasp is realized. The proposed approach has been tested in experiments, which found that a mobile robot with a parallel-jaw gripper can successfully grasp a wide variety of objects using the proposed algorithm. The results illustrate the validity of the proposed algorithm in term of the grasping time.     

Two-arm manipulation tasks with friction-assisted grasping

—The main objective of this paper is to study human dual-arm manipulation tasks and to develop a computational model that predicts the trajectories and force distribution for the coordination of two arms moving an object between two given positions and orientations in a horizontal plane. Our ultimate goal is to understand the dynamics of human dual-arm coordination in order to develop better robot control algorithms. We propose a computational model based on the hypothesis proposed by Uno et al. that suggests that human movements minimize the integral of the norm of the rate of change of actuator torques. We compare the experimental trajectories and force distributions with those obtained from the computational model. The observed trajectories show a significant degree of repeatability across trials and across subjects. We show that the computational model predicts the trajectories and the distribution of forces (torques) for a certain class of trajectories. However, the trajectories in the sagittal and frontal plane are characterized by asymmetric features that are hard to model using any integral cost function. Finally, we show that the computational model can be used to generate smooth trajectories and actuator forces for cooperating robots and discuss the advantages of such an approach to motion planning.     

Data-driven grasping

This paper propose a novel framework for a data driven grasp planner that indexes partial sensor data into a database of 3D models with known grasps and transfers grasps from those models to novel objects. We show how to construct such a database and also demonstrate multiple methods for matching into it, aligning the matched models with the known sensor data of the object to be grasped, and selecting an appropriate grasp to use. Our approach is experimentally validated in both simulated trials and trials with robots.     

改进形状上下文特征代价矩阵的快速计算

形状上下文特征的最大缺憾在于不具备旋转不变性。为了解决这个问题可以采用二维或一维傅里叶变换来解决,然而这样改进会增加特征提取和匹配的运算时间。鉴于傅里叶变换的对称性,对于改进后的形状上下文特征在特征匹配步骤,提出代价矩阵的快速算法,能够减少运算时间。形状点集匹配的实验结果验证了这种新算法的有效性。