Coupled dynamical system based arm–hand grasping model for learning fast adaptation strategies

Performing manipulation tasks interactively in real environments requires a high degree of accuracy and stability. At the same time, when one cannot assume a fully deterministic and static environment, one must endow the robot with the ability to react rapidly to sudden changes in the environment. These considerations make the task of reach and grasp difficult to deal with. We follow a Programming by Demonstration (PbD) approach to the problem and take inspiration from the way humans adapt their reach and grasp motions when perturbed. This is in sharp contrast to previous work in PbD that uses unperturbed motions for training the system and then applies perturbation solely during the testing phase. In this work, we record the kinematics of arm and fingers of human subjects during unperturbed and perturbed reach and grasp motions. In the perturbed demonstrations, the target’s location is changed suddenly after the onset of the motion. Data show a strong coupling between the hand transport and finger motions. We hypothesize that this coupling enables the subject to seamlessly and rapidly adapt the finger motion in coordination with the hand posture. To endow our robot with this competence, we develop a coupled dynamical system based controller, whereby two dynamical systems driving the hand and finger motions are coupled. This offers a compact encoding for reach-to-grasp motions that ensures fast adaptation with zero latency for re-planning. We show in simulation and on the real iCub robot that this coupling ensures smooth and “human-like” motions. We demonstrate the performance of our model under spatial, temporal and grasp type perturbations which show that reaching the target with coordinated hand–arm motion is necessary for the success of the task.     

Dynamic grasp planning of multifingered robot hands based onasymptotic stability

We describe an approach for planning grasps of multifingered robot hands based on a small vibration model. Using features of the grasp configuration, we analyze asymptotic stability, contact situations, and uniaxial fingertip force constraints for the combined planning of finger posture and finger position, and characterize the generalized mass, damping, and stiffness. Choosing the largest time constant of the vibration model as an optimization criterion for planning finger postures and positions, the original problem of dynamic grasp planning is formulated as a nonlinear program. Simulation examples for a three-fingered robot hand grasping a spherical object demonstrate the effectiveness of the approach.     

Semantic grasping: planning task-specific stable robotic grasps

We present an example-based planning framework to generate semantic grasps, stable grasps that are functionally suitable for specific object manipulation tasks. We propose to use partial object geometry, tactile contacts, and hand kinematic data as proxies to encode task-related constraints, which we call semantic constraints. We introduce a semantic affordance map, which relates local geometry to a set of predefined semantic grasps that are appropriate to different tasks. Using this map, the pose of a robot hand with respect to the object can be estimated so that the hand is adjusted to achieve the ideal approach direction required by a particular task. A grasp planner is then used to search along this approach direction and generate a set of final grasps which have appropriate stability, tactile contacts, and hand kinematics. We show experiments planning semantic grasps on everyday objects and applying these grasps with a physical robot.     

An instrumented glove for grasp specification invirtual-reality-based point-and-direct telerobotics

Hand posture and force, which define aspects of the way an object is grasped, are features of robotic manipulation. A means for specifying these grasping “flavors” has been developed that uses an instrumented glove equipped with joint and force sensors. The new grasp specification system will be used at the Pennsylvania State University (Penn State) in a Virtual Reality based Point-and-Direct (VR-PAD) robotics implementation. Here, an operator gives directives to a robot in the same natural way that human may direct another. Phrases such as “put that there” cause the robot to define a grasping strategy and motion strategy to complete the task on its own. In the VR-PAD concept, pointing is done using virtual tools such that an operator can appear to graphically grasp real items in live video. Rather than requiring full duplication of forces and kinesthetic movement throughout a task as is required in manual telemanipulation, hand posture and force are now specified only once. The grasp parameters then become object flavors. The robot maintains the specified force and hand posture flavors for an object throughout the task in handling the real workpiece or item of interest     

Efficient models for grasp planning with a multi-fingered hand

This paper presents a simple grasp planning method for a multi-fingered hand. Its purpose is to compute a context-independent and dense set or list of grasps, instead of just a small set of grasps regarded as optimal with respect to a given criterion. By context-independent, we mean that only the robot hand and the object to grasp are considered. The environment and the position of the robot base with respect to the object are considered in a further stage. Such a dense set can be computed offline and then used to let the robot quickly choose a grasp adapted to a specific situation. This can be useful for manipulation planning of pick-and-place tasks. Another application is human–robot interaction when the human and robot have to hand over objects to each other. If human and robot have to work together with a predefined set of objects, grasp lists can be employed to allow a fast interaction.The proposed method uses a dense sampling of the possible hand approaches based on a simple but efficient shape feature. As this leads to many finger inverse kinematics tests, hierarchical data structures are employed to reduce the computation times. The data structures allow a fast determination of the points where the fingers can realize a contact with the object surface. The grasps are ranked according to a grasp quality criterion so that the robot will first parse the list from best to worse quality grasps, until it finds a grasp that is valid for a particular situation.     

A human-like real-time grasp synthesis method for humanoid robot hands

This paper addresses a real-time grasp synthesis of multi-fingered robot hands to find grasp configurations which satisfy the force closure condition of arbitrary shaped objects. We propose a fast and efficient grasp synthesis algorithm for planar polygonal objects, which yields the contact locations on a given polygonal object to obtain a force closure grasp by a multi-fingered robot hand. For an optimum grasp and real-time computation, we develop the preference and the hibernation process and assign the physical constraints of a humanoid hand to the motion of each finger. The preferences consist of each sublayer reflecting the primitive preference similar to the conditional behaviors of humans for given objectives and their arrangements are adjusted by the heuristics of human grasping. The proposed method reduces the computational time significantly at the sacrifice of global optimality, and enables grasp posture to be changeable within 2-finger and 3-finger grasp. The performance of the presented algorithm is evaluated via simulation studies to obtain the force-closure grasps of polygonal objects with fingertip grasps. The architecture suggested is verified through experimental implementation to our developed robot hand system by solving 2- or 3-finger grasp synthesis.     

Momentum Transfer from Preshape to Grasping

A fully immersed object, suspended in water can be rotated from distance by a preshaped robot hand approaching and closing upon the object prior to contacting it. Momentum transfer from robot fingers closing into a grasp, to the fluid medium particles, and from these particles to the object surface generates the motion tendencies of that object in terms of rotational and translational displacements. In this paper, we propose the novel concept of a controller that determines either: 1) given initial position and orientation of a robot hand, what preshape is suitable for generating a desired momentum distribution on the surface of a given object in order to trigger a desired rotation in a desired direction when approaching with this preshaped hand; or 2) given a predetermined hand preshape, what initial position, orientation and hand aperture are suitable to generate a desired rotation upon approach and, without causing the retroceeding of the object. The desired object motion generated from distance by the approach of a hand preshape is to be used seamlessly for the subsequent manipulation of the object upon grasp. Towards this end, we propose in our work, a new model based on computational fluid dynamics, for determining the continuity in momentum transfer from robot hand fingers to the fluid medium, and to the object, until landing on that immersed object. Our experimental results demonstrate how different hand preshapes initiated from different locations in the medium surrounding an object of different cross sections suspended in equilibrium in the fluid, affects its motion tendencies in terms of rotation and translation. Our further contribution, in this paper, includes the modelling of robot fingers and object as fluidic elements which rigidity can be relaxed to induce compliance.     

Pose Planning for a Mobile Manipulator Based on Joint Motions for Posture Adjustment to End-Effector Error

This paper proposes a motion planning method for a mobile manipulator. In general, humans can grasp an object by various ways which depend on object posture, position and so on. The objective of this paper is to present how to detect the pose of a mobile manipulator under the condition that several ways of grasping are given to the robot. Motion errors and object position errors are considered to detect robot pose in our method because these affect the grasp motion of the robot hand. Coping with these errors, we will propose an effective pose searching method for a mobile manipulator from numerous pose candidates. The performance of the proposed method is illustrated by simulation and experiment.     

Towards cognitive grasping: modeling of unknown objects and its corresponding grasp types

This paper describes an intuitive approach for a cognitive grasp of a robot. The cognitive grasp means the chain of processes that make a robot to learn and execute a grasping method for unknown objects like a human. In the learning step, a robot looks around a target object to estimate the 3D shape and understands the grasp type for the object through a human demonstration. In the execution step, the robot correlates an unknown object to one of known grasp types by comparing the shape similarity of the target object based on previously learned models. For this cognitive grasp, we mainly deal with two functionalities such as reconstructing an unknown 3D object and classifying the object by grasp types. In the experiment, we evaluate the performance of object classification according to the grasp types for 20 objects via human demonstration.     

机器人多指操作的递阶控制1)

This paper presents a hierarchical control system for robot multifingered coordinate manipulation. Given a manipulation,the task planner generates a sequence of object's motion velocities at first,and then generates for coordinate motion the desired velocities of finger's motion and desired orientation change of the grasped object according to the desired velocities of object's motion.At the same time,the force planner generates the grasp forces on the fingers in order to resist the external forces on the object,according to the grasp posture.Finally,the system generates a result compliance velocity from both the desired finger's velocities and desired grasp forces,and transfers it into joint velocites through the finger's inverse Jacobian.Then the controller of joint motion implements the control of both forces and velocities for the fingers.The approach has been applied to the development of control system HKUST dexterous hand successfully.Experiment results show that it is not only possible to trail and control the object's track,but also possible to realize force control and the hybrid control of both forces and velocities through this method.     

A 3D shape segmentation approach for robot grasping by parts

Neuro-psychological findings have shown that human perception of objects is based on part decomposition. Most objects are made of multiple parts which are likely to be the entities actually involved in grasp affordances. Therefore, automatic object recognition and robot grasping should take advantage from 3D shape segmentation. This paper presents an approach toward planning robot grasps across similar objects by part correspondence. The novelty of the method lies in the topological decomposition of objects that enables high-level semantic grasp planning.In particular, given a 3D model of an object, the representation is initially segmented by computing its Reeb graph. Then, automatic object recognition and part annotation are performed by applying a shape retrieval algorithm. After the recognition phase, queries are accepted for planning grasps on individual parts of the object. Finally, a robot grasp planner is invoked for finding stable grasps on the selected part of the object. Grasps are evaluated according to a widely used quality measure. Experiments performed in a simulated environment on a reasonably large dataset show the potential of topological segmentation to highlight candidate parts suitable for grasping.     

Seven fingers allow force-torque closure grasps on any convex polyhedron

We prove that a robot hand whose fingers make frictionless contact with a convex polyhedral object will be able to find a grasp where the hand can exert any desired force-torque on the object provided the hand has seven fingers. We present an algorithm for grasping any convex polyhedron and we prove rigorously that it works for any convex polyhedron. The algorithm requiresO(n ^3/2logn) steps (in the worst case) wheren is the number of vertices.     

A biomimetic reach and grasp approach for mechanical hands

Reach and grasp are the two key functions of human prehension. The Central Nervous System controls these two functions in a separate but interdependent way. The choice between different solutions to reach and grasp an object–provided by multiple and redundant degrees of freedom (dof)–depends both on the properties and on the use (affordance) of the object to be manipulated. This same control paradigm, i.e. subdivision of prehension into reach and grasp as well as the corresponding multimodal (sensory/motor) information fusion schemes, can also be applied to a mechanical hand carried by a robotic arm. The robotic arm will then be responsible for positioning the hand with respect to the object, and the hand will then grasp and manipulate the object. In this article, we present a biomimetic sensory–motor control scheme in the aim of providing an object-dependent and intelligent reach and grasp ability to such systems. The proposed model is based on a multi-network architecture which incorporates multiple Matching Units trained by a statistical learning algorithm (LWPR). Matching Units perform a multimodal signal integration by correlating sensory and motor information analogous to that observed in cerebral neuronal networks. The simulated network of multiple Matching Units provided estimations of object-dependent 5-finger grasp configurations with endpoint positional errors in the order of a few millimeters. For validation, these estimations were then applied to the control of movement kinematics on an experimental robot composed of a 6 dof robot arm carrying a 16 dof mechanical 4-finger hand. Precision of the kinematics control was such that successful reach, grasp and lift was obtained in all the tests.     

A New Methodology for Human Grasp Prediction

Grasping is an essential requirement for digital human models (DHMs). It is a complex process and thus a challenging problem for DHMs, involving a skeletal structure with many degrees-of-freedom (DOFs), cognition, and interaction between the human and objects in the environment. Furthermore, grasp planning involves not only finding the shape of the hand and the position and orientation of the wrist but also the posture of the upper body required for producing realistic grasping simulations. In this paper, a new methodology is developed for grasping prediction by combining a shape-matching method and an optimization-based posture prediction technique. We use shape matching to pick a hand shape from a database of stored grasps, then position the hand around the object. The posture prediction algorithm then calculates the optimal posture for the whole upper body necessary to execute the grasp. The proposed algorithm is tested on a variety of objects in a 3-D environment. The results are realistic and suggest that the new method is more suitable for grasp planning than conventional methods. This improved performance is particularly apparent when the nature of the grasped objects is not known a priori , and when a complex high-DOF hand model is necessary.      

Vision-based three-finger grasp synthesis constrained by hand geometry

This paper addresses the problem of designing a practical system able to grasp real objects with a three-fingered robot hand. A general approach for synthesizing two- and three-finger grasps on planar unknown objects is presented. Visual perception is used to reduce the uncertainty and to obtain relevant information about the objects.We focus on non-modeled planar extruded objects, which can represent many real-world objects. In addition, particular mechanical constraints of the robot hand are considered.First, a vision processing module that extracts from object images the relevant information for the grasp synthesis is developed. This is completed with a set of algorithms for synthesizing two- and three-finger grasps taking into account force-closure and contact stability conditions, with a low computational effort. Finally, a procedure for constraining these results to the kinematics of the particular hand, is also developed. In addition, a set of heuristic metrics for assessing the quality of the computed grasps is described.All these components are integrated in a complete system. Experimental results using the Barrett hand are shown and discussed.     

机器人多指操作的递阶控制

为机器人多指协调操作建立一递阶控制系统.给定一操作任务,任务规划器首先生 成一系列物体的运动速度;然后,协调运动规划器根据期望的物体运动速度生成期望的手指 运动速度和期望的抓取姿态变化;同时,抓取力规划器为平衡作用在物体上的外力,根据当前 的抓取姿态,生成各手指所需的抓取力;最后,系统将手指的期望运动速度与为实现期望抓取 力而生成的顺应速度合并,并通过手指的逆雅可比转化为手指关节运动速度后,由手指的关 节级运动控制器实现手指的运动和抓取力的控制.该控制方法已成功应用于香港科技大学 (HKUST)灵巧手控制系统的开发.实验证明该方法不仅能完成物体轨迹的跟踪控制任务, 而且能完成物体对环境的力控制和力与速度的混合控制.     

Ensuring kinetostatic consistency in observation of human manipulation

The aim of this paper is to present a method to guarantee the kinetostatic consistency in observation of human manipulation, i.e. the consistency between the observed hand posture and the tactile information on the contact between the fingertips and the objects. The core idea of the proposed algorithm is to compare the fingertip contact information, obtained by tactile sensors, with the contact information computed in a virtual environment, that reproduces the real environment where the observation is carried out. In case the estimation of the joint angles and the relative pose between the hand and the object are not consistent, a correction of the hand posture is computed. For some tasks, collisions might occur between parts of the hand (e.g. palm) and the grasped object. To handle this problem, the corrected hand posture is computed by adopting a closed loop inverse kinematic (CLIK) approach that exploits the redundant Degrees of Freedom (DoFs) of the hand. The algorithm has been designed to work on-line. This feature is particularly important for Programming by Demonstration (PbD) applications, since it allows the trainer to actively adapt the demonstration to measurement noise and model errors. The effectiveness of the proposed method has been tested in five different tasks: grasping a cup, unscrewing a bottle, grasping a plate, grasping a ketchup bottle, and grasping a measuring cup.     

Robotic execution of everyday tasks by means of external vision/force control

In this article, we present an integrated manipulation framework for a service robot, that allows to interact with articulated objects at home environments through the coupling of vision and force modalities. We consider a robot which is observing simultaneously his hand and the object to manipulate, by using an external camera (i.e. robot head). Task-oriented grasping algorithms (Proc of IEEE Int Conf on robotics and automation, pp 1794–1799, 2007) are used in order to plan a suitable grasp on the object according to the task to perform. A new vision/force coupling approach (Int Conf on advanced robotics, 2007), based on external control, is used in order to, first, guide the robot hand towards the grasp position and, second, perform the task taking into account external forces. The coupling between these two complementary sensor modalities provides the robot with robustness against uncertainties in models and positioning. A position-based visual servoing control law has been designed in order to continuously align the robot hand with respect to the object that is being manipulated, independently of camera position. This allows to freely move the camera while the task is being executed and makes this approach amenable to be integrated in current humanoid robots without the need of hand-eye calibration. Experimental results on a real robot interacting with different kind of doors are presented.     

Occlusion-aware multi-view reconstruction of articulated objects for manipulation

We present an algorithm called Procrustes-Lo-RANSAC (PLR) to recover complete 3D models of articulated objects. Structure-from-motion techniques are used to capture 3D point cloud models of an object in two different configurations. Procrustes analysis, combined with a locally optimized RANSAC sampling strategy, facilitates a straightforward geometric approach to recovering the joint axes, as well as classifying them automatically as either revolute or prismatic. With the resulting articulated model, a robotic system is then able to manipulate the object along its joint axes at a specified grasp point in order to exercise its degrees of freedom. Because the models capture all sides of the object, they are occlusion-aware, meaning that the robot has knowledge of parts of the object that are not visible in the current view. Our algorithm does not require prior knowledge of the object, nor does it make any assumptions about the planarity of the object or scene. Experiments with a PUMA 500 robotic arm demonstrate the effectiveness of the approach on a variety of real-world objects containing both revolute and prismatic joints.