A survey of robotic caging and its applications

Abstract

This paper presents a brief survey of robotic caging and its applications. Caging is a kind of grasping methods, which can be accomplished geometrically by position-controlled robotic agents, and has advantages over conventional manipulation which requires to fulfill mechanical conditions. Due to the advantages, caging was extensively studied and applied in robotics. This paper reviews the robotic literature related to caging ranging from its historical background, state-of-the-art developments, to practical applications. It provides our insights on some open problems and promising research and application directions. We hope the paper could help researchers quickly catch the strength and limitations of caging, and make impacting contributions to the research community.     

A brief review of affordance in robotic manipulation research

Abstract

This paper presents a brief review of affordance research in robotics, with special concentrations on its applications in grasping and manipulation of objects. The concept of affordance could be a key to realize human-like advanced manipulation intelligence. First, we discuss the concept of affordance while associating with the applications in robotics. Then, we intensively explore the studies that utilize affordance for robotic manipulation applications, such as object recognition, grasping, and object manipulation including tool-use. They obtain and use affordance by several ways like learning from human, using simulation, and real-world execution. Moreover, we show our current work, which is a cloud database for advanced manipulation intelligence. The database accumulates various data related to manipulation task execution and will be an open platform to leverage various affordance techniques.     

Dynamics and Stability of Blind Grasping of a 3-Dimensional Object under Non-holonomic Constraints

A mathematical model expressing the motion of a pair of multi-DOF robot fingers with hemi-spherical ends, grasping a 3-D rigid object with parallel flat surfaces, is derived, together with non-holonomic constraints. By referring to the fact that humans grasp an object in the form of precision prehension, dynamically and stably by opposable forces, between the thumb and another finger (index or middle finger), a simple control signal constructed from finger-thumb opposition is proposed, and shown to realize stable grasping in a dynamic sense without using object information or external sensing (this is called "blind grasp" in this paper). The stability of grasping with force/torque balance under non-holonomic constraints is analyzed on the basis of a new concept named "stability on a manifold". Preliminary simulation results are shown to verify the validity of the theoretical results.     

一种基于抓取簇和碰撞体素的工业零件抓取姿态检测算法

针对工业上常见的散乱堆叠零件的抓取问题,提出一种基于抓取簇和碰撞体素的抓取姿态检测算法。所提出的抓取簇是定义在零件上的连续抓取姿态集合,解决了传统方法中因采用离散固定抓取点而导致可抓取点丢失、筛选效率低的问题。先对料箱和场景点云进行体素化;然后把包含料箱或点云的体素标记为碰撞体素,并把与碰撞体素相邻的体素标记为风险体素,从而建立体素化的碰撞模型;接下来,根据抓取簇的几何性质计算出候选的抓取姿态及其对应的抓取路径;最后,通过检测抓取路径所经过的体素类型来完成快速的碰撞检测,从而筛选出最优抓取姿态。基于所提算法搭建了完整的Bin-Picking系统,并对多种实际工业场景中常见的零件进行仿真实验和实际抓取实验,结果表明：该算法能够快速、准确地检测出安全的抓取姿态,实际抓取的平均成功率达92.2%,料箱平均清空率达87.2%,较传统方法有明显提升,且抓取过程均未发生碰撞,可满足实际工业应用的要求。     

虚拟环境中灵巧手主从抓持的实现

研究了虚拟现实环境中人手和灵巧手的抓持动作.利用数据手套采集人手的运动信息,将人手的运动 映射给灵巧手,通过搭建人手和灵巧手的模型,在虚拟环境下实现了主从抓持操作.探讨了关键技术问题:异构系统 运动映射、碰撞检测、虚拟力建模、稳定抓持的判据.􀁱     

抓取任务中的融差控制方法

依据“融差性思维”,提出了无需精确感知依旧可以在一定范围内有效工作的融差控制方法。具体分析了融差抓取方法如何运用相同控制量实现不同抓取任务的工作原理,这一原理使得融差抓取方法在面对一大类抓取任务时,不需要知道物体的具体参数,只需要知道这一大类物体的边界条件。进一步分析了融差抓取方法在欠驱动手爪上的适用性,并发现了欠驱动手爪的局限性。实验表明,在控制量设定不变的情况下,依据融差抓取方法,柔性手爪可以抓住且不抓坏宽度范围为5～45 mm的嫩豆腐,且能够成功抓取宽度范围为5～60 mm的硬质长方体;弹簧关节欠驱动手爪可以抓住且不抓坏宽度范围为20～40 mm的嫩豆腐,且能够成功抓取宽度范围为5～60 mm的硬质长方体。这体现了融差抓取方法的通用性和欠驱动手爪在抓取柔性物体时的局限性。最后,展示了柔性手爪使用融差抓取方法在桌面抓取应用中以简单的控制策略成功抓取不同形状、不同材质的物体。这充分说明了融差抓取方法不依赖于精确的对象感知及物体模型,能够简化控制策略。     

Few-experiential learning system of robotic picking task with selective dual-arm grasping

Recently, robots are introduced to warehouses and factories for automation and are expected to execute dual-arm manipulation as human does and to manipulate large, heavy and unbalanced objects. We focus on target picking task in the cluttered environment and aim to realize a robot picking system which the robot selects and executes proper grasping motion from single-arm and dual-arm motion. In this paper, we propose a few-experiential learning-based target picking system with selective dual-arm grasping. In our system, a robot first learns grasping points and object semantic and instance label with automatically synthesized dataset. The robot then executes and collects grasp trial experiences in the real world and retrains the grasping point prediction model with the collected trial experiences. Finally, the robot evaluates candidate pairs of grasping object instance, strategy and points and selects to execute the optimal grasping motion. In the experiments, we evaluated our system by conducting target picking task experiments with a dual-arm humanoid robot Baxter in the cluttered environment as warehouse.     

A strategy for fast grasping of unknown objects using partial shape information from range sensors

In this paper, we present a strategy for fast grasping of unknown objects based on the partial shape information from range sensors for a mobile robot with a parallel-jaw gripper. The proposed method can realize fast grasping of an unknown object without needing complete information of the object or learning from grasping experience. Information regarding the shape of the object is acquired by a 2D range sensor installed on the robot at an inclined angle to the ground. Features for determining the maximal contact area are extracted directly from the partial shape information of the unknown object to determine the candidate grasping points. Note that since the shape and mass are unknown before grasping, a successful and stable grasp cannot be in fact guaranteed. Thus, after performing a grasping trial, the mobile robot uses the 2D range sensor to judge whether the object can be lifted. If a grasping trial fails, the mobile robot will quickly find other candidate grasping points for another trial until a successful and stable grasp is realized. The proposed approach has been tested in experiments, which found that a mobile robot with a parallel-jaw gripper can successfully grasp a wide variety of objects using the proposed algorithm. The results illustrate the validity of the proposed algorithm in term of the grasping time.     

Development of multi-fingered dexterous hand for grasping manipulation

In this paper, a developed multi-fingered dexterous hand with flexible tactile skin is described. The dexterous hand has 5-fingers with 6-DOFs and each finger is equipped with a small harmonic drive gear and a fine high-power mini actuator. To achieve the goal of grasping with high accuracy, each fingertip is covered with the tactile array sensors for determination of the force between the finger and the grasped object. Some preliminary experiments are conducted to illustrate the performance of the grasping of the developed dexterous hand.     

Fast grasping of unknown objects through automatic determination of the required number of mobile robots

In this paper, we present a strategy for fast grasping of unknown objects by mobile robots through automatic determination of the number of robots. An object handling system consisting of a Gripper robot and a Lifter robot is designed. The Gripper robot moves around an unknown object to acquire partial shape information for determination of grasping points. The object is transported if it can be lifted by the Gripper robot. Otherwise, if all grasping trials fail, a Lifter robot is used. In order to maximize use of the Gripper robot’s payload, the detected grasping points that apply the largest force to the gripper are selected for the Gripper robot when the object is grasped by two mobile robots. The object is measured using odometry and scanned data acquired while the Gripper robot moves around the object. Then, the contact point for calculating the insert position for the Lifter robot can be acquired quickly. Finally, a strategy for fast grasping of known objects by considering the transition between stable states is used to realize grasping of unknown objects. The proposed approach is tested in experiments, which find that a wide variety of objects can be grasped quickly with one or two mobile robots.     

Grasping objects localized from uncertain point cloud data

Robotic grasping is very sensitive to how accurate is the pose estimation of the object to grasp. Even a small error in the estimated pose may cause the planned grasp to fail. Several methods for robust grasp planning exploit the object geometry or tactile sensor feedback. However, object pose range estimation introduces specific uncertainties that can also be exploited to choose more robust grasps. We present a grasp planning method that explicitly considers the uncertainties on the visually-estimated object pose. We assume a known shape (e.g. primitive shape or triangle mesh), observed as a–possibly sparse–point cloud. The measured points are usually not uniformly distributed over the surface as the object is seen from a particular viewpoint; additionally this non-uniformity can be the result of heterogeneous textures over the object surface, when using stereo-vision algorithms based on robust feature-point matching. Consequently the pose estimation may be more accurate in some directions and contain unavoidable ambiguities.The proposed grasp planner is based on a particle filter to estimate the object probability distribution as a discrete set. We show that, for grasping, some ambiguities are less unfavorable so the distribution can be used to select robust grasps. Some experiments are presented with the humanoid robot iCub and its stereo cameras.     

Dynamic Analysis and Control Synthesis of Undesired Slippage of End-Effectors in a Cooperative Grasping

This paper addresses dynamic analysis and control synthesis of object grasping in a cooperative multirobot system with n-serial manipulators from an undesired slippage point of view. Two control approaches are presented in this article; a modified version of a conventional method in grasp synthesis and a new method based on a new modeling of system dynamics. A new formulation for frictional contact is used in dynamical modeling, where equality and inequality equations of the standard Coulomb friction model are all converted to a single second-order differential equation. A multiphase controller is utilized to control the object trajectory tracking as well as object slippage in the new control approach. Performance and robustness of both approaches are studied numerically. The results show superiority of the new method and its desirable and excellent performance.     

Robotic execution of everyday tasks by means of external vision/force control

In this article, we present an integrated manipulation framework for a service robot, that allows to interact with articulated objects at home environments through the coupling of vision and force modalities. We consider a robot which is observing simultaneously his hand and the object to manipulate, by using an external camera (i.e. robot head). Task-oriented grasping algorithms (Proc of IEEE Int Conf on robotics and automation, pp 1794–1799, 2007) are used in order to plan a suitable grasp on the object according to the task to perform. A new vision/force coupling approach (Int Conf on advanced robotics, 2007), based on external control, is used in order to, first, guide the robot hand towards the grasp position and, second, perform the task taking into account external forces. The coupling between these two complementary sensor modalities provides the robot with robustness against uncertainties in models and positioning. A position-based visual servoing control law has been designed in order to continuously align the robot hand with respect to the object that is being manipulated, independently of camera position. This allows to freely move the camera while the task is being executed and makes this approach amenable to be integrated in current humanoid robots without the need of hand-eye calibration. Experimental results on a real robot interacting with different kind of doors are presented.     

Performance improvements of a sweet pepper harvesting robot in protected cropping environments

Using robots to harvest sweet peppers in protected cropping environments has remained unsolved despite considerable effort by the research community over several decades. In this paper, we present the robotic harvester, Harvey, designed for sweet peppers in protected cropping environments that achieved a 76.5% success rate on 68 fruit (within a modified scenario) which improves upon our prior work which achieved 58% on 24 fruit and related sweet pepper harvesting work which achieved 33% on 39 fruit (for their best tool in a modified scenario). This improvement was primarily achieved through the introduction of a novel peduncle segmentation system using an efficient deep convolutional neural network, in conjunction with three‐dimensional postfiltering to detect the critical cutting location. We benchmark the peduncle segmentation against prior art demonstrating an improvement in performance with a $F_1$ score of 0.564 compared to 0.302. The robotic harvester uses a perception pipeline to detect a target sweet pepper and an appropriate grasp and cutting pose used to determine the trajectory of a multimodal harvesting tool to grasp the sweet pepper and cut it from the plant. A novel decoupling mechanism enables the gripping and cutting operations to be performed independently. We perform an in‐depth analysis of the full robotic harvesting system to highlight bottlenecks and failure points that future work could address.     

Micro robot arm utilizing rapid deformations of piezoelectric elements

A unique precise positioning mechanism is introduced. It utilizes inertial force and friction, and can make step-like motion of several nanometres up to several micrometres. Repeating step-like movements, precise positioning with a resolution of several nonometres and for an unlimited movable range is possible. Prototypes of joint mechanisms for a micro robot and a 4-degrees of freedom micro robot arm were constructed using this mechanism. These arms and joints proved to have a high positioning resolution and a practical maximum velocity. Applications of this micro robot arm are also discussed.     

Evaluation of quality of partial caging by a planar two-fingered hand

In partial caging, an object is partially constrained by robots and is able to escape from there. Although complete caging ensures the hand never releases the confined object, insufficient degrees of freedom of robots does not often satisfy the conditions for caging. Partial caging, however, can be accomplished even by robots having such mechanical restriction. We consider a case that an object moves in the semi-closed region formed by a planar robot hand with two fingers, as an example of partial caging in two-dimensional space. Then the parameters of fingers: joint angles interfere in the object motion to escape from the hand through the gap between the fingertips. Some simulation results show differences of difficulty of escaping according to arrangement of fingers, and factors interfering in the difficulty are analyzed. Additionally, we also evaluate ease of entering the hand through the gap and define an ability index of robot hand for partial caging with the above two evaluation scores. Then a high index score indicates that the hand assumed to be able to capture objects easily and confine it without any finger motion. It can be utilized for mechanical design and controlling strategies of robots in capturing objects.     

利用变结构系统理论实现人工肌肉的夹持力控制

本文推导了人工肌肉系统的数学模型.探讨了用变结构系统理论实现人工肌肉的夹持力的控制方法,给出了仿真和实验结果.人工肌肉既可以输出力(力矩),又可以调节刚性,是一种在柔顺运动控制中很有应用前途的新型作动器.     

基于ROS的七自由度机械臂抓取系统研究

针对移动机器人上的七自由度机械臂进行系统设计,以完成目标检测、跟踪和抓取等工作.该系统包括上位机、图像采集设备和机械臂控制三大部分,在ROS Kinetic框架下让Franka Panda机械臂和Kinect V1摄像头协同工作.采用Linemod方法提取图像特征,可在较短训练时间内从RGBD信息中得到物体的信息、位姿...     

虚拟环境下物体的手抓取

本文分析了手模型及其虚拟环境下物体的操作方法，提出了点接触平面法矢的手握持抓取方法。在此基础上运用虚拟手模型和抓取规则实现了在虚拟环境下对物体的抓取、移动和释放，实例说明能够使虚拟手正确地抓取虚拟环境中的物体。     

多目标背景下双目视觉深度信息优化研究及其实现

针对机器人领域应用视觉进行目标物体抓取问题,提出了一种针对多目标背景下,新的深度优化处理方法.通过设定一个阈值块,以遍历成块的深度信息用类似聚类的方法,提出目标物体的具体坐标,传递给机器人手臂,完成准确的抓取操作.依次介绍了双目视觉原理、摄像机标定、双目矫正和双目匹配等内容,以及呈现出原始的深度信息图以及优化后的深度信息图,比较它们的差距.最后在实验中给出了证明:此种深度信息优化方法能够有效的提高机器人抓取目标物体的成功率.最后,还在文章最后给出了下一步的研究方向.