结合增量学习的物体抓取检测框架

为了解决未知类别物体的抓取问题，提出了一种结合增量学习的物体抓取检测框架，该框架分为抓取学习和增量学习两个阶段.在第1阶段，对已知的物体使用密集注意力网络进行训练，该网络利用注意力机制对特征通道和密集残差连接之间的关系进行建模.在第2阶段，引入了聚类优先样本选择策略，该策略会挑选出那些与其聚类质心距离相近的样本，用这些新样本替换掉示例集中的部分旧样本进行训练.此外在未知类别物体上训练网络时，还引入了蒸馏损失，以保留之前在已知类中学到的知识.通过在Jacquard数据集和UR10e机器人上进行的实验，表明了该方法在抓取未知类别物体方面有一定的可行性和有效性，克服了机器人在抓取未知类别物体上的缺陷.     

基于视觉感知和触觉先验知识学习的机器人稳定抓取

针对机器人抓取过程中需要实时评估抓取质量以动态调整抓取构型的问题,提出了基于触觉先验知识的机器人稳定抓取方法.首先,根据抓取过程中物体抵抗外界扰动的能力,提出了一种基于触觉信息的抓取质量评估方法.在此基础上,建立了视触觉联合数据集并学习触觉先验知识.其次,提出了融合视觉图像和触觉先验知识的稳定抓取构型生成方法.最后,在搭建的机器人抓取系统中对10种目标物体进行了实验验证.结果表明,相比传统的视觉方法,该方法的抓取稳定性提高了55％;针对已知物体和未知物体,稳定抓取成功率分别为86％和79％,体现了较好的泛化效果.     

基于级联卷积神经网络的机器人平面抓取位姿快速检测

针对任意姿态的未知不规则物体,提出一种基于级联卷积神经网络的机器人平面抓取位姿快速检测方法．建立了一种位置-姿态由粗到细的级联式两阶段卷积神经网络模型,利用迁移学习机制在小规模数据集上训练模型,以R-FCN（基于区域的全卷积网络）模型为基础提取抓取位置候选框进行筛选及角度粗估计,并针对以往方法在姿态检测上的精度不足,提出一种Angle-Net模型来精细估计抓取角度．在Cornell数据集上的测试及机器人在线抓取实验结果表明,该方法能够对任意姿态、不同形状的不规则物体快速计算最优抓取点及姿态,其识别准确性和快速性相比以往方法有所提高,鲁棒性和稳定性强,且能够泛化适应未训练过的新物体．     

基于非结构基本组成分析的自主抓取方法

针对现实生活中的非结构化抓取环境,提出一种基于非规则物体基本形体组成的自主抓取方法.机器人自主抓取的关键不仅仅在于对物体类型的识别,更大一部分在于对物体形状(例如形状基元的组成)判断后的良好抓取.将不规则的复杂物体简化为一些简单物体的组合,利用基于特征点和核心提取的网格分割(MFC)算法将被抓取物体3D数据点分割为主体和分支部分,依据最优拟合算法将各部分拟合为球体、椭球体、圆柱体、平行六面体中的一种,并依据简化结果对抓取位姿进行约束,再对简化后物体进行抓取训练,获取最优抓取框,从而实现对未知物体的自主抓取.本文方法最终在Baxter机器人上实现了93.3%的抓取准确率.实验结果表明,该方法可应用于不同形状、不同位姿的未知非规则物体,鲁棒性较强.     

一种视觉引导的作业型飞行机器人设计

针对作业型飞行机器人执行抓取、投放等作业任务时飞行机器人与被抓取目标之间难以相对定位的问题,提出了一种视觉引导的作业型飞行机器人设计方法.首先,介绍作业型飞行机器人系统的整体机构设计,建立飞行器和空中作业装置的运动学和动力学模型.然后,根据针孔成像模型,在ArUco标记尺寸已知的前提下,通过机载的单目摄像头检测被抓目标上的ArUco标记,利用n点透视(PnP)算法解算摄像头位姿,进而利用摄像头位姿信息对飞行器和作业装置进行分级控制.最后,通过静止实验和户外悬停实验验证了位姿估计算法的有效性,并通过自主抓取直径2 cm、质量100 g的管状物体进一步验证视觉引导的有效性和合理性.     

基于多重几何约束的未知物体抓取位姿估计

针对机器人在非结构化环境下面临的未知物体难以快速稳定抓取的问题,提出一种基于多重几何约束的未知物体抓取位姿估计方法.通过深度相机获取场景的几何点云信息,对点云进行预处理得到目标物体,利用简化的夹持器几何形状约束生成抓取位姿样本.然后,利用简化的力封闭约束对样本进行快速粗筛选.对抓取位姿的抓取几何轮廓进行力平衡约束分析,将稳定的位姿传送至机器人执行抓取.采用深度相机与6自由度机械臂组成实验平台,对不同姿态形状的物体进行抓取实验.实验结果表明,本文方法能够有效应对物体种类繁多、缺乏3维模型的情况,在单目标和多目标场景均具有良好的适用性.     

一种先验知识引导的基于二阶段渐进网络的自主抓取策略

针对未知不规则物体在堆叠场景下的抓取任务,提出一种基于二阶段渐进网络（two-stage progressive network,TSPN）的自主抓取方法．首先利用端对端策略获取全局可抓性分布,然后基于采样评估策略确定最优抓取配置．将以上2种策略融合,使得TSPN的结构更加精简,显著减少了需评估样本的数量,能够在保证泛化能力的同时提升抓取效率．为了加快抓取模型学习进程,引入一种先验知识引导的自监督学习策略,并利用220种不规则物体进行抓取学习．在仿真和真实环境下分别进行实验,结果表明该抓取模型适用于多物体、堆叠物体、未知不规则物体、物体位姿随机等多种抓取场景,其抓取准确率和探测速度较其他基准方法有明显提升．整个学习过程历时10天,结果表明使用先验知识引导的学习策略能显著加快学习进程．     

基于区域姿态解算的五指机械手抓取系统设计与实现

针对五指机械手抓取成功率低和抓取任务相对简单的问题，基于区域姿态解算方法设计并实现了一种满足不同复杂任务的五指抓取系统。首先，设计了一种气动五指软爪，该软爪使用多种不同刚度的材料制成，由1根主气管和5根支气管驱动，控制复杂度较低，机械性能和抓取性能良好，适用不同的抓取策略。进一步结合软爪的特点提出了基于区域姿态解算的抓取策略。通过预测人手抓取物体时在物体上的接触区域，求解接触区域与软爪指尖在空间上的姿态解，计算软爪的抓取姿态和关节弯曲角度，该策略能够生成高鲁棒性的抓取姿态。然后，设计了包含大量物体的接触数据集。对数据集中的物体标注人手在抓取操作中指尖的接触区域，并尽可能地去除场景信息，提高了数据集的通用性，可用作基准数据集测试算法性能。最后，设计了一系列实验来验证软爪和抓取策略在复杂场景下的抓取性能，实验结果表明了所设计的五指软爪抓取系统在复杂场景下的有效性和可靠性。     

基于神经网络的工业机器人视觉抓取系统设计

针对机器人示教编程方法导致的工件位置固定、抓取效率低下的问题,研究神经网络在机器人视觉识别与抓取规划中的应用,建立了视觉引导方案,通过YOLOV5神经网络模型开发视觉识别系统,识别物体的种类,同时获取待抓取物体定位点坐标。提出了机器人六点手眼标定原理并进行标定实验,提出了针对俯视图为圆形或长方形物体的定位方法。最后针对3种物体进行了180次的抓取实验,实验的综合平均抓取成功率约为92.8%,验证了视觉识别和抓取机器人系统具备实际应用的可能性,有效提高了抓取效率。     

基于视觉与RFID的机器人自定位抓取算法

为实现机器人灵活的自定位,并使其准确地抓取物体,提出一种基于视觉与无线射频识别(RFID)技术的机器人自定位抓取算法。构建网格化环境,利用RFID技术确定机器人的初始位置、行进路线和方向,使用视觉系统获取物体的空间坐标,将其转换到手臂坐标系,采用改进的D-H模型对手臂进行建模,并给出机械臂逆解抓取算法。实验结果表明,该算法使得机器人定位的成功率达到76.7%,抓取成功率高达90%。     

A strategy for fast grasping of unknown objects using partial shape information from range sensors

In this paper, we present a strategy for fast grasping of unknown objects based on the partial shape information from range sensors for a mobile robot with a parallel-jaw gripper. The proposed method can realize fast grasping of an unknown object without needing complete information of the object or learning from grasping experience. Information regarding the shape of the object is acquired by a 2D range sensor installed on the robot at an inclined angle to the ground. Features for determining the maximal contact area are extracted directly from the partial shape information of the unknown object to determine the candidate grasping points. Note that since the shape and mass are unknown before grasping, a successful and stable grasp cannot be in fact guaranteed. Thus, after performing a grasping trial, the mobile robot uses the 2D range sensor to judge whether the object can be lifted. If a grasping trial fails, the mobile robot will quickly find other candidate grasping points for another trial until a successful and stable grasp is realized. The proposed approach has been tested in experiments, which found that a mobile robot with a parallel-jaw gripper can successfully grasp a wide variety of objects using the proposed algorithm. The results illustrate the validity of the proposed algorithm in term of the grasping time.     

抓取任务中的融差控制方法

依据“融差性思维”,提出了无需精确感知依旧可以在一定范围内有效工作的融差控制方法。具体分析了融差抓取方法如何运用相同控制量实现不同抓取任务的工作原理,这一原理使得融差抓取方法在面对一大类抓取任务时,不需要知道物体的具体参数,只需要知道这一大类物体的边界条件。进一步分析了融差抓取方法在欠驱动手爪上的适用性,并发现了欠驱动手爪的局限性。实验表明,在控制量设定不变的情况下,依据融差抓取方法,柔性手爪可以抓住且不抓坏宽度范围为5～45 mm的嫩豆腐,且能够成功抓取宽度范围为5～60 mm的硬质长方体;弹簧关节欠驱动手爪可以抓住且不抓坏宽度范围为20～40 mm的嫩豆腐,且能够成功抓取宽度范围为5～60 mm的硬质长方体。这体现了融差抓取方法的通用性和欠驱动手爪在抓取柔性物体时的局限性。最后,展示了柔性手爪使用融差抓取方法在桌面抓取应用中以简单的控制策略成功抓取不同形状、不同材质的物体。这充分说明了融差抓取方法不依赖于精确的对象感知及物体模型,能够简化控制策略。     

基于深度强化学习的固体放射性废物抓取方法研究

针对固体放射性废物分拣作业中,放射性废物杂乱无序、远程遥操作抓取效率低、人工分拣危险性大等典型问题,提出一种基于深度强化学习的放射性固体废物抓取方法。该方法使用改进深度Q网络算法,通过获取的图像信息,使机器人与环境不断进行交互并获得回报奖励,回报奖励由机械臂动作执行结果和放射性区域内放射性活度的高低构成,根据◢Q◣值的大小得到机械臂的最佳抓取位置。用V-REP软件对UR5机械臂建立仿真模型,在仿真环境中完成不同类型固体放射性废物抓取的训练与测试。仿真结果表明,固体废物在松散放置时该方法可使机械臂抓取成功率大于90%,在紧密放置时抓取成功率大于65%,机械臂不会受到废物堆叠的影响,并且会优先抓取放射性区域内具有高放射性活度的物体。     

Active learning of visual descriptors for grasping using non-parametric smoothed beta distributions

One of the basic skills for a robot autonomous grasping is to select the appropriate grasping point for an object. Several recent works have shown that it is possible to learn grasping points from different types of features extracted from a single image or from more complex 3D reconstructions. In the context of learning through experience, this is very convenient, since it does not require a full reconstruction of the object and implicitly incorporates kinematic constraints as the hand morphology. These learning strategies usually require a large set of labeled examples which can be expensive to obtain. In this paper, we address the problem of actively learning good grasping points to reduce the number of examples needed by the robot. The proposed algorithm computes the probability of successfully grasping an object at a given location represented by a feature vector. By autonomously exploring different feature values on different objects, the systems learn where to grasp each of the objects. The algorithm combines beta–binomial distributions and a non-parametric kernel approach to provide the full distribution for the probability of grasping. This information allows to perform an active exploration that efficiently learns good grasping points even among different objects. We tested our algorithm using a real humanoid robot that acquired the examples by experimenting directly on the objects and, therefore, it deals better with complex (anthropomorphic) hand–object interactions whose results are difficult to model, or predict. The results show a smooth generalization even in the presence of very few data as is often the case in learning through experience.     

Primitive Object Grasping for Finger Motion Synthesis

We developed a new framework to generate hand and finger grasping motions. The proposed framework provides online adaptation to the position and orientation of objects and can generate grasping motions even when the object shape differs from that used during motion capture. This is achieved by using a mesh model, which we call primitive object grasping (POG), to represent the object grasping motion. The POG model uses a mesh deformation algorithm that keeps the original shape of the mesh while adapting to varying constraints. These characteristics are beneficial for finger grasping motion synthesis that satisfies constraints for mimicking the motion capture sequence and the grasping points reflecting the shape of the object. We verify the adaptability of the proposed motion synthesizer according to its position/orientation and shape variations of different objects by using motion capture sequences for grasping primitive objects, namely, a sphere, a cylinder, and a box. In addition, a different grasp strategy called a three‐finger grasp is synthesized to validate the generality of the POG‐based synthesis framework.     

杂乱场景下小物体抓取检测研究

目的 杂乱场景下的物体抓取姿态检测是智能机器人的一项基本技能。尽管六自由度抓取学习取得了进展,但先前的方法在采样和学习中忽略了物体尺寸差异,导致在小物体上抓取表现较差。方法 提出了一种物体掩码辅助采样方法,在所有物体上采样相同的点以平衡抓取分布,解决了采样点分布不均匀问题。此外,学习时采用多尺度学习策略,在物体部分点云上使用多尺度圆柱分组以提升局部几何表示能力,解决了由物体尺度差异导致的学习抓取操作参数困难问题。通过设计一个端到端的抓取网络,嵌入了提出的采样和学习方法,能够有效提升物体抓取检测性能。结果 在大型基准数据集GraspNet-1Billion上进行评估,本文方法取得对比方法中的最优性能,其中在小物体上的抓取指标平均提升了7%,大量的真实机器人实验也表明该方法具有抓取未知物体的良好泛化性能。结论 本文聚焦于小物体上的抓取,提出了一种掩码辅助采样方法嵌入到提出的端到端学习网络中,并引入了多尺度分组学习策略提高物体的局部几何表示,能够有效提升在小尺寸物体上的抓取质量,并在所有物体上的抓取评估结果都超过了对比方法。     

Demonstration-based learning and control for automatic grasping

We present a method for automatic grasp generation based on object shape primitives in a Programming by Demonstration framework. The system first recognizes the grasp performed by a demonstrator as well as the object it is applied on and then generates a suitable grasping strategy on the robot. We start by presenting how to model and learn grasps and map them to robot hands. We continue by performing dynamic simulation of the grasp execution with a focus on grasping objects whose pose is not perfectly known.     

Learning how to grasp based on neural network retraining

Humans have an incredible capacity to manipulate objects using dextrous hands. A large number of studies indicate that robot learning by demonstration is a promising strategy to improve robotic manipulation and grasping performance. Concerning this subject we can ask: How does a robot learn how to grasp? This work presents a method that allows a robot to learn new grasps. The method is based on neural network retraining. With this approach we aim to enable a robot to learn new grasps through a supervisor. The proposed method can be applied for 2D and 3D cases. Extensive object databases were generated to evaluate the method performance in both 2D and 3D cases. A total of 8100 abstract shapes were generated for 2D cases and 11700 abstract shapes for 3D cases. Simulation results with a computational supervisor show that a robotic system can learn new grasps and improve its performance through the proposed HRH (Hopfield-RBF-Hopfield) grasp learning approach.