基于未知物体三维点云特征的机器人六自由度抓取

针对非结构化环境中任意位姿的未知物体,提出了一种基于点云特征的机器人六自由度抓取位姿检测方法,以解决直接从点云中获取目标抓取位姿的难题.首先,根据点云的基本几何信息生成抓取候选,并通过力平衡等方法优化这些候选;然后,利用可直接处理点云的卷积神经网络ConvPoint评估样本,得分最高的抓取将被执行,其中抓取位姿采样和评估网络都是以原始点云作为输入;最后,利用仿真和实际抓取实验进行测试.结果表明,该方法在常用对象上实现了88.33%的抓取成功率,并可以有效地拓展到抓取其他形状的未知物体.     

基于点云采样权重估计的未知物体抓取位姿生成方法

针对机械臂在非结构环境中对未知物体抓取位姿生成困难及抓取稳定性差的问题,提出一种基于点云采样权重估计的抓取位姿生成方法.首先通过移动深度相机的方式拼接得到较完整的物体点云信息,并对物体的几何特性进行分析,有效避开物体不宜抓取的位置进行抓取位姿样本生成;然后结合几何约束条件实现抓取位姿搜索,并利用力封闭条件对样本稳定性进行评估;最后为了对实际的抓取位姿进行评价,根据其稳定性、夹取深度、夹取角度等设定抓取可行性指标,据此在工作空间输出最佳抓取位姿并完成指定的抓取任务.实验结果表明,采用所提方法能够高效生成大量且稳定的抓取位姿,并在仿真环境中有效实现机械臂对单个或多个随机摆放的未知物体的抓取任务.     

基于多尺度特征融合的抓取位姿估计

抓取目标多样性、位姿随机性严重制约了机器人抓取的任务适应性,为提高机器人抓取成功率,提出一种融合多尺度特征的机器人抓取位姿估计方法.该方法以RGD信息为输入,采用ResNet-50主干网络,融合FPN(feature pyramid networks)获得多尺度特征作为抓取生成网络的输入,以生成抓取候选框;并将抓取方向...     

基于上下文聚合策略的轻量级编/解码抓取位姿检测

针对多样性目标在非结构化环境中的抓取位姿难以估计的问题,提出一种基于上下文聚合策略的轻量级编/解码抓取位姿检测网络。首先,以编/解码网络架构为基础,利用深度可分离卷积层与混洗单元构建目标特征深度分离-融合提取块,减少编码网络参数量,增强网络对抓取区域特征的提取能力;其次,利用双线性插值法和深度可分离卷积层建立深度分离-重构块,在恢复高层特征丢失信息的同时,有效减少解码网络的参数量;最后,针对可抓取区域像素点与目标物体全貌之间的非一致性问题,基于交叉熵辅助损失和自注意力机制,提出一种抓取区域上下文聚合策略,引导网络增强可抓取目标区域特征的表征能力,抑制非抓取像素点的冗余特征。实验结果表明,所提网络在Cornell数据集的图像拆分与对象拆分子集上抓取检测准确率分别可达97.8%与93.8%,单张图像检测速度可达64.93张/秒;在Jacquard数据集上抓取检测准确率可达95.1%,单张图像检测速度可达60.6张/秒。与对比网络相比,所提网络不仅计算量与参数量较小,而且抓取检测的准确率与速度均有明显提升,在真实场景下对9种物体的抓取检测验证中,抓取成功率达到93.3%。     

基于级联卷积神经网络的机器人平面抓取位姿快速检测

针对任意姿态的未知不规则物体,提出一种基于级联卷积神经网络的机器人平面抓取位姿快速检测方法．建立了一种位置-姿态由粗到细的级联式两阶段卷积神经网络模型,利用迁移学习机制在小规模数据集上训练模型,以R-FCN（基于区域的全卷积网络）模型为基础提取抓取位置候选框进行筛选及角度粗估计,并针对以往方法在姿态检测上的精度不足,提出一种Angle-Net模型来精细估计抓取角度．在Cornell数据集上的测试及机器人在线抓取实验结果表明,该方法能够对任意姿态、不同形状的不规则物体快速计算最优抓取点及姿态,其识别准确性和快速性相比以往方法有所提高,鲁棒性和稳定性强,且能够泛化适应未训练过的新物体．     

面向机器人抓取过程中目标位姿估计方法

针对机械臂抓取过程中场景的复杂性和存在遮挡问题,提出了一种基于深度相机的物体位姿估计方法。采用Kinect相机获取点云图像,提取点云的FPFH特征。采用奇异值分解算法和随机一致算法来进行位姿估计。将得到的位姿经过手眼转换转换为抓取位姿。通过与ICP算法和NDT算法进行对比实验,结果验证了该方法的稳定性和精确性。     

基于三级卷积神经网络的物体抓取检测

借鉴人类抓取物体的特点,提出一种三级串联卷积神经网络用于物体抓取框的检测,实现了对未知物体的高准确度抓取．在所提出的三级串联卷积神经网络中：第1级用于物体的初步定位,为下一级卷积神经网络搜索抓取框确定位置;第2级用于获取预选抓取框,以较小的网络获取较少的特征,从而快速地找出物体的可用抓取框,剔除不可用的抓取框;第3级用于重新评判预选抓取框,以较大的网络获取较多的特征,从而准确地评估每个预选抓取框,获取最佳抓取框．测试结果表明,与单一卷积神经网络相比,三级网络获得抓取框的正确率提高了6.1%,最终在实际Youbot机器人上实现了高准确度的抓取操作．     

面向机械臂的抓取位姿检测

针对结构化环境和单一已知目标的抓取方面,搭载末端夹持器机械臂抓取技术取得了显著进展。但当工作场景环境不明、工作目标形状各异时,机械臂容易受环境光照变化、物体遮挡的影响,为此,本文深入研究了机械臂抓取系统流程和抓取方法,设计了基于卷积神经网络自主检测机械臂抓取位姿的方法。     

结合增量学习的物体抓取检测框架

为了解决未知类别物体的抓取问题,提出了一种结合增量学习的物体抓取检测框架,该框架分为抓取学习和增量学习两个阶段.在第1阶段,对已知的物体使用密集注意力网络进行训练,该网络利用注意力机制对特征通道和密集残差连接之间的关系进行建模.在第2阶段,引入了聚类优先样本选择策略,该策略会挑选出那些与其聚类质心距离相近的样本,用这些新样本替换掉示例集中的部分旧样本进行训练.此外在未知类别物体上训练网络时,还引入了蒸馏损失,以保留之前在已知类中学到的知识.通过在Jacquard数据集和UR10e机器人上进行的实验,表明了该方法在抓取未知类别物体方面有一定的可行性和有效性,克服了机器人在抓取未知类别物体上的缺陷.     

Collaborative Pushing and Grasping of Tightly Stacked Objects via Deep Reinforcement Learning

Directly grasping the tightly stacked objects may cause collisions and result in failures,degenerating the functionality of robotic arms.Inspired by the observation that first pushing objects to a state of mutual separation and then grasping them individually can effectively increase the success rate,we devise a novel deep Q-learning framework to achieve collaborative pushing and grasping.Specifically,an efficient non-maximum suppression policy(PolicyNMS)is proposed to dynamically evaluate pushing and grasping actions by enforcing a suppression constraint on unreasonable actions.Moreover,a novel data-driven pushing reward network called PR-Net is designed to effectively assess the degree of separation or aggregation between objects.To benchmark the proposed method,we establish a dataset containing common household items dataset(CHID)in both simulation and real scenarios.Although trained using simulation data only,experiment results validate that our method generalizes well to real scenarios and achieves a 97%grasp success rate at a fast speed for object separation in the real-world environment.     

基于力矩外环的软物体抓取控制

为减少机械手抓取软物体造成的损害,提出基于关节力矩预测的力矩外环在线抓取控制方案。该方案通过建立灰色预测模型直接预测关节力矩的变化代替预测软物体刚度的变化;并能同时控制机械手抓取过程中关节力矩偏差和角速度关系。在实际关节力矩和灰色预测力矩之间采用线性插值,按采样周期将预测力矩逐渐加到力矩回路中。仿真表明,当抓取具有刚度变化的软物体时,基于关节力矩预测的力矩外环软物体抓取控制系统对机械手和软物体之间的动态接触过程具有一定的自适应性,可以快速跟踪设定值,而且无大的超调和振荡出现,使控制性能和动态品质得到较大改善,具有较强的鲁棒性,是适合软物体抓取的一种控制方法。     

基于自由度分析的三维几何约束推理求解

三维几何约束求解对于装配设计、装配工艺规划和并行工程等多个领域的研究具有重要意义。以自由度分析为基础，对冗余约束和约束冲突的情况进行判断。采用几何推理的方法，直接从零件上的点、线、面、圆柱面、球面等基本几何元素出发，对这些基本几何元素之间的位置关系进行分析，求解三维几何约束。避免了数值解法中对初始值的依赖和迭代过程的不稳定，提高了三维几何约束求解的速度和稳定性。     

Pose Estimation in Conformal Geometric Algebra Part I: The Stratification of Mathematical Spaces

2D-3D pose estimation means to estimate the relative position and orientation of a 3D object with respect to a reference camera system. This work has its main focus on the theoretical foundations of the 2D-3D pose estimation problem: We discuss the involved mathematical spaces and their interaction within higher order entities. To cope with the pose problem (how to compare 2D projective image features with 3D Euclidean object features), the principle we propose is to reconstruct image features (e.g. points or lines) to one dimensional higher entities (e.g. 3D projection rays or 3D reconstructed planes) and express constraints in the 3D space. It turns out that the stratification hierarchy [11] introduced by Faugeras is involved in the scenario. But since the stratification hierarchy is based on pure point concepts a new algebraic embedding is required when dealing with higher order entities. The conformal geometric algebra (CGA) [24] is well suited to solve this problem, since it subsumes the involved mathematical spaces. Operators are defined to switch entities between the algebras of the conformal space and its Euclidean and projective subspaces. This leads to another interpretation of the stratification hierarchy, which is not restricted to be based solely on point concepts. This work summarizes the theoretical foundations needed to deal with the pose problem. Therefore it contains mainly basics of Euclidean, projective and conformal geometry. Since especially conformal geometry is not well known in computer science, we recapitulate the mathematical concepts in some detail. We believe that this geometric model is useful also for many other computer vision tasks and has been ignored so far. Applications of these foundations are presented in Part II [36].Bodo Rosenhahn gained his diploma degree in Computer Science in 1999. Since then he has been pursuing his Ph.D. at the Cognitive Systems Group, Institute of Computer Science, Christian-Albrechts University Kiel, Germany. He is working on geometric applications of Clifford algebras in computer vision.Prof. Dr. Gerald Sommer received a diploma degree in physics from the Friedrich-Schiller-Universität Jena, Germany, in 1969, a Ph.D. degree in physics from the same university in 1975, and a habilitation degree in engineering from the Technical University Ilmenau, Germany, in 1988. Since 1993 he is leading the research group Cognitive Systems at the Christian-Albrechts-Universität Kiel, Germany. Currently he is also the scientific coordinator of the VISATEC project.     

Estimation of 2D Displacement Field Based on Affine Geometric Invariance and Scene Constraints

In the paper, a novel approach using affine transfer and scene constraint for estimation of 2D displacement field was developed. In this approach, we derived a system of 5 linear equations for computing corresponding point of any image point via the utilisation of affine invariant. Subsequently, the characteristics of such a linear system was thoroughly studied, and the way to obtain a reliable solution of the system via robust least squares (RLS) approach, in conjunction with total least squares technique, was proposed. In addition, through the interpretation of the approach from both geometric point of view and numerical point of view, we gave the limitation of the algorithm. The limitation was then relaxed to a certain extent by using full fundamental matrix. These findings were further verified through the experimental results.     

Dynamics and Stability of Blind Grasping of a 3-Dimensional Object under Non-holonomic Constraints

A mathematical model expressing the motion of a pair of multi-DOF robot fingers with hemi-spherical ends, grasping a 3-D rigid object with parallel flat surfaces, is derived, together with non-holonomic constraints. By referring to the fact that humans grasp an object in the form of precision prehension, dynamically and stably by opposable forces, between the thumb and another finger (index or middle finger), a simple control signal constructed from finger-thumb opposition is proposed, and shown to realize stable grasping in a dynamic sense without using object information or external sensing (this is called "blind grasp" in this paper). The stability of grasping with force/torque balance under non-holonomic constraints is analyzed on the basis of a new concept named "stability on a manifold". Preliminary simulation results are shown to verify the validity of the theoretical results.     

虚拟环境中灵巧手主从抓持的实现

研究了虚拟现实环境中人手和灵巧手的抓持动作.利用数据手套采集人手的运动信息,将人手的运动 映射给灵巧手,通过搭建人手和灵巧手的模型,在虚拟环境下实现了主从抓持操作.探讨了关键技术问题:异构系统 运动映射、碰撞检测、虚拟力建模、稳定抓持的判据.􀁱     

基于改进头姿估计方法的机器人轮椅交互控制

针对现有迭代最近点（ICP）头姿估计算法存在迭代次数偏多且易陷于局部最优、而随机森林（RF）头姿估计算法准确性和稳定性不高的问题,提出一种新的头姿估计改进方法,并基于该改进方法构建机器人轮椅实时交互控制接口．首先,分析现有迭代最近点头姿算法与随机森林头姿算法在准确性、实时性及稳定性方面存在的问题,并提出一种新的基于随机森林与迭代最近点算法融合的头姿估计改进方法;其次,为实现头姿估计到机器人轮椅交互控制的无缝连接,建立基于传统机器人轮椅操纵杆的头部姿态运动空间映射;最后,在基于标准头姿数据库分析改进头姿估计方法性能的基础上,构建机器人轮椅实验平台并规划运动轨迹,以进一步验证基于改进头姿估计方法的人机交互接口在机器人轮椅实时控制方面的有效性．实验结果表明,改进后的头姿估计方法较传统迭代最近点算法减少了迭代次数且避免了陷于局部最优,在仅增加少量运算时间的基础上,其准确性和稳定性都优于传统随机森林算法;同时,基于改进头姿估计方法的人机交互接口亦能实时平稳地控制机器人轮椅沿既定的轨迹运动．     

虚拟手抓持规则融合策略研究

群体虚拟手抓持规则是虚拟手和虚拟物体进行抓持操作的交互规则,用于判定虚拟手是否能够成功抓持物体。对基于几何的虚拟手抓持规则和基于物理的虚拟手抓持规则分别进行了研究,针对基于几何的虚拟手抓持规则规则简单、仿真效果较差,基于物理模型的虚拟手抓持规则计算复杂、难以实现实时仿真的问题:（1）改进基于几何的虚拟手抓持规则,通过接触点位置、法矢和抓持面法矢制定抓持规则,使其效果逼近力封闭虚拟手抓持规则;（2）利用力封闭计算中抓持接触点和法矢不变的特性,通过内力配比避免了抓持操作中的非线性规划求解,使抓持操作阶段实现实时仿真;（3）通过几何约束进行初始抓持判断-力封闭计算校正-内力配比力封闭计算的策略,实现了完整的抓持过程实时仿真。设计的交互实验说明该抓持规则能实现高沉浸感和实时性的抓持仿真,可以应用到虚拟训练、虚拟装配等仿真平台。