基于多视角RGB-D图像帧数据融合的室内场景理解

对于智能机器人来说,正确地理解环境是一项非常重要且充满挑战性的能力,从而成为机器人学领域一个关键问题.随着服务机器人进入家庭成为趋势,让机器人能够依靠自身搭载的传感器和场景理解算法,以自主、可靠的方式感知并理解其所处的环境,识别环境中的各类物体及其相互关系,并建立环境模型,成为自主完成任务和实现人-机器人智能交互的前提.在规模较大的室内空间中,由于机器人常用的RGB-D(RGB depth)视觉传感器(同时获取彩色图像和深度信息)视野有限,使之难以直接获取包含整个区域的单帧图像,但机器人能够运动到不同位置,采集多种视角的图像数据,这些数据总体上能够覆盖整个场景.在此背景下,提出了基于多视角RGB-D图像帧信息融合的室内场景理解算法,在单帧RGB-D图像上进行物体检测和物体关系提取,在多帧RGB-D图像上进行物体实例检测,同时构建对应整个场景的物体关系拓扑图模型.通过对RGB-D图像帧进行划分,提取图像单元的颜色直方图特征,并提出基于最长公共子序列的跨帧物体实例检测方法,确定多帧图像之间的物体对应关联,解决了RGB-D摄像机视角变化影响图像帧融合的问题.最后,在NYUv2(NYU depth dataset v2)数据集上验证了本文算法的有效性.     

机器人的CP-nets优化类人轨迹规划

机器人移动轨迹按照人的手臂来模拟是提高机器人安全性和人机交互能力的有效方法,特别是针对机器人抓取路径不唯一的场合,类人行为对于人机系统表现更加自然。此前,通常利用Kinect等设备,基于人工神经网络和K近邻算法等智能算法对类人轨迹进行规划,但无法获得未采样过的最优轨迹。本文基于CP-nets采用偏好模型研究类人运动轨迹,然后将该模型应用于机器人控制,在没有采样的情况下,也可得到最优的类人轨迹。实验结果表明,基于CP-nets 的类人规划轨迹具有较高的效率和舒适性,符合人的运动特征。     

A Novel Approach for Mobile Robot Navigation with Dynamic Obstacles Avoidance

This paper proposes a new approach for trajectory optimization of a mobile robot in a general dynamic environment. The new method combines the static and dynamic modes of trajectory planning to provide an algorithm that gives fast and optimal solutions for static environments, and generates a new path when an unexpected situation occurs. The particularity of the method is in the representation of the static environment in a judicious way facilitating the path planning and reducing the processing time. Moreover, when an unexpected obstacle blocks the robot trajectory, the method uses the robot sensors to detect the obstacle, finds a best way to circumvent it and then resumes its path toward the desired destination. Experimental results showed the effectiveness of the proposed approach.     

利用法向矢量计算三维物体间的距离

机器人路径规划和装配路径规划都要计算空间两个物体之间的距离，实时仿真系统则要求算法既简便、快速，又能够支持规划算法．本文提出了基于Ｂ－Ｒｅｐ的计算三维物体间距离的方法，即利用物体的表面法向矢量直接计算三维物体间的定向距离．该方法达到了上述系统对物体间距离的检测要求．     

Driving on Point Clouds: Motion Planning,Trajectory Optimization,and Terrain Assessment in Generic Nonplanar Environments

We present a practical approach to global motion planning and terrain assessment for ground robots in generic three‐dimensional (3D) environments, including rough outdoor terrain, multilevel facilities, and more complex geometries. Our method computes optimized six‐dimensional trajectories compliant with curvature and continuity constraints directly on unordered point cloud maps, omitting any kind of explicit surface reconstruction, discretization, or topology extraction. We assess terrain geometry and traversability on demand during motion planning, by fitting robot‐sized planar patches to the map and analyzing the local distribution of map points. Our motion planning approach consists of sampling‐based initial trajectory generation, followed by precise local optimization according to a custom cost measure, using a novel, constraint‐aware trajectory optimization paradigm. We embed these methods in a complete autonomous navigation system based on localization and mapping by means of a 3D laser scanner and iterative closest point matching, suitable for both static and dynamic environments. The performance of the planning and terrain assessment algorithms is evaluated in offline experiments using recorded and simulated sensor data. Finally, we present the results of navigation experiments in three different environments—rough outdoor terrain, a two‐level parking garage, and a dynamic environment, demonstrating how the proposed methods enable autonomous navigation in complex 3D terrain.     

Efficient models for grasp planning with a multi-fingered hand

This paper presents a simple grasp planning method for a multi-fingered hand. Its purpose is to compute a context-independent and dense set or list of grasps, instead of just a small set of grasps regarded as optimal with respect to a given criterion. By context-independent, we mean that only the robot hand and the object to grasp are considered. The environment and the position of the robot base with respect to the object are considered in a further stage. Such a dense set can be computed offline and then used to let the robot quickly choose a grasp adapted to a specific situation. This can be useful for manipulation planning of pick-and-place tasks. Another application is human–robot interaction when the human and robot have to hand over objects to each other. If human and robot have to work together with a predefined set of objects, grasp lists can be employed to allow a fast interaction.The proposed method uses a dense sampling of the possible hand approaches based on a simple but efficient shape feature. As this leads to many finger inverse kinematics tests, hierarchical data structures are employed to reduce the computation times. The data structures allow a fast determination of the points where the fingers can realize a contact with the object surface. The grasps are ranked according to a grasp quality criterion so that the robot will first parse the list from best to worse quality grasps, until it finds a grasp that is valid for a particular situation.     

基于RGB-D信息的移动机器人SLAM和路径规划方法研究与实现

移动机器人在各种辅助任务中需具备自主定位、建图、路径规划与运动控制的能力。本文利用RGB-D信息和ORB-SLAM算法进行自主定位,结合点云数据和GMapping算法建立环境栅格地图,基于二次规划方法进行平滑可解析的路径规划,并设计非线性控制器,实现了由一个运动底盘、一个RGB-D传感器和一个运算平台组成的自主移动机器人系统。经实验验证,这一系统实现了复杂室内环境下的实时定位与建图、自主移动和障碍物规避。由此,为移动机器人的推广应用提供了一个硬件结构简单、性能良好、易扩展、经济性好、开发维护方便的解决方案。     

基于自适应动态窗口改进细菌算法与移动机器人路径规划

针对移动机器人在复杂环境下的路径规划问题,提出一种新的自适应动态窗口改进细菌算法,并将新算法应用于移动机器人路径规划。改进细菌算法继承了细菌算法与动态窗口算法（dynamic window algorithm, DWA）在避障时的优点,能较好实现复杂环境中移动机器人静态和动态避障。该改进算法主要分三步完成移动机器人路径规划。首先,利用改进细菌趋化算法在静态环境中得到初始参考规划路径。接着,基于参考路径,机器人通过自身携带的传感器感知动态障碍物进行动态避障并利用自适应DWA完成局部动态避障路径规划。最后,根据移动机器人局部动态避障完成情况选择算法执行步骤,如果移动机器人能达到最终目标点,结束该算法,否则移动机器人再重回初始路径,直至到达最终目标点。仿真比较实验证明,改进算法无论在收敛速度还是路径规划精确度方面都有明显提升。     

Visibility-based spatial reasoning for object manipulation in cluttered environments

In this paper, we present visibility-based spatial reasoning techniques for real-time object manipulation in cluttered environments. When a robot is requested to manipulate an object, a collision-free path should be determined to access, grasp, and move the target object. This often requires processing of time-consuming motion planning routines, making real-time object manipulation difficult or infeasible, especially in a robot with a high DOF and/or in a highly cluttered environment. This paper places special emphasis on developing real-time motion planning, in particular, for accessing and removing an object in a cluttered workspace, as a local planner that can be integrated with a general motion planner for improved overall efficiency. In the proposed approach, the access direction of the object to grasp is determined through visibility query, and the removal direction to retrieve the object grasped by the gripper is computed using an environment map. The experimental results demonstrate that the proposed approach, when implemented by graphics hardware, is fast and robust enough to manipulate 3D objects in real-time applications.     

Optimal path generation for a simulated autonomous mobile robot

The paper deals with a set of algorithms including path planning, trajectory planning, and path tracking for a tricycle type wheeled mobile robot. Path planning is carried out with parametric polynomial interpolation using an optimization algorithm based on robot geometric constraints. Trajectory characteristics are then derived from the planned geometric path with time varying parameters. A sliding mode control algorithm combined with an adaptive control law are used to track the planned trajectory. The technique deals with an environment free of obstacles. However, it can be easily integrated in a piecewise non colliding path generation. Simulation results are presented to show the validity of the different algorithms.Bissé Emmanuel is a Ph.D. student at the École Polytechnique de Montréal, Department of Mechanical Engineering.Bentounes Mohamed is a Ph.D. student at the École Polytechnique de Montréal, Department of Mechanical Engineering.Boukas El-Kébir is a Professor at the École Polytechnique de Montréal, Department of Mechanical Engineering.     

矢量场逐次逼近的康复机器人柔顺交互控制

为了实现康复机器人的主动柔顺交互,提出了一种基于矢量场逐次逼近的控制模型;设计了矢量场逐次逼近系统,可输出机器人关节期望位移,该输出能与输入的扭矩、表面肌电及脑电等信号在振幅、频率和相位上保持同步,且通过调节遗忘因子参数值,可改变主动柔顺交互的积极性;利用自行设计的穿着型下肢康复机器人样机进行柔顺辅助实验,以验证所提出控制模型的有效性;通过FFT（Fast Fourier transformation）频谱对机器人关节扭矩的组成成分进行了分析,并采用基于最小二乘法的参数辨识方法实施了重力补偿,以便康复机器人实时控制.实验结果表明,该控制模型对于实现康复机器人与人之间的柔顺交互是有效的.     

自主路径规划的同步机械臂的研究与实现

为了解决卧床老人或者病人无人照顾,并且提高其生活自理能力的问题,提出一种具有自主路径规划的同步机械臂的研究与实现。使用者通过手机APP指定移动机械臂到达目的地后,通过手臂上的同步装置指导机械臂同步执行本体手臂行为,进行物体的抓取工作。采用密集无源RFID标签定位方法在室内布置4*4m^2的RFID标签阵列,基于模糊逻辑控制进行路径规划,实验结果表明本系统可以帮助使用者完成生活中90%的抓取工作。     

Perceiving,learning, and exploiting object affordances for autonomous pile manipulation

Autonomous manipulation in unstructured environments will enable a large variety of exciting and important applications. Despite its promise, autonomous manipulation remains largely unsolved. Even the most rudimentary manipulation task—such as removing objects from a pile—remains challenging for robots. We identify three major challenges that must be addressed to enable autonomous manipulation: object segmentation, action selection, and motion generation. These challenges become more pronounced when unknown man-made or natural objects are cluttered together in a pile. We present a system capable of manipulating unknown objects in such an environment. Our robot is tasked with clearing a table by removing objects from a pile and placing them into a bin. To that end, we address the three aforementioned challenges. Our robot perceives the environment with an RGB-D sensor, segmenting the pile into object hypotheses using non-parametric surface models. Our system then computes the affordances of each object, and selects the best affordance and its associated action to execute. Finally, our robot instantiates the proper compliant motion primitive to safely execute the desired action. For efficient and reliable action selection, we developed a framework for supervised learning of manipulation expertise. To verify the performance of our system, we conducted dozens of trials and report on several hours of experiments involving more than 1,500 interactions. The results show that our learning-based approach for pile manipulation outperforms a common sense heuristic as well as a random strategy, and is on par with human action selection.     

Depth-assisted rectification for real-time object detection and pose estimation

RGB-D sensors have become in recent years a product of easy access to general users. They provide both a color image and a depth image of the scene and, besides being used for object modeling, they can also offer important cues for object detection and tracking in real time. In this context, the work presented in this paper investigates the use of consumer RGB-D sensors for object detection and pose estimation from natural features. Two methods based on depth-assisted rectification are proposed, which transform features extracted from the color image to a canonical view using depth data in order to obtain a representation invariant to rotation, scale and perspective distortions. While one method is suitable for textured objects, either planar or non-planar, the other method focuses on texture-less planar objects. Qualitative and quantitative evaluations of the proposed methods are performed, showing that they can obtain better results than some existing methods for object detection and pose estimation, especially when dealing with oblique poses.     

Optimal rendezvous-point selection for robotic interception ofmoving objects

A number of active prediction planning and execution (APPE) systems have recently been proposed for robotic interception of moving objects. The cornerstone of such systems is the selection of a robot-object rendezvous-point on the predicted object trajectory. Unlike tracking-based systems, which minimize the state difference between the object and the robot at each control period, in this methodology the robot is sent directly to the selected rendezvous-point. A fine-motion tracking strategy would then be employed for grasping the moving object. Herein, a novel strategy for selecting the optimal (earliest) rendezvous-point is presented. For objects with predictable trajectories, this is a significant improvement over previous APPE strategies which select the rendezvous-point from a limited number of non-optimally chosen candidates.     

A Generalized Real-Time Obstacle Avoidance Method Without the Cspace Calculation

An important concept proposed in the early stage of robot path planning field is the shrinking of a robot to a point and meanwhile the expanding of obstacles in the workspace as a set of new obstacles. The resulting grown obstacles are called the Configuration Space (Cspace) obstacles. The find-path problem is then transformed into that of finding a collision-free path for a point robot among the Cspace obstacles. However, the research experiences have shown that the Cspace transform is very hard when the following situations occur: 1) both the robot and obstacles are not polygons, and 2) the robot is allowed to rotate. This situation gets even worse when the robot and obstacles are three dimensional (3D) objects with various shapes. For this reason, direct path planning approaches without the Cspace transformation is quite useful and expected.Motivated by the practical requirements of robot path planning, a generalized constrained optimization problem (GCOP) with not only logic AND but also logic OR relationships was proposed and a mathematical solution developed previously. This paper inherits the fundamental ideas of inequality and optimization techniques from the previous work, converts the obstacle avoidance problem into a semi-infinite constrained optimization problem with the help of the mathematical transformation, and proposes a direct path planning approach without Cspace calculation, which is quite different from traditional methods. To show its merits, simulation results in 3D space have been presented.     

Augmented reality-based robot teleoperation system using RGB-D imaging and attitude teaching device

Augmented reality (AR)-based programming using the demonstration method has been widely studied. However, studies on AR-based programming for remote robots are lacking because of the limitation of human–computer interaction. This paper proposes an AR-based robot teleoperation system and method using RGB-D imaging and an attitude teaching device. By sending the color and depth images of the remote robot environment to the local side, the operators can complete the teleoperation of the robot at the local side. First, the operators select key positions on the motion path of the robot endpoint from color images via a mouse, and the computer calculates the 3D coordinates of these key points in the robot base coordinate system to complete the position teaching process. In the robot attitude teaching process, the AR technology is used to superimpose the virtual robot model onto the color images of the robot teleoperation environment, so as to make the virtual robot endpoint to move along the teaching path. An operator can use the portable attitude teaching device designed in this study to control the robot movement parameters, such as the attitude and motion speed, during the movement of the virtual robot. After the position and attitude teaching processes, the robot movement trajectory can be generated. To make the base coordinate system of the virtual model consistent with that of the physical robot, we propose an online AR registration method, which does not require manually placing the AR registration marker. The proposed AR-based robot teleoperation system can quickly and easily complete robot teleoperation at the local side.     

基于深度学习初始位姿估计的机器人摄影测量视点规划

针对机器人摄影测量中离线规划受初始位姿标定影响的问题, 提出融合初始位姿估计的机器人摄影测量系统视点规划方法. 首先构建基于YOLO (You only look once) 的深度学习网络估计被测对象3D包围盒, 利用PNP (Perspective-N-point)算法快速求解对象姿态; 然后随机生成机器人无奇异无碰撞的视点, 基于相机成像的2D-3D正逆性映射, 根据深度原则计算每个视角下目标可见性矩阵; 最后, 引入熵权法, 以最小化重建信息熵为目标建立优化模型, 并基于旅行商问题(Travelling saleman problem, TSP)模型规划机器人路径. 结果表明, 利用深度学习估计的平移误差低于5 mm, 角度误差低于2°. 考虑熵权的视点规划方法提高了摄影测量质量, 融合深度学习初始姿态的摄影测量系统提高了重建效率. 利用本算法对典型零件进行摄影测量质量和效率的验证, 均获得优异的位姿估计和重建效果. 提出的算法适用于实际工程应用, 尤其是快速稀疏摄影重建, 促进了工业摄影测量速度与自动化程度提升.     

Robust RGB-D simultaneous localization and mapping using planar point features

RGB-D cameras like PrimeSense and Microsoft Kinect are popular sensors in the simultaneous localization and mapping researches on mobile robots because they can provide both vision and depth information. Most of the state-of-the-art RGB-D SLAM systems employ the Iterative Closest Point (ICP) algorithm to align point features, whose spatial positions are computed by the corresponding depth data of the sensors. However, the depth measurements of features are often disturbed by noise because visual features tend to lie at the margins of real objects. In order to reduce the estimation error, we propose a method that extracts and selects the features with reliable depth values, i.e. planar point features. The planar features can benefit the accuracy and robustness of traditional ICP, while holding a reasonable computation cost for real-time applications. An efficient RGB-D SLAM system based on planar features is also demonstrated, with trajectory and map results from open datasets and a physical robot in real-world experiments.