A system to navigate a robot into a ship structure

Abstract. A prototype system has been built to navigate a walking robot into a ship structure. The 8-legged robot is equipped with an active stereo head. From the CAD-model of the ship good view points are selected, such that the head can look at locations with sufficient edge features, which are extracted automatically for each view. The pose of the robot is estimated from the features detected by two vision approaches. One approach searches in stereo images for junctions and measures the 3-D position. The other method uses monocular image and tracks 2-D edge features. Robust tracking is achieved with a method of edge projected integration of cues (EPIC). Two inclinometres are used to stabilise the head while the robot moves. The results of the final demonstration to navigate the robot within centimetre accuracy are given.     

Stereo vision based automation for a bin-picking solution

As flexibility becomes an important factor in factory automation, the bin-picking system, where a robot performs pick-and-place tasks for randomly piled parts in a bin through measuring the 3D pose of an object by a 3D vision sensor, has been actively studied. However, conventional bin-picking systems that are employed for particular tasks are limited by such things as the FOV (Field of View), the shape of landmark features, and computation time. This paper proposes a general-purpose stereo vision based bin-picking system. To detect the workpiece to be picked, a geometric pattern matching (GPM) method with respect to the 2D image with a wide FOV is applied. The accurate 3D pose of a selected workpiece among the pick-up candidates is acquired by measuring the 3D positions of three features in the workpiece using the stereo camera. In order to improve the 3D position estimation performance, the GPM method is also used instead of the stereo matching method. The multiple pattern registration and ellipse fitting techniques are additionally applied to increase the reliability. The grasp position of a workpiece without collision is determined using the pose of the object and the bin information. By using these methods a practical bin-picking strategy is established to operate robustly with minimum help from the human workers in the factory. Through experiments on commercial industrial workpieces and industrial robot, we validated that the proposed vision system accurately measures the 3D pose of part and the robot successfully manipulates the workpiece among randomly stacked parts.     

SLAM in Indoor Environments using Omni-directional Vertical and Horizontal Line Features

An autonomous mobile robot must have the ability to navigate in an unknown environment. The simultaneous localization and map building (SLAM) problem have relation to this autonomous ability. Vision sensors are attractive equipment for an autonomous mobile robot because they are information-rich and rarely have restrictions on various applications. However, many vision based SLAM methods using a general pin-hole camera suffer from variation in illumination and occlusion, because they mostly extract corner points for the feature map. Moreover, due to the narrow field of view of the pin-hole camera, they are not adequate for a high speed camera motion. To solve these problems, this paper presents a new SLAM method which uses vertical lines extracted from an omni-directional camera image and horizontal lines from the range sensor data. Due to the large field of view of the omni-directional camera, features remain in the image for enough time to estimate the pose of the robot and the features more accurately. Furthermore, since the proposed SLAM does not use corner points but the lines as the features, it reduces the effect of illumination and partial occlusion. Moreover, we use not only the lines at corners of wall but also many other vertical lines at doors, columns and the information panels on the wall which cannot be extracted by a range sensor. Finally, since we use the horizontal lines to estimate the positions of the vertical line features, we do not require any camera calibration. Experimental work based on MORIS, our mobile robot test bed, moving at a human’s pace in the real indoor environment verifies the efficacy of this approach.     

一种新颖的基于两个特征点的室内移动机器人定位方法

提出了一种新颖的基于两个特征点的室内移动机器人定位方法。与已有的几何位姿估计方法或航标匹配方法不同，该方法不需要人工航标，也不需要准确的环境地图，只需一幅由传统的CCD相机拍摄的图像。从机器人接近的目标上选取相对于地面等高的两个点作为两个特征点。基于这两点建立一个目标坐标系。在相机平视且这两个特征点与相机投影中心相对于地面不是恰好等高的条件下，就可以根据这两个特征点在图像中的坐标确定机器人相对于目标坐标系的位置和运动方向。该方法非常灵活，适用范围广，可以大大简化机器人定位问题。试验结果表明这一新的方法不仅简单灵活而且具有很高的定位精度。     

低成本激光和视觉相结合的同步定位与建图研究

激光雷达和视觉传感是目前两种主要的服务机器人定位与导航技术,但现有的低成本激光雷 达定位精度较低且无法实现大范围闭环检测,而单独采用视觉手段构建的特征地图又不适用于导航应用。因此,该文以配备低成本激光雷达与视觉传感器的室内机器人为研究对象,提出了一种激光和视觉相结合的定位与导航建图方法：通过融合激光点云数据与图像特征点数据,采用基于稀疏姿态调整的优化方法,对机器人位姿进行优化。同时,采用基于视觉特征的词袋模型进行闭环检测,并进一步优化基于激光点云的栅格地图。真实场景下的实验结果表明,相比于单一的激光或视觉定位建图方 法,基于多传感器数据融合的方法定位精度更高,并有效地解决了闭环检测问题。     

基于语义分割的室内动态场景同步定位与语义建图

针对动态物体在室内同步定位与地图构建（SLAM）系统中影响位姿估计的问题，提出一种动态场景下基于语义分割的SLAM系统。在相机捕获图像后，首先用PSPNet（Pyramid Scene Parsing Network）对图像进行语义分割；之后提取图像特征点，剔除分布在动态物体内的特征点，并用静态的特征点进行相机位姿估计；最后完成语义点云图和语义八叉树地图的构建。在公开数据集上的五个动态序列进行多次对比测试的结果表明，相对于使用SegNet网络的SLAM系统，所提系统的绝对轨迹误差的标准偏差有6.9%~89.8%的下降，平移和旋转漂移的标准偏差在高动态场景中的最佳效果也能分别提升73.61%和72.90%。结果表明，改进的系统能够显著减小动态场景下位姿估计的误差，准确地在动态场景中进行相机位姿估计。     

定位图像匹配尺度与区域的摄像机位姿实时跟踪

目的 提出一种定位图像匹配尺度及区域的有效算法,通过实现当前屏幕图像特征点与模板图像中对应尺度下部分区域中的特征点匹配,实现摄像机对模板图像的实时跟踪,解决3维跟踪算法中匹配精度与效率问题。方法 在预处理阶段,算法对模板图像建立多尺度表示,各尺度下的图像进行区域划分,在每个区域内采用ORB（oriented FAST and rotated BRIEF）方法提取特征点并生成描述子,由此构建图像特征点的分级分区管理模式。在实时跟踪阶段,对于当前摄像机获得的图像,首先定位该图像所对应的尺度范围,在相应尺度范围内确定与当前图像重叠度大的图像区域,然后将当前图像与模板图像对应的尺度与区域中的特征点集进行匹配,最后根据匹配点对计算摄像机的位姿。结果 利用公开图像数据库（stanford mobile visual search dataset）中不同分辨率的模板图像及更多图像进行实验,结果表明,本文算法性能稳定,配准误差在1个像素左右;系统运行帧率总体稳定在2030 帧/s。结论 与多种经典算法对比,新方法能够更好地定位图像匹配尺度与区域,采用这种局部特征点匹配的方法在配准精度与计算效率方面比现有方法有明显提升,并且当模板图像分辨率较高时性能更好,特别适合移动增强现实应用。     

Autonomous robot calibration using vision technology

Unlike the traditional robot calibration methods, which need external expensive calibration apparatus and elaborate setups to measure the 3D feature points in the reference frame, a vision-based self-calibration method for a serial robot manipulator, which only requires a ground-truth scale in the reference frame, is proposed in this paper. The proposed algorithm assumes that the camera is rigidly attached to the robot end-effector, which makes it possible to obtain the pose of the manipulator with the pose of the camera. By designing a manipulator movement trajectory, the camera poses can be estimated up to a scale factor at each configuration with the factorization method, where a nonlinear least-square algorithm is applied to improve its robustness. An efficient approach is proposed to estimate this scale factor. The great advantage of this self-calibration method is that only image sequences of a calibration object and a ground-truth length are needed, which makes the robot calibration procedure more autonomous in a dynamic manufacturing environment. Simulations and experimental studies on a PUMA 560 robot reveal the convenience and effectiveness of the proposed robot self-calibration approach.     

自主导航移动机器人视觉系统的一种标定方法

设计了一种基于视场中单个目标点的视觉系统标定方法,任意选取视场中的一点作为目标点,以该目标点为基准,机器人作相对运动来获得多个特征点。建立图像系列对应点之间的几何约束关系及各坐标系之间的变换矩阵,确定变换矩阵关系式,进一步求解摄像机的内外参数。该标定方法只需提取场景中的一个景物点,对机器人的运动控制操作方便、算法实现简洁。实验结果验证了该方法的有效性。     

基于单目视频路径移动机器人导航方法研究

以单目视觉中双视图定位为基础,提出了一种基于单目视频路径的导航方法.算法基于单目摄像机提取一组有序的关键图像,以代表视频路径,使用Harris角点算法提取图像特征点,利用零均值归一化互相关算法进行图像匹配,根据单目摄像机双视图匹配点之间的约束关系求出基础矩阵并优化,得到位姿参数,完成了移动机器人定位,使其在视频路径的指导下实现自主导航.验证表明:方法在室外环境中具有较好的实用性.     

Omnidirectional vision-based ego-pose estimation for an autonomous in-pipe mobile robot

This paper describes the omnidirectional vision-based ego-pose estimation method of an in-pipe mobile robot. An in-pipe mobile robot has been developed for inspecting the inner surface of various pipeline configurations, such as the straight pipeline, the elbow and the multiple-branch. Because the proposed in-pipe mobile robot has four individual drive wheels, it has the ability of flexible motions in various pipelines. The ego-pose estimation is indispensable for the autonomous navigation of the proposed in-pipe robot. An omnidirectional camera and four laser modules mounted on the mobile robot are used for ego-pose estimation. An omnidirectional camera is also used for investigating the inner surface of the pipeline. The pose of the in-pipe mobile robot is estimated from the relationship equation between the pose of a robot and the pixel coordinates of four intersection points where light rays that emerge from four laser modules intersect the inside of the pipeline. This relationship equation is derived from the geometry analysis of an omnidirectional camera and four laser modules. In experiments, the performance of the proposed method is evaluated by comparing the result of our algorithm with the measurement value of a specifically designed sensor, which is a kind of a gyroscope.     

视觉伺服机械臂手机抓取最佳位姿检测

针对下水道、勾缝等狭窄位置的手机拾取的问题, 本文提出一种基于机器视觉的伺服机械臂抓取方法. 首先对机械臂eye-in-hand上的相机进行标定, 图像预处理及目标检测等, 在位姿检测中提出一种基于二维坐标系下手机位姿解算算法, 得出解算的最佳位姿角只与夹持点的像素坐标的差值有关, 其位姿角的大小决定了手抓旋转的角度. 实验用Matlab软件对位姿检测进行仿真分析, 其包括SURF不变特征点的目标检测实验和位姿解算实验, 最后用Rethink双臂机器人的右臂对手机进行抓取验证. 实验表明, 在误差允许范围内, 提出的算法具有一定的有效性, 其结果为伺服机械臂抓取手机提供良好的准确性.     

Kinect相机的机器人视觉里程计

Kinect相机不仅能够获取彩色图像,还可以得到相应的深度信息.本文探讨了一种通过Kinect相机来实现里程计的方法,该方法由Kinect相机获取周围环境的连续帧信息,提取并匹配连续帧间的SIFT特征点,获取图像帧的描述子,通过特征点的位置变化计算机器人的位姿、里程编码等,从而实现视觉里程计的功能.     

基于亚像素的乒乓球机器人本体位姿视觉测量

提出一种基于亚像素精度的特征点提取算法,结合PnP方法和正交迭代(OI)算法计算乒乓球机器人本体的位姿.根据摄像机成像时弥散斑的近似高斯分布,以亚像素精度精确求取色标块的边缘,利用边缘直线交点得到高精度的角点作为特征点.基于PnP算法利用上述特征点求取机器人位姿的初值,再通过OI算法进行优化,以保证其姿态矩阵的正交性.实验结果表明,该方法能快速准确地实现机器人本体的位姿测量.     

Alignment of a flexible sheet object with position-based and image-based visual servoing

This paper describes visual servoing methods to align an end edge of a flexible sheet object such as paper to a specified line segment in 3D space. In order to manipulate the end edge pose which may be distant from a grasping point by a robot hand on the flexible sheet, we incorporate online estimation of the relative pose between the end edge and the robot hand into visual servoing control laws. Two types of feature values to be regulated, position-based and image-based ones, are defined. The position-based one consists of 3D pose parameters of the end edge, which are estimated by fitting a polynomial surface model to stereo-measured edge points around the sheet. The image-based one consists of image pixel values within narrow regions around the end edge. Through numerical simulations, we confirm that the position-based features are effective in achieving quick response while the image-based features are robust against measurement noise. In order to combine these contrasting behaviors, we introduce a weighted combination of the position-based and image-based features. Experimental results show both of the position-based method and the weighted combination method work well and suggest the possibility that the weighted combination method can reduce final alignment errors.     

基于RGB-D图像的室内机器人同时定位与地图构建

同时定位与地图构建（SLAM）是机器人在未知环境实现自主导航的关键技术，针对目前常用的RGB-D SLAM系统实时性差和精确度低的问题，提出一种新的RGB-D SLAM系统，以进一步提升实时性和精确度。首先，采用ORB算法检测图像特征点，并对提取的特征点采用基于四叉树的均匀化策略进行处理，并结合词袋模型（BoW）进行特征匹配。然后，在系统相机姿态初始值估计阶段，结合PnP和非线性优化方法为后端优化提供一个更接近最优值的初始值；在后端优化中，使用光束法平差（BA）对相机姿态初始值进行迭代优化，从而得到相机姿态的最优值。最后，根据相机姿态和每帧点云地图的对应关系，将所有的点云数据注册到同一个坐标系中，得到场景的稠密点云地图，并对点云地图利用八叉树进行递归式的压缩以得到一种用于机器人导航的三维地图。在TUM RGB-D数据集上，将构建的RGB-D SLAM同RGB-D SLAMv2、ORB-SLAM2系统进行了对比，实验结果表明所构建的RGB-D SLAM系统在实时性和精确度上的综合表现更优。     

基于双目相机的视觉里程计

针对移动机器人视觉导航定位需求,提出一种基于双目相机的视觉里程计改进方案。对于特征信息冗余问题,改进ORB(oriented FAST and rotated BRIEF)算法,引入多阈值FAST图像分割思想,为使误匹配尽可能减少,主要运用快速最近邻和随机采样一致性算法;一般而言,运用的算法主要是立体匹配算法,此算法的特征主要指灰度,对此算法做出改进,运用一种新型的双目视差算法,此算法主要以描述子为特征,据此恢复特征点深度;为使所得位姿坐标具有相对较高的准确度,构造一种特定的最小二乘问题,使其提供初值,以相应的特征点三维坐标为基础,基于有效方式对相机运动进行估计。根据数据集的实验结果可知,所提双目视觉里程具有相对而言较好的精度及较高的实时性。     

基于RGB-D图像的室内机器人同时定位与地图构建

同时定位与地图构建（SLAM）是机器人在未知环境实现自主导航的关键技术,针对目前常用的RGB-D SLAM系统实时性差和精确度低的问题,提出一种新的RGB-D SLAM系统,以进一步提升实时性和精确度。首先,采用ORB算法检测图像特征点,并对提取的特征点采用基于四叉树的均匀化策略进行处理,并结合词袋模型（BoW）进行特征匹配。然后,在系统相机姿态初始值估计阶段,结合PnP和非线性优化方法为后端优化提供一个更接近最优值的初始值;在后端优化中,使用光束法平差（BA）对相机姿态初始值进行迭代优化,从而得到相机姿态的最优值。最后,根据相机姿态和每帧点云地图的对应关系,将所有的点云数据注册到同一个坐标系中,得到场景的稠密点云地图,并对点云地图利用八叉树进行递归式的压缩以得到一种用于机器人导航的三维地图。在TUM RGB-D数据集上,将构建的RGB-D SLAM同RGB-D SLAMv2、ORB-SLAM2系统进行了对比,实验结果表明所构建的RGB-D SLAM系统在实时性和精确度上的综合表现更优。     

Uncalibrated visual servoing of robots using a depth-independent interaction matrix

This paper presents a new adaptive controller for image-based dynamic control of a robot manipulator using a fixed camera whose intrinsic and extrinsic parameters are not known. To map the visual signals onto the joints of the robot manipulator, this paper proposes a depth-independent interaction matrix, which differs from the traditional interaction matrix in that it does not depend on the depths of the feature points. Using the depth-independent interaction matrix makes the unknown camera parameters appear linearly in the closed-loop dynamics so that a new algorithm is developed to estimate their values on-line. This adaptive algorithm combines the Slotine-Li method with on-line minimization of the errors between the real and estimated projections of the feature points on the image plane. Based on the nonlinear robot dynamics, we prove asymptotic convergence of the image errors to zero by the Lyapunov theory. Experiments have been conducted to verify the performance of the proposed controller. The results demonstrated good convergence of the image errors.