基于点线结合特征的单目视觉里程计

SLAM(即时定位与地图构建)系统是近年来计算机视觉领域的一大重要课题,其中特征法的SLAM凭借稳定性好、计算效率高的优点成为SLAM算法的主流。目前特征法SLAM主要基于点特征进行。针对基于点特征的视觉里程计依赖于数据质量,相机运动过快时容易跟丢,且生成的特征地图不包含场景结构信息等缺点,提出了一种基于点线结合特征的优化算法。相较于传统基于线段端点的六参数表达方式,算法采用一种四参数的方式表示空间直线,并使用点线特征进行联合图优化估计相机位姿。使用公开数据集和自采集鱼眼影像数据分别进行实验的结果表明,与仅使用点特征的方法相比,该方法可有效改善因相机运动过快产生的跟丢问题,增加轨迹长度,提升位姿估计精度,且生成的稀疏特征地图更能反映场景结构特征。     

SLAM in Indoor Environments using Omni-directional Vertical and Horizontal Line Features

An autonomous mobile robot must have the ability to navigate in an unknown environment. The simultaneous localization and map building (SLAM) problem have relation to this autonomous ability. Vision sensors are attractive equipment for an autonomous mobile robot because they are information-rich and rarely have restrictions on various applications. However, many vision based SLAM methods using a general pin-hole camera suffer from variation in illumination and occlusion, because they mostly extract corner points for the feature map. Moreover, due to the narrow field of view of the pin-hole camera, they are not adequate for a high speed camera motion. To solve these problems, this paper presents a new SLAM method which uses vertical lines extracted from an omni-directional camera image and horizontal lines from the range sensor data. Due to the large field of view of the omni-directional camera, features remain in the image for enough time to estimate the pose of the robot and the features more accurately. Furthermore, since the proposed SLAM does not use corner points but the lines as the features, it reduces the effect of illumination and partial occlusion. Moreover, we use not only the lines at corners of wall but also many other vertical lines at doors, columns and the information panels on the wall which cannot be extracted by a range sensor. Finally, since we use the horizontal lines to estimate the positions of the vertical line features, we do not require any camera calibration. Experimental work based on MORIS, our mobile robot test bed, moving at a human’s pace in the real indoor environment verifies the efficacy of this approach.     

一种基于点线不变量的视觉SLAM算法

点线特征结合的视觉SLAM（同步定位与地图构建）算法中,线特征匹配准确度差会引入新的误差,点线特征误差的累积加剧了数据关联失败情况的发生．针对这一问题,本文设计了一种基于点线不变量的线特征匹配方法,该点线不变量对线段与相邻2个特征点的局部几何关系进行编码,直接在现有特征点的基础上完成线匹配,可有效提高线段匹配的速度和准确度;此外,在点线特征的融合过程引入加权思想,根据场景特征丰富程度,在构造误差函数时对点线特征的权重进行合理分配．在TUM室内数据集和KITTI道路数据集上的实验表明,与现有的点线SLAM系统相比,本文提出的点线SLAM系统有效地提高了视觉SLAM中线特征匹配的准确度,提高了特征匹配环节的运行效率,使线特征在SLAM过程中发挥了积极有效的作用,提高了系统数据关联的稳定性．     

线特征融合光流的单目SLAM算法

针对移动机器人的定位与建图问题,提出了基于图优化的单目线特征融合光流的同时定位和地图构建(SLAM)的算法。首先,针对主流视觉SLAM算法因采用点作为特征而导致构建的点云地图稀疏、难以准确表达环境结构信息等缺点,采用直线作为特征来构建地图,并采用图优化方法来提高定位精度和地图构建的准确性。然后,针对定位系统的处理速度很难达到实时性要求,将光流法引入以达到实时定位的效果。实验表明,基于线特征的地图构建有较高的建图精度,并且融合算法克服了光流法定位精度差和特征法处理速度慢的缺点,可提供较准确的实时定位输出,并对光照变化和场景纹理较少的情况有一定的鲁棒性。     

Discovering Higher Level Structure in Visual SLAM

In this paper, we describe a novel method for discovering and incorporating higher level map structure in a real-time visual simultaneous localization and mapping (SLAM) system. Previous approaches use sparse maps populated by isolated features such as 3-D points or edgelets. Although this facilitates efficient localization, it yields very limited scene representation and ignores the inherent redundancy among features resulting from physical structure in the scene. In this paper, higher level structure, in the form of lines and surfaces, is discovered concurrently with SLAM operation, and then, incorporated into the map in a rigorous manner, attempting to maintain important cross-covariance information and allow consistent update of the feature parameters. This is achieved by using a bottom-up process, in which subsets of low-level features are “folded in” to a parameterization of an associated higher level feature, thus collapsing the state space as well as building structure into the map. We demonstrate and analyze the effects of the approach for the cases of line and plane discovery, both in simulation and within a real-time system operating with a handheld camera in an office environment.      

多源融合SLAM的现状与挑战

同时定位与地图构建(simultaneous localization and mapping,SLAM)技术在过去几十年中取得了惊人的进步,并在现实生活中实现了大规模的应用。由于精度和鲁棒性的不足,以及场景的复杂性,使用单一传感器(如相机、激光雷达)的SLAM系统往往无法适应目标需求,故研究者们逐步探索并改进多源融合的SLAM解决方案。本文从3个层面回顾总结该领域的现有方法:1)多传感器融合(由两种及以上传感器组成的混合系统,如相机、激光雷达和惯性测量单元,可分为松耦合、紧耦合);2)多特征基元融合(点、线、面、其他高维几何特征等与直接法相结合);3)多维度信息融合(几何、语义、物理信息和深度神经网络的推理信息等相融合)。惯性测量单元和视觉、激光雷达的融合可以解决视觉里程计的漂移和尺度丢失问题,提高系统在非结构化或退化场景中的鲁棒性。此外,不同几何特征基元的融合,可以大大减少有效约束的程度,并可为自主导航任务提供更多的有用信息。另外,数据驱动下的基于深度学习的策略为SLAM系统开辟了新的道路。监督学习、无监督学习和混合监督学习等逐渐应用于SLAM系统的各个模块,如相对姿势估计、地图表...     

Enhanced Geometric Map: a 2D&3D Hybrid City Model of Large Scale Urban Environment for Robot Navigation

To facilitate scene understanding and robot navigation in large scale urban environment, a two-layer enhanced geometric map(EGMap) is designed using videos from a monocular onboard camera. The 2D layer of EGMap consists of a 2D building boundary map from top-down view and a 2D road map, which can support localization and advanced map-matching when compared with standard polyline-based maps. The 3D layer includes features such as 3D road model,and building facades with coplanar 3D vertical and horizontal line segments, which can provide the 3D metric features to localize the vehicles and flying-robots in 3D space. Starting from the 2D building boundary and road map, EGMap is initially constructed using feature fusion with geometric constraints under a line feature-based simultaneous localization and mapping(SLAM) framework iteratively and progressively. Then, a local bundle adjustment algorithm is proposed to jointly refine the camera localizations and EGMap features. Furthermore, the issues of uncertainty, memory use, time efficiency and obstacle effect in EGMap construction are discussed and analyzed. Physical experiments show that EGMap can be successfully constructed in large scale urban environment and the construction method is demonstrated to be very accurate and robust.     

语义线特征辅助的动态SLAM

动态环境干扰是视觉同时定位与地图构建（simultaneous localization and mapping,SLAM）领域内一个亟待解决的问题,场景中的运动对象会严重影响系统定位精度。结合语义信息和几何约束更强的线特征辅助基于传统ORB特征的SLAM系统来解决动态SLAM问题。首先采用深度学习领域的优秀成果SOLOv2作为场景分割网络,并赋予线特征语义信息;完成物体跟踪和静态区域初始化后,使用mask金字塔提取并分类特征点;再使用极线约束完成动态物体上点线特征的剔除;最后融合静态点线特征完成位姿的精确估计。在TUM动态数据集上的实验表明,提出的系统比ORB-SLAM3的位姿估计精度提高了72.20％,比DynaSLAM提高了20.42％,即使与近年来同领域内的优秀成果相比也有较好的精度表现。     

基于点线特征融合的视觉惯性SLAM算法

针对目前视觉SLAM方法鲁棒性差、耗时高,使系统定位不够精确的问题,提出了一种基于点线特征融合的视觉惯性SLAM算法。首先通过短线剔除和近似线段合并策略改进LSD（line segment detection）提取质量,以提高线特征检测的速率和准确度;然后在后端优化中有效融合了点、线和IMU数据,建立最小化目标函数进行优化,得到更精确的相机位姿;最后在EuRoC数据集和现实走廊场景进行了实验验证。实验表明,所提算法可以有效提升线特征提取的质量和速度,同时有效提高了SLAM系统的定位精度,获得更为丰富的点线结构地图。     

Line Segment-Based Indoor Mapping with Salient Line Feature Extraction

We present a method of simultaneous localization and mapping (SLAM) in a large indoor environment using a Rao-Blackwellized particle filter (RBPF) along with a line segment as a landmark. To represent the environment in a compact form, we use only two end points of a line segment, thus reducing computational cost in modeling line segment uncertainty. With a modified scan point clustering method, the proposed adaptive iterative end point fitting contributes to the estimation of line parameters by considering noisy scan points near end points. Thus, by line segment matching the robot is localized well in a local frame. We also introduce an online and offline method of global line merging, which provides a more compact map by removing spurious lines and merging collinear lines. Each of our approaches is efficiently integrated into the proposed RBPF-SLAM framework. In experiments with well-known data sets, the proposed method provides reliable SLAM and compact map representation even in a cluttered environment.     

基于RGB-D图像的室内机器人同时定位与地图构建

同时定位与地图构建（SLAM）是机器人在未知环境实现自主导航的关键技术,针对目前常用的RGB-D SLAM系统实时性差和精确度低的问题,提出一种新的RGB-D SLAM系统,以进一步提升实时性和精确度。首先,采用ORB算法检测图像特征点,并对提取的特征点采用基于四叉树的均匀化策略进行处理,并结合词袋模型（BoW）进行特征匹配。然后,在系统相机姿态初始值估计阶段,结合PnP和非线性优化方法为后端优化提供一个更接近最优值的初始值;在后端优化中,使用光束法平差（BA）对相机姿态初始值进行迭代优化,从而得到相机姿态的最优值。最后,根据相机姿态和每帧点云地图的对应关系,将所有的点云数据注册到同一个坐标系中,得到场景的稠密点云地图,并对点云地图利用八叉树进行递归式的压缩以得到一种用于机器人导航的三维地图。在TUM RGB-D数据集上,将构建的RGB-D SLAM同RGB-D SLAMv2、ORB-SLAM2系统进行了对比,实验结果表明所构建的RGB-D SLAM系统在实时性和精确度上的综合表现更优。     

基于RGB-D图像的室内机器人同时定位与地图构建

同时定位与地图构建（SLAM）是机器人在未知环境实现自主导航的关键技术，针对目前常用的RGB-D SLAM系统实时性差和精确度低的问题，提出一种新的RGB-D SLAM系统，以进一步提升实时性和精确度。首先，采用ORB算法检测图像特征点，并对提取的特征点采用基于四叉树的均匀化策略进行处理，并结合词袋模型（BoW）进行特征匹配。然后，在系统相机姿态初始值估计阶段，结合PnP和非线性优化方法为后端优化提供一个更接近最优值的初始值；在后端优化中，使用光束法平差（BA）对相机姿态初始值进行迭代优化，从而得到相机姿态的最优值。最后，根据相机姿态和每帧点云地图的对应关系，将所有的点云数据注册到同一个坐标系中，得到场景的稠密点云地图，并对点云地图利用八叉树进行递归式的压缩以得到一种用于机器人导航的三维地图。在TUM RGB-D数据集上，将构建的RGB-D SLAM同RGB-D SLAMv2、ORB-SLAM2系统进行了对比，实验结果表明所构建的RGB-D SLAM系统在实时性和精确度上的综合表现更优。     

基于视觉语义与激光点云交融构建的SLAM算法

激光雷达作为同时定位与地图构建(SLAM)传感器之一,因精度高、性能稳定等特点而被广泛研究使用.但其获得的点云数据较稀疏,包含特征信息少,会导致误匹配、位姿估计误差大等问题,影响SLAM的定位和建图精度.对此,提出一种将视觉语义信息与激光点云数据融合的SLAM算法(VSIL-SLAM).首先,基于投影思想将聚类后的点云映射到语义检测框内,生成语义物体,解决原始激光点云特征稀缺问题;然后,在形状特征的基础上引入拓扑特征对语义物体进行表述,提出基于匹配的拓扑相似性度量方法,解决单一特征造成的误匹配问题,提高匹配准确度;最后,加入语义物体点到点的几何约束,基于几何特征和语义物体构建前端里程计,并完成后端回环检测和位姿图优化设计.实验结果表明,所提出算法在定位和建图效果上都有显著提高,改善了激光SLAM算法的性能.     

Large-Scale 6-DOF SLAM With Stereo-in-Hand

In this paper, we describe a system that can carry out simultaneous localization and mapping (SLAM) in large indoor and outdoor environments using a stereo pair moving with 6 DOF as the only sensor. Unlike current visual SLAM systems that use either bearing-only monocular information or 3-D stereo information, our system accommodates both monocular and stereo. Textured point features are extracted from the images and stored as 3-D points if seen in both images with sufficient disparity, or stored as inverse depth points otherwise. This allows the system to map both near and far features: the first provide distance and orientation, and the second provide orientation information. Unlike other vision-only SLAM systems, stereo does not suffer from “scale drift” because of unobservability problems, and thus, no other information such as gyroscopes or accelerometers is required in our system. Our SLAM algorithm generates sequences of conditionally independent local maps that can share information related to the camera motion and common features being tracked. The system computes the full map using the novel conditionally independent divide and conquer algorithm, which allows constant time operation most of the time, with linear time updates to compute the full map. To demonstrate the robustness and scalability of our system, we show experimental results in indoor andoutdoor urban environments of 210 m and 140 m loop trajectories, with the stereo camera being carried in hand by a person walking at normal walking speeds of 4--5 km/h.      

Camera calibration by vanishing lines for 3-D computer vision

A novel approach to camera calibration by vanishing lines is proposed. Calibrated parameters include the orientation, position, and focal length of a camera. A hexagon is used as the calibration target to generate a vanishing line of the ground plane from its projected image. It is shown that the vanishing line includes useful geometric hints about the camera orientation parameters and the focal length, from which the orientation parameters can be solved easily and analytically. And the camera position parameters can be calibrated by the use of related geometric projective relationships. The simplicity of the target eliminates the complexity of the environment setup and simplifies the feature extraction in relevant image processing. The calibration formulas are also simple to compute. Experimental results show the feasibility of the proposed approach     

基于点线特征融合的低纹理单目视觉同时定位与地图构建算法

当图像因相机快速运动造成模糊或者处在低纹理场景时,仅使用点特征的同步定位与地图构建（SLAM）算法难以跟踪提取足够多的特征点,导致定位精度和匹配鲁棒性较差。而如果造成误匹配,甚至系统都无法工作。针对上述问题,提出了一种基于点线特征融合的低纹理单目SLAM算法。首先,加入了线特征来加强系统稳定性,并解决了点特征算法在低纹理场景中提取不足的问题;然后,对点、线特征提取数量的选择引入了加权的思想,根据场景的丰富程度,对点线特征的权重进行了合理分配。所提算法是在低纹理场景下运行的,因而设置以线特征为主、点特征为辅。在TUM室内数据集上的实验结果表明,与现有的点线特征算法相比,所提算法有效地提高了线特征的匹配精度,使得轨迹误差减小了大约9个百分点,也使得特征提取时间减少了30个百分点,使加入的线特征在低纹理场景中发挥出积极有效的作用,提高了数据整体的准确度和可信度。     

单视未标定图像的正交灭点检测算法

传统方法只能计算标定图像的正交灭点,同时没有考虑图像直线检测结果的误差、 直线的长度以及候选灭点与约束直线之间的位置关系对灭点检测精度的影响。针对此类问题, 提出了一种针对单视未标定图像的正交灭点检测方法。首先利用J-Linkage 完成灭点的初始化估 计,得到假设灭点集合;然后根据假设灭点与图像直线之间的一致性约束、图像直线的长度, 基于投票机制先得到精确的垂直方向灭点;后利用灭点、灭线的定义和性质,计算得到图像相 机参数;根据正交灭点的特性,得到准确的水平方向和纵深方向的灭点。因引入了一种新的假 设灭点和图像直线之间的一致性度量方法,正交灭点检测精度不受直线检测结果的误差、直线 的长度以及候选灭点与约束直线之间的位置关系的影响,在未知图像相机参数的情况下能精准 的得到三个正交灭点信息。正交灭点检测方法在室内场景下可以得到更加精确的检测结果。     

基于半直接法SLAM的大场景稠密三维重建系统

当前三维重建系统大多基于特征点法和直接法的同时定位与地图重建(SLAM)系统,特征点法SLAM难以在特征点缺失的地方具有较好的重建结果,直接法SLAM在相机运动过快时难以进行位姿估计,从而造成重建效果不理想.针对上述问题,文中提出基于半直接法SLAM的大场景稠密三维重建系统.通过深度相机(RGB-D相机)扫描,在特征点丰富的区域使用特征点法进行相机位姿估计,在特征点缺失区域使用直接法进行位姿估计,减小光度误差,优化相机位姿.然后使用优化后较准确的相机位姿进行地图构建,采用面元模型,应用构建变形图的方法进行点云的位姿估计和融合,最终获得较理想的三维重建模型.实验表明,文中系统可适用于各个场合的三维重建,得到较理想的三维重建模型.     

Novel Sonar Salient Feature Structure for Extended Kalman Filter-Based Simultaneous Localization and Mapping of Mobile Robots

Abstract

Not all line or point features capable of being extracted by sonar sensors from a cluttered home environment are useful for simultaneous localization and mapping (SLAM) of a mobile robot. This is due to unfavorable conditions such as environmental ambiguity and sonar measurement uncertainty. We present a novel sonar feature structure suitable for a cluttered environment and the extended Kalman filter (EKF)-based SLAM scheme. The key concept is to extract circle feature clouds on salient convex objects by sonar data association called convex saliency circling. The centroid of each circle cloud, called a sonar salient feature, is used as a natural landmark for EKF-based SLAM. By investigating the environmental inherent feature locality, cylindrical objects are augmented conveniently at the weak SLAM-able area as a natural supplementary saliency to achieve consistent SLAM performance. Experimental results demonstrate the validity and robustness of the proposed sonar salient feature structure for EKF-based SLAM.     

Hypercatadioptric line images for 3D orientation and image rectification

In central catadioptric systems 3D lines are projected into conics. In this paper we present a new approach to extract conics in the raw catadioptric image, which correspond to projected straight lines in the scene. Using the internal calibration and two image points we are able to compute analytically these conics which we name hypercatadioptric line images. We obtain the error propagation from the image points to the 3D line projection in function of the calibration parameters. We also perform an exhaustive analysis on the elements that can affect the conic extraction accuracy. Besides that, we exploit the presence of parallel lines in man-made environments to compute the dominant vanishing points (VPs) in the omnidirectional image. In order to obtain the intersection of two of these conics we analyze the self-polar triangle common to this pair. With the information contained in the vanishing points we are able to obtain the 3D orientation of the catadioptric system. This method can be used either in a vertical stabilization system required by autonomous navigation or to rectify images required in applications where the vertical orientation of the catadioptric system is assumed. We use synthetic and real images to test the proposed method. We evaluate the 3D orientation accuracy with a ground truth given by a goniometer and with an inertial measurement unit (IMU). We also test our approach performing vertical and full rectifications in sequences of real images.