三维重建系统下的特征点处理与位姿恢复优化算法

基于SIFT算法原理,构建了一个全新的算法框架,使用FCN（fully convolutional networks,全卷积神经网络）和BP（back propagation,反向传播）神经网络,综合考虑图像主目标的语义分割、图像灰度共生矩阵等方面的影响,实现了自适应的特征点检测范围、数量调整,并在特征点匹配阶段利用相机位姿偏移稳定性剔除误匹配,同时采用基于图优化的方法对位姿恢复结果进行非线性优化,得到了更加精确的相机位姿。最后与现有主流算法进行分析比对,实验结果验证了该算法的有效性,提高了特征点检测的场景自适应程度与特征点匹配、位姿恢复的精度,实现了更加高效的三维重建。     

基于特征匹配的单目相机姿态解算

本文采用了一种基于AKAZE特征检测和PnP算法的单目视觉测量方法对相机的相对姿态进行解算,用于快速准确地确定空间中两个目标间的位姿关系.采集合作目标的模板图像,提取附加到合作目标上的4个特征点的像素坐标,利用AKAZE关键点对模板图像和待测图像进行匹配并计算映射矩阵,通过映射矩阵得到4个特征点在待测图像中的像素坐标,然后结合合作目标的尺寸信息求解基于4个共面特征点的PnP问题,解算相机与合作目标的相对位置.实验分析表明该方法计算的实时图像相机位姿与真实结果接近,验证了本文方法的有效性.     

基于动态遍历的分层特征网络视觉定位

采用分层特征网络估计查询图像的相机位姿,会出现检索失败和检索速度慢的问题。对分层特征网络进行分析,提出采用动态遍历与预聚类的视觉定位方法。依据场景地图进行图像预聚类,利用图像全局描述符获得候选帧集合并动态遍历查询图像,利用图像局部特征描述符进行特征点匹配,通过PnP算法估计查询图像的相机位姿,由此构建基于MobileNetV3的分层特征网络,以准确提取全局描述符与局部特征点。在典型数据集上与AS、CSL、DenseVLAD、NetVLAD等主流视觉定位方法的对比结果表明,该方法能够改善光照与季节变化场景下对候选帧的检索效率,提升位姿估计精度和候选帧检索速度。     

基于深度学习SuperGlue算法的单目视觉里程计

基于特征点法的视觉里程计中,光照和视角变化会导致特征点提取不稳定,进而影响相机位姿估计精度,针对该问题,提出了一种基于深度学习SuperGlue匹配算法的单目视觉里程计建模方法.首先,通过SuperPoint检测器获取特征点,并对得到的特征点进行编码,得到包含特征点坐标和描述子的向量;然后,通过注意力GNN网络生成更具代表性的描述子,并创建M×N型得分分配矩阵,采用Sinkhorn算法求解最优得分分配矩阵,从而得到最优特征匹配;最后,根据最优特征匹配进行相机位姿恢复,采用最小化投影误差法进行相机位姿优化.实验结果表明,在无后端优化的条件下,该算法与基于ORB或SIFT算法的视觉里程计相比,不仅对视角和光线变化更鲁棒,而且其绝对轨迹误差和相对位姿误差的精度均有显著提升,进一步验证了基于深度学习的SuperGlue匹配算法在视觉SLAM中的可行性和优越性.     

一种融合稀疏几何特征与深度流的深度视觉SLAM算法

为了克服移动机器人在视觉退化场景下的位姿估计问题,通过将稠密的深度流与稀疏几何特征相结合,提出了一种实时、鲁棒和低漂移的深度视觉SLAM(同时定位与地图构建)算法.该算法主要由3个优化层组成,基于深度流的视觉里程计层、基于ICP(迭代最近点)的位姿优化层和基于位姿图的优化层.基于深度流的视觉里程计层通过建立深度变化约束方程实现相机帧间快速的6自由度位姿估计;基于ICP的位姿优化层通过构建局部地图来消除局部漂移;基于位姿图的优化层从深度信息中提取、匹配稀疏几何特征,从而建立闭环约束并通过位姿图来实现全局位姿优化.对本文所提出的算法分别在TUM数据集和实际场景中进行了性能测试.实验结果表明本文的前端算法的性能优于当前深度视觉主流算法,后端算法可以较为鲁棒地建立闭环约束并消除前端位姿估计所产生的全局漂移.     

基于目标检测的室内动态场景定位与建图

针对室内场景中动态对象严重影响相机位姿估计准确性的问题,提出一种基于目标检测的室内动态场景同步定位与地图构建（SLAM）系统。当相机捕获图像后,首先,利用YOLOv4目标检测网络检测环境中的动态对象,并生成对应边界框的掩膜区域;然后,提取图像中的ORB特征点,并将掩膜区域内部的特征点剔除掉;同时结合GMS算法进一步剔除误匹配,并仅利用剩余静态特征点来估计相机位姿;最后,完成滤除动态对象的静态稠密点云地图和八叉树地图的构建。在TUM RGB-D公开数据集上进行的多次对比测试的结果表明,相对于ORB-SLAM2系统、GCNv2_SLAM系统和YOLOv4+ORB-SLAM2系统,所提系统在绝对轨迹误差（ATE）和相对位姿误差（RPE）上有明显的降低,说明该系统能够显著提高室内动态环境中相机位姿估计的准确性。     

面向机器人抓取过程中目标位姿估计方法

针对机械臂抓取过程中场景的复杂性和存在遮挡问题,提出了一种基于深度相机的物体位姿估计方法。采用Kinect相机获取点云图像,提取点云的FPFH特征。采用奇异值分解算法和随机一致算法来进行位姿估计。将得到的位姿经过手眼转换转换为抓取位姿。通过与ICP算法和NDT算法进行对比实验,结果验证了该方法的稳定性和精确性。     

基于ABM与FBM结合的精确快速相机自标定方法

针对BA在场景重构时精度达不到摄影测量规范要求这一问题,提出一种SFM流程,旨在最大化增量重建的精度。采用分层匹配方法,在图像金字塔顶层进行基于特征点的匹配(FBM),采用带有改进的NCC和LSM的基于区域匹配(ABM)方法逐层对匹配点位置进行调整,得到高精确度的匹配位置;将修正后的匹配点用于基础矩阵F和单应矩阵H的估计,计算tracks信息,生成图像连通图G,用于决定图像添加到BA中的顺序,从而得到高精度的相机位姿和3D点。实验结果表明,该流程能够提高重建结果的精度并实现可扩展性。     

RTM框架下基于点线特征的视觉SLAM算法

针对图像纹理较为单一及相对模糊时仅仅依靠点特征难以实现精确位姿估计的问题,采用分散模块化技术提出了一种基于点线特征的视觉SLAM(同时定位与地图创建)算法.首先,提取相机采集环境中的点特征及线特征,并根据帧间特征匹配进行跟踪;随后,采用改进的NICP(normal iterative closest point)算法与关键帧匹配策略构建里程计系统.在此基础上,引入基于点线特征词典的闭环检测与GTSAM(Georgia Tech smoothing and mapping library)图优化方法获取具有全局一致性位姿的3维点云地图.以机器人技术中间件构筑系统框架,在提高系统实时性的同时增强功能模块的可扩展性与灵活性.标准数据集与实际实验室场景下的实验结果验证了所提方法的可行性和有效性.     

基于输电杆塔区域提取的图像匹配算法

针对无人机（UAV）视觉定位过程中传统特征提取与匹配算法匹配质量不佳的问题,提出了一种基于输电杆塔区域提取的图像匹配算法。首先,将图像划分为若干相互重叠的网格区域,并对每个区域采用双层金字塔结构提取特征点,从而保证特征点的均匀分布;其次,使用直线分割检测（LSD）算法提取图像中的直线,从而利用输电杆塔的特殊结构得到输电杆塔的支撑区域;最后,在连续图像中对输电杆塔区域与背景区域内的特征点分别进行匹配,以进一步估计相机运动。在旋转和平移估计实验中,与传统的ORB特征提取与匹配算法相比,所提算法的特征匹配准确率提升了10.1个百分点,相对位姿误差的均值降低了0.049;在UAV巡检实验中,采用所提算法进行UAV轨迹估计的相对误差为2.89%,表明该算法可在实时绕塔飞行过程中实现对UAV位姿的鲁棒、精确估计。     

Multi-view structure-from-motion for hybrid camera scenarios

We describe a pipeline for structure-from-motion (SfM) with mixed camera types, namely omnidirectional and perspective cameras. For the steps of this pipeline, we propose new approaches or adapt the existing perspective camera methods to make the pipeline effective and automatic. We model our cameras of different types with the sphere camera model. To match feature points, we describe a preprocessing algorithm which significantly increases scale invariant feature transform (SIFT) matching performance for hybrid image pairs. With this approach, automatic point matching between omnidirectional and perspective images is achieved. We robustly estimate the hybrid fundamental matrix with the obtained point correspondences. We introduce the normalization matrices for lifted coordinates so that normalization and denormalization can be performed linearly for omnidirectional images. We evaluate the alternatives of estimating camera poses in hybrid pairs. A weighting strategy is proposed for iterative linear triangulation which improves the structure estimation accuracy. Following the addition of multiple perspective and omnidirectional images to the structure, we perform sparse bundle adjustment on the estimated structure by adapting it to use the sphere camera model. Demonstrations of the end-to-end multi-view SfM pipeline with the real images of mixed camera types are presented.     

基于单目视频路径移动机器人导航方法研究

以单目视觉中双视图定位为基础,提出了一种基于单目视频路径的导航方法.算法基于单目摄像机提取一组有序的关键图像,以代表视频路径,使用Harris角点算法提取图像特征点,利用零均值归一化互相关算法进行图像匹配,根据单目摄像机双视图匹配点之间的约束关系求出基础矩阵并优化,得到位姿参数,完成了移动机器人定位,使其在视频路径的指导下实现自主导航.验证表明:方法在室外环境中具有较好的实用性.     

Object Pose: The Link between Weak Perspective,Paraperspective, and Full Perspective

Recently, DeMenthon and Davis (1992, 1995) proposed a method for determining the pose of a 3-D object with respect to a camera from 3-D to 2-D point correspondences. The method consists of iteratively improving the pose computed with a weak perspective camera model to converge, at the limit, to a pose estimation computed with a perspective camera model. In this paper we give an algebraic derivation of DeMenthon and Davis' method and we show that it belongs to a larger class of methods where the perspective camera model is approximated either at zero order (weak perspective) or first order (paraperspective). We describe in detail an iterative paraperspective pose computation method for both non coplanar and coplanar object points. We analyse the convergence of these methods and we conclude that the iterative paraperspective method (proposed in this paper) has better convergence properties than the iterative weak perspective method. We introduce a simple way of taking into account the orthogonality constraint associated with the rotation matrix. We analyse the sensitivity to camera calibration errors and we define the optimal experimental setup with respect to imprecise camera calibration. We compare the results obtained with this method and with a non-linear optimization method.     

定位图像匹配尺度与区域的摄像机位姿实时跟踪

目的 提出一种定位图像匹配尺度及区域的有效算法,通过实现当前屏幕图像特征点与模板图像中对应尺度下部分区域中的特征点匹配,实现摄像机对模板图像的实时跟踪,解决3维跟踪算法中匹配精度与效率问题。方法 在预处理阶段,算法对模板图像建立多尺度表示,各尺度下的图像进行区域划分,在每个区域内采用ORB（oriented FAST and rotated BRIEF）方法提取特征点并生成描述子,由此构建图像特征点的分级分区管理模式。在实时跟踪阶段,对于当前摄像机获得的图像,首先定位该图像所对应的尺度范围,在相应尺度范围内确定与当前图像重叠度大的图像区域,然后将当前图像与模板图像对应的尺度与区域中的特征点集进行匹配,最后根据匹配点对计算摄像机的位姿。结果 利用公开图像数据库（stanford mobile visual search dataset）中不同分辨率的模板图像及更多图像进行实验,结果表明,本文算法性能稳定,配准误差在1个像素左右;系统运行帧率总体稳定在2030 帧/s。结论 与多种经典算法对比,新方法能够更好地定位图像匹配尺度与区域,采用这种局部特征点匹配的方法在配准精度与计算效率方面比现有方法有明显提升,并且当模板图像分辨率较高时性能更好,特别适合移动增强现实应用。     

Realtime Object-aware Monocular Depth Estimation in Onboard Systems

This paper proposes the object depth estimation in real-time, using only a monocular camera in an onboard computer with a low-cost GPU. Our algorithm estimates scene depth from a sparse feature-based visual odometry algorithm and detects/tracks objects’ bounding box by utilizing the existing object detection algorithm in parallel. Both algorithms share their results, i.e., feature, motion, and bounding boxes, to handle static and dynamic objects in the scene. We validate the scene depth accuracy of sparse features with KITTI and its ground-truth depth map made from LiDAR observations quantitatively, and the depth of detected object with the Hyundai driving datasets and satellite maps qualitatively. We compare the depth map of our algorithm with the result of (un-) supervised monocular depth estimation algorithms. The validation shows that our performance is comparable to that of monocular depth estimation algorithms which train depth indirectly (or directly) from stereo image pairs (or depth image), and better than that of algorithms trained with monocular images only, in terms of the error and the accuracy. Also, we confirm that our computational load is much lighter than the learning-based methods, while showing comparable performance.

     

Image feature detection from phase congruency based on two-dimensional Hilbert transform

The theory of phase congruency is that features such as step edges, roofs, and deltas always reach the maximum phase of image harmonic components. We propose a modified algorithm of phase congruency to detect image features based on two-dimensional (2-D) discrete Hilbert transform. Windowing technique is introduced to locate image features in the algorithm. Local energy is obtained by convoluting original image with two operators of removing direct current (DC) component over current window and 2-D Hilbert transform, respectively. Then, local energy is divided with the sum of Fourier amplitude of current window to retrieve the value of phase congruency. Meanwhile, we add the DC component of current window on original image to the denominator of phase congruency model to reduce the noise. Finally, the proposed algorithm is compared with some existing algorithm in systematical way. The experimental results of images in Berkeley Segmentation Dataset (BSDS) and remotely sensed images show that this algorithm is readily to detect image features.     

Non-Rigid Stereo Factorization

In this paper we address the problem of recovering 3D non-rigid structure from a sequence of images taken with a stereo pair. We have extended existing non-rigid factorization algorithms to the stereo camera case and presented an algorithm to decompose the measurement matrix into the motion of the left and right cameras and the 3D shape, represented as a linear combination of basis-shapes. The added constraints in the stereo camera case are that both cameras are viewing the same structure and that the relative orientation between both cameras is fixed. Our focus in this paper is on the recovery of flexible 3D shape rather than on the correspondence problem. We propose a method to compute reliable 3D models of deformable structure from stereo images. Our experiments with real data show that improved reconstructions can be achieved using this method. The algorithm includes a non-linear optimization step that minimizes image reprojection error and imposes the correct structure to the motion matrix by choosing an appropriate parameterization. We show that 3D shape and motion estimates can be successfully disambiguated after bundle adjustment and demonstrate this on synthetic and real image sequences. While this optimization step is proposed for the stereo camera case, it can be readily applied to the case of non-rigid structure recovery using a monocular video sequence. Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.     

单目视觉定位方法研究综述

根据单目视觉定位所用图像帧数不同把定位方法分为基于单帧图像的定位和基于双帧或多帧图像的定位两类。单帧图像定位常利用已知的点特征、直线特征或曲线特征与其在图像上的投影关系进行定位,其中用点特征和直线特征的定位方法简单有效而应用较多;基于双帧或多帧图像的定位方法因其操作复杂、精度不高而研究的还较少。通过对各方法的介绍和评述,为单目视觉定位问题的研究提供参考。     

Large-Scale 6-DOF SLAM With Stereo-in-Hand

In this paper, we describe a system that can carry out simultaneous localization and mapping (SLAM) in large indoor and outdoor environments using a stereo pair moving with 6 DOF as the only sensor. Unlike current visual SLAM systems that use either bearing-only monocular information or 3-D stereo information, our system accommodates both monocular and stereo. Textured point features are extracted from the images and stored as 3-D points if seen in both images with sufficient disparity, or stored as inverse depth points otherwise. This allows the system to map both near and far features: the first provide distance and orientation, and the second provide orientation information. Unlike other vision-only SLAM systems, stereo does not suffer from “scale drift” because of unobservability problems, and thus, no other information such as gyroscopes or accelerometers is required in our system. Our SLAM algorithm generates sequences of conditionally independent local maps that can share information related to the camera motion and common features being tracked. The system computes the full map using the novel conditionally independent divide and conquer algorithm, which allows constant time operation most of the time, with linear time updates to compute the full map. To demonstrate the robustness and scalability of our system, we show experimental results in indoor andoutdoor urban environments of 210 m and 140 m loop trajectories, with the stereo camera being carried in hand by a person walking at normal walking speeds of 4--5 km/h.