Robust active stereo vision using Kullback-Leibler divergence

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.     

基于区域匹配的实时加速技术

针对区域立体匹配计算量大实时性差的困难,分析了相关匹配算法的实际工作过程,采用消除冗余因子和Box滤波、多级分辨率匹配减小计算复杂度,对算法结构进行了改进和优化,并利用超线程和OpenMP技术对算法进行了加速,提出了一种实时区域匹配算法.对算法进行实验,结果表明算法符合了视觉导航的准确性和实时性要求,并且对于提高其他区域匹配算法实时性也具有重要借鉴意义.     

改进的跨尺度引导滤波立体匹配算法

作为双目三维重建中的关键步骤,双目立体匹配算法完成了从平面视觉到立体视觉的转化.但如何平衡双目立体匹配算法的运行速度和精度仍然是一个棘手的问题.本文针对现有的局部立体匹配算法在弱纹理、深度不连续等特定区域匹配精度低的问题,并同时考虑到算法实时性,提出了一种改进的跨多尺度引导滤波的立体匹配算法.首先融合AD和Census变换两种代价计算方法,然后采用基于跨尺度的引导滤波进行代价聚合,在进行视差计算时通过制定一个判断准则判断图像中每一个像素点的最小聚合代价对应的视差值是否可靠,当判断对应的视差值不可靠时,对像素点构建基于梯度相似性的自适应窗口,并基于自适应窗口修正该像素点对应的视差值.最后通过视差精化得到最终的视差图.在Middlebury测试平台上对标准立体图像对的实验结果表明,与传统基于引导滤波器的立体匹配算法相比具有更高的精度.     

基于视觉的三维目标检测算法研究综述

基于视觉的目标检测是环境感知系统的重要组成,一直以来是计算机视觉、机器人等相关领域的研究热点。三维目标检测是在二维目标检测的基础上,增加目标尺寸、深度、姿态等信息的估计。相比于二维目标检测,三维目标检测在准确性、实时性等方面仍有较大的提升空间。系统总结了基于视觉的三维目标检测方法,调研了现有的基于单目视觉、双目、深度相机的三维目标检测方法,并依据室内外场景进行了分类。此外,在KITTI、SUN RGB-D等数据集上对最新的三维目标检测算法进行了对比分析,并针对目前算法中存在的难点和问题,讨论了未来的研究方向。     

基于立体视觉的在线实时测量系统设计与实现

为了满足深空探测器实时测量天体表面形貌的需求,设计并实现了一套基于立体视觉的在线实时测量原型系统。该系统通过立体相机实时获取空间天体的立体影像,利用每次观测的一组立体影像来重建其局部表面形状;再对每次重建的局部模型进行连接,得到空间天体完整的表面形貌模型。通过仿真实验验证了该系统的可行性,数据处理的速度与精度可以满足对深空目标进行实时测量的需要。     

基于双目内窥镜的散斑三维重建系统研究

传统的内窥镜只能提供清晰的图像,无法进行三维测量和三维重建.该文提出一种基于立体视觉原理的双目内窥镜系统,用于实现三维测量和三维重建,并开发了一套基于双目内窥镜的散斑三维重建系统.为了提高系统的标定精度和三维重建质量,该文提出一种高精度双目内窥镜标定参数优化方法及基于光轴双次旋转的立体校正算法.其中,测量系统由一个结构...     

基于FPGA的双目立体视觉系统

立体视觉的目的之一就是为了获得周围场景的3维信息,其关键在于匹配算法。然而即便是使用目前先进的通用处理器,其计算致密视差图所需的时间仍无法满足高速自主导航的需求。为了解决这个问题,提出了一种基于现场可编程门阵列(FPGA)的双目立体视觉系统的设计方案,同时介绍了系统的硬件结构,并在讨论区域匹配的快速算法的基础上,提出了基于FPGA的像素序列和并行窗口算法框架,用以实现零均值像素灰度差平方和(ZSSD)的匹配算法。该算法是先将视频信号经解码芯片生成场景立体图像对,并由FPGA来完成立体图像对的几何校正和ZSSD匹配算法,然后将获得的致密视差图通过PC I总线发送至上位机。实践表明,该算法效果好、速度快,不仅具有较强的鲁棒性,并且硬件系统性能稳定、可靠。此外,该方案还适用于像素灰度差的绝对值和(SAD)和像素灰度差的平方和(SSD)等多种传统区域匹配算法的快速实现和实时处理。     

Stereo vision during adverse weather — Using priors to increase robustness in real-time stereo vision

Stereo vision can deliver a dense 3D reconstruction of the environment in real-time for driver assistance as well as autonomous driving. Semi-Global Matching (SGM) is a popular method of choice for solving this task which is already in use for production vehicles. Despite the enormous progress in the field and the high level of performance of modern stereo methods, one key challenge remains: robust stereo vision in automotive scenarios during rain, snow and darkness. Under these circumstances, current methods generate strong temporal noise, many disparity outliers and false positives on object level. These problems are addressed in this work by regularizing stereo vision via prior information. We formulate a temporal prior and a scene prior, which we apply to SGM in order to overcome the deficiencies. The temporal prior integrates knowledge from the previous disparity map to exploit the high temporal correlation, the scene prior exploits knowledge of a representative traffic scene. Using these priors, the object detection rate improves significantly on a driver assistance dataset of 3000 frames including bad weather while reducing the rate of erroneous object detections. We also outperform the ECCV Robust Vision Challenge 2012 winner, iSGM, on this dataset. In addition, results are presented for the KITTI dataset, even showing improvements under good weather conditions when exploiting the temporal prior.We also show that the temporal and scene priors are easy and efficient to implement on a hybrid CPU/reconfigurable hardware platform. The use of these priors can be extended to other application areas such as mobile robotics.     

Accurate 3D-vision-based obstacle detection for an autonomous train

In this paper we present a 3D-vision based obstacle detection system for an autonomously operating train in open terrain environments. The system produces dense depth data in real-time from a stereo camera system with a baseline of 1.4 m to fulfill accuracy requirements for reliable obstacle detection 80 m ahead. On an existing high speed stereo engine, several modifications have been applied to significantly improve the overall performance of the system. Hierarchical stereo matching and slanted correlation masks increased the quality of the depth data in a way that the obstacle detection rate increased from 89.4% to 97.75% while the false positive detection rate could be kept as low as 0.25%. The evaluation results have been obtained from extensive real-world test data. An additional stereo matching speed-up of factor 2.15 was achieved and the overall latency of obstacle detection is considerably faster than 300 ms.     

Stereo vision based automation for a bin-picking solution

As flexibility becomes an important factor in factory automation, the bin-picking system, where a robot performs pick-and-place tasks for randomly piled parts in a bin through measuring the 3D pose of an object by a 3D vision sensor, has been actively studied. However, conventional bin-picking systems that are employed for particular tasks are limited by such things as the FOV (Field of View), the shape of landmark features, and computation time. This paper proposes a general-purpose stereo vision based bin-picking system. To detect the workpiece to be picked, a geometric pattern matching (GPM) method with respect to the 2D image with a wide FOV is applied. The accurate 3D pose of a selected workpiece among the pick-up candidates is acquired by measuring the 3D positions of three features in the workpiece using the stereo camera. In order to improve the 3D position estimation performance, the GPM method is also used instead of the stereo matching method. The multiple pattern registration and ellipse fitting techniques are additionally applied to increase the reliability. The grasp position of a workpiece without collision is determined using the pose of the object and the bin information. By using these methods a practical bin-picking strategy is established to operate robustly with minimum help from the human workers in the factory. Through experiments on commercial industrial workpieces and industrial robot, we validated that the proposed vision system accurately measures the 3D pose of part and the robot successfully manipulates the workpiece among randomly stacked parts.     

Vision Based Target Tracking and Collision Avoidance for Mobile Robots

A real-time object tracking and collision avoidance method is presented for mobile robot navigation in indoors environments using stereo vision and a laser sensor. Stereo vision is used to identify the target and to calculate its relative distance from the mobile robot while laser based range measurements are utilized to avoid collision with surrounding objects. The target is tracked by its predetermined or dynamically defined color. The mobile robot’s velocity is dynamically adjusted according to its distance from the target. Experimental results in indoor environments demonstrate the effectiveness of the method.     

视觉目标跟踪方法研究综述

目的 随着军事侦察任务设备的发展,红外与可见光侦察技术成为军事装备中的主要侦察手段。研究视觉目标跟踪技术对提高任务设备的全天候目标侦察、目标跟踪、目标定位等战场情报获取能力具有重要意义。目前,对视觉目标跟踪技术的研究越来越深入,目标跟踪的方法和种类也越来越丰富。本文对目前应用较为广泛的4种视觉目标跟踪方法进行研究综述,为后续国内外研究者对目标跟踪相关理论及发展研究工作提供基础。方法 通过对视觉目标跟踪技术难点问题进行分析,根据目标跟踪方法建模方式的不同,将视觉目标跟踪方法分为生成式模型方法与判别式模型方法。分别对生成式模型跟踪算法中的均值漂移目标跟踪方法和粒子滤波目标跟踪方法,判别式模型跟踪算法中的相关滤波目标跟踪方法和深度学习目标跟踪方法进行研究。首先分别对4种跟踪算法的基本原理进行介绍,然后针对4种跟踪算法基本原理的不足和对应目标跟踪中的难点问题进行分析,最后针对目标跟踪的难点问题,给出对应算法的主流改进方案。结果 针对视觉目标跟踪相关技术研究进展,结合无人机侦察任务需求,对跟踪算法实际应用中存在的重点解决问题与相关目标跟踪的难点问题进行分析,给出目前的解决方案与不足,探讨研究未来无人机目标侦察跟踪技术的发展方向。结论 视觉目标跟踪技术已经取得了显著的进展,在侦察任务中的应用越来越广泛。但目标跟踪技术仍然是非常具有挑战性的问题,目标跟踪中的相关理论有待进一步完善和改进,由于实际应用中的场景复杂,目标跟踪的难点问题的挑战性更大,因此容易导致跟踪效果不佳。针对不同的应用环境,结合具体不同军事装备的特点,研究相对精确和鲁棒并且满足实时性要求的视觉目标跟踪算法,对提升装备的全天候侦察目标信息获取能力具有重要意义。     

利用BLI快速相关的机器立体视觉方法

本文提出了两种BLI快速相关算法.一种是利用差累积向量,另一种是利用持续行程编码.对两种快速算法的计算量进行了分析,它们比经典二值图象相关技术快几十倍.对立体视觉连续性与唯一性原理在二值拉普拉斯图象相关方法中的具体应用提出了规划.实验结果证明了本文方法的有效性.     

Efficient normalized cross correlation calculation method for stereo vision based robot navigation

Stereo vision systems are widely used for autonomous robot navigation. Most of them apply local window based methods for real-time purposes. Normalized cross correlation (NCC) is notorious for its high computational cost, though it is robust to different illumination conditions between two cameras. It is rarely used in real-time stereo vision systems. This paper proposes an efficient normalized cross correlation calculation method based on the integral image technique. Its computational complexity has no relationship to the size of the matching window. Experimental results show that our algorithm can generate the same results as traditional normalized cross correlation with a much lower computational cost. Our algorithm is suitable for planet rover navigation.     

Epipolar geometry for prism-based single-lens stereovision

In order to simplify the design and implementation of a stereo vision system, prism has been used to capture stereo images with a single camera. This kind of system not only provides advantages over traditional two-camera stereo, but also reduces the complexity and cost of acquiring stereoscopic image. This paper investigated the characteristics of epipolar geometry for a single-lens prism-based stereovision. The prism was considered as a single optical lens. By analyzing each plane individually and then combining them together, an affine transformation matrix which can express the relationship between an object point and its image was derived. Then, the homography between object point and its image was established. Finally, the epipolar geometry as well as the epipolar rectification method was proposed. Experimental results verify that rectification of the image pair based on our proposed model can achieve better performance with much less geometric distortion.     

基于立体视觉的高精度标定与测量方法

立体视觉测量系统中,光学系统产生的畸变使目标的成像偏离了理论成像点,导致系统产生测量误差。针对提高系统测量精度的问题,提出一种基于立体视觉的测量方法。首先,根据标定板上各角点的像素分辨率,拟合整个成像平面的四次多项式,且多项式的系数与物体到相机的距离成比例;然后,应用双目测距原理,测量被测物体的纵向距离;最后,基于所得的多项式,应用单目相机测量待测物体的横向尺寸。实验结果表明,对于所提方法,当物体距离相机5 m以内时,其纵向距离误差可以减小到5%以内;当物体距离相机1 m时,其横向宽度测量误差在0.5 mm内,逼近理论最高分辨率。     

Using models to improve stereo reconstruction

The authors propose the combination of photometric and stereometric information to solve the stereo vision problem in the case of a man-made environment. A method to introduce geometrical models in the stereo process in order to improve the accuracy of the depth measurement and to extend the depth map to points where no measurements have been made is presented. This method is based on a parameterization of the object surfaces and relies on a systematic comparison of the result of a stereo process with the photometric (or gray-level) image. The proposed approach improves the accuracy of the stereo information and its density by introducing a hypothesis on the object surfaces. Two kinds of hypothesis are developed: planar and quadratic objects. Reconstructions of complex scenes are given     

Recurrent learning of context for salient region detection

In this paper, a novel type of saliency region detection method is proposed based on the recurrent learning of context. It aims to find the image regions that can represent the main content. It is different with previous definitions the goal of which is to either find fixation points or seek the dominant object. The regions should own semantic information, thus being a challenging task for computer vision, especially when the imaging quality is poor with complicated background clutter and uncontrolled viewing conditions. To improve attribute recognition given small-sized training data with poor-quality images, we formulate a joint recurrent learning model for exploring context and correlation, based on which salient region can be detected. Moreover, by the way of incorporating semantic information of image contents, an object oriented pooling strategy is proposed to further improve the performance. We conduct experiments on several challenging publically available saliency detection datasets and it demonstrates the effectiveness of our proposed saliency region detection method.     

基于改进SIFT特征的双目图像匹配算法

针对SIFT（尺度不变特征变换）算法无法准确定位物体形状特征的问题,提出了一种结合了Harris角点和SIFT算法的立体匹配方法。在DOG尺度空间提取Harris算子作为图像的特征点并为每个特征点定义主方向,计算出特征点的32维特征向量描述子并用BBF算法检索同名特征点之间的欧式距离进行匹配。在降低SIFT算法的时间复杂度的同时提高了算法提取特征点的形状意义,在双目图像匹配实验中取得了较好的结果。