Dense disparity estimation with a divide-and-conquer disparityspace image technique

A new divide-and-conquer technique for disparity estimation is proposed in this paper. This technique performs feature matching following the high confidence first principle, starting with the strongest feature point in the stereo pair of scanlines. Once the first matching pair is established, the ordering constraint in disparity estimation allows the original intra-scanline matching problem to be divided into two smaller subproblems. Each subproblem can then be solved recursively until there is no reliable feature point within the subintervals. This technique is very efficient for dense disparity map estimation for stereo images with rich features. For general scenes, this technique can be paired up with the disparity-space image (DSI) technique to compute dense disparity maps with integrated occlusion detection. In this approach, the divide-and-conquer part of the algorithm handles the matching of stronger features and the DSI-based technique handles the matching of pixels in between feature points and the detection of occlusions. An extension to the standard disparity-space technique is also presented to compliment the divide-and-conquer algorithm. Experiments demonstrate the effectiveness of the proposed divide-and-conquer DSI algorithm     

一种利用动态规划和左右一致性的立体匹配算法

立体匹配是计算机视觉领域研究的一个重要课题,为了得到准确、稠密的视差图,提出了一种利用动态规划和左右一致性的立体匹配算法。该算法首先分别以左、右图像为基元图像,计算各自的视差空间图像,在视差空间图像上利用动态规划,计算得到左视差图和右视差图;然后通过使用左右视差图之间的一致性关系,消除误匹配点,得到较为准确的部分视差图;最后利用视差图的顺序约束关系,给出未匹配视差点的搜索空间计算方法,并利用一种简单有效的方法来计算这些点的视差值。在一些标准立体图像对上所做的实验结果表明,该算法效果良好。     

Dense Features for Semi-Dense Stereo Correspondence

We present a new feature based algorithm for stereo correspondence. Most of the previous feature based methods match sparse features like edge pixels, producing only sparse disparity maps. Our algorithm detects and matches dense features between the left and right images of a stereo pair, producing a semi-dense disparity map. Our dense feature is defined with respect to both images of a stereo pair, and it is computed during the stereo matching process, not a preprocessing step. In essence, a dense feature is a connected set of pixels in the left image and a corresponding set of pixels in the right image such that the intensity edges on the boundary of these sets are stronger than their matching error (which is the difference in intensities between corresponding boundary pixels). Our algorithm produces accurate semi-dense disparity maps, leaving featureless regions in the scene unmatched. It is robust, requires little parameter tuning, can handle brightnessdifferences between images, nonlinear errors, and is fast (linear complexity).     

Trinocular Stereo Using Shortest Paths and the Ordering Constraint

This paper describes a new algorithm for disparity estimation using trinocular stereo. The three cameras are placed in a right angled configuration. A graph is then constructed whose nodes represent the individual pixels and whose edges are along the epipolar lines. Using the well known uniqueness and ordering constraint for pair by pair matches simultaneously, a path with the least matching cost is found using dynamic programming and the disparity filled along the path. This process is repeated iteratively until the disparity at all the pixels are filled up. To demonstrate the effectiveness of our approach, we present results from real world images and compare it with the traditional line by line stereo using dynamic programming.     

Matching free stereovision for detecting obstacles on a ground plane

This paper presents a method for detecting obstacles on a ground plane from a stereo pair of images. Although we use stereovision, the obstacle detection algorithm relies neither on stereo matching nor 3D reconstruction. The principle here is to apply the projective transformation constraining the left and right images to obtain a frame of superimposed features (e.g. edges). By analysing feature superimposition after the projective transformation, a free moving space or space occupied by obstacles/occluded features can be determined.     

自适应窗口的时间规整立体匹配算法

针对立体视觉中图像对应点的误匹配问题,以时间规整算法(DTW)为基础,提出了自适应窗口的立体匹配算法．根据外极线的约束,在自适应窗口内采用灰度相关技术得到长度不相等的两个灰度段作为相容的匹配序列;利用动态规划法及连续性约束寻找一条最佳的匹配路径．根据回溯得到的匹配路径及其坐标值得到高密度视差图．实验结果表明,该算法具有较高的运行效率和良好的匹配效果．     

基于双目视觉的基准差梯度立体匹配法􀀂

因灰度相关只是从一个侧面来描述左右图像特征点区域之间的灰度相似性,没有考虑特征点之间的空间相关性,因此利用灰度间的相似性作为测量标准进行匹配,不可避免地出现误匹配,提出了在进行双目视觉立体匹配时,采用灰度相关匹配技术,提取复峰特征点作为初始匹配集,采用视差梯度有限约束优化初始匹配集.利用左右图像一对已知对应基准点,通过计算基准点与复峰集各点间的基准差梯度,采用基准差梯度极小化评判标准,确定唯一匹配,并将匹配结果确定为新的基准点以不断更新基准点,直至左(右)图像特征点匹配完毕.通过分别对一幅弱纹理实际自然图像及已知三维坐标标准件的三维重建,证实了所提方法的有效性和可靠性.     

A Maximum Likelihood Stereo Algorithm

A stereo algorithm is presented that optimizes a maximum likelihood cost function. The maximum likelihood cost function assumes that corresponding features in the left and right images are normally distributed about a common true value and consists of a weighted squared error term if two features are matched or a (fixed) cost if a feature is determined to be occluded. The stereo algorithm finds the set of correspondences that maximize the cost function subject to ordering and uniqueness constraints. The stereo algorithm is independent of the matching primitives. However, for the experiments described in this paper, matching is performed on the $cf4$individual pixel intensities.$cf3$ Contrary to popular belief, the pixel-based stereo appears to be robust for a variety of images. It also has the advantages of (i) providing adensedisparity map, (ii) requiringnofeature extraction, and (iii)avoidingthe adaptive windowing problem of area-based correlation methods. Because feature extraction and windowing are unnecessary, a very fast implementation is possible. Experimental results reveal that good stereo correspondences can be found using only ordering and uniqueness constraints, i.e., withoutlocalsmoothness constraints. However, it is shown that the original maximum likelihood stereo algorithm exhibits multiple global minima. The dynamic programming algorithm is guaranteed to find one, but not necessarily the same one for each epipolar scanline, causing erroneous correspondences which are visible as small local differences between neighboring scanlines. Traditionally, regularization, which modifies the original cost function, has been applied to the problem of multiple global minima. We developed several variants of the algorithm that avoid classical regularization while imposing several global cohesiveness constraints. We believe this is a novel approach that has the advantage of guaranteeing that solutions minimize the original cost function and preserve discontinuities. The constraints are based on minimizing the total number of horizontal and/or vertical discontinuities along and/or between adjacent epipolar lines, and local smoothing is avoided. Experiments reveal that minimizing the sum of the horizontal and vertical discontinuities provides the most accurate results. A high percentage of correct matches and very little smearing of depth discontinuities are obtained. An alternative to imposing cohesiveness constraints to reduce the correspondence ambiguities is to use more than two cameras. We therefore extend the two camera maximum likelihood toNcameras. TheN-camera stereo algorithm determines the “best” set of correspondences between a given pair of cameras, referred to as the principal cameras. Knowledge of the relative positions of the cameras allows the 3D point hypothesized by an assumed correspondence of two features in the principal pair to be projected onto the image plane of the remainingN− 2 cameras. TheseN− 2 points are then used to verify proposed matches. Not only does the algorithm explicitly model occlusion between features of the principal pair, but the possibility of occlusions in theN− 2 additional views is also modeled. Previous work did not model this occlusion process, the benefits and importance of which are experimentally verified. Like other multiframe stereo algorithms, the computational and memory costs of this approach increase linearly with each additional view. Experimental results are shown for two outdoor scenes. It is clearly demonstrated that the number of correspondence errors is significantly reduced as the number of views/cameras is increased.     

一种快速立体视觉边缘匹配算法

提出了一种立体视觉边缘匹配快速算法。通过小波变换,得到了图像的边缘和边缘幅角 并定义了边缘幅角约束。由视差梯度的分布密度函数,导出了左图像连续边缘上相邻两点在右图像 中的对应点的坐标间的相互约束关系,从而限定了右图像中匹配点的搜索范围。最后给出了基于视 差梯度约束和边缘幅角约束的快速边缘匹配算法。     

基于立体匹配的低纹理图像重构算法

针对立体匹配中低纹理区域容易产生误匹配及传统动态规划固有的条纹问题,提出一种改进的基于双目立体视觉的低纹理图像三维重构算法。该算法首先基于像素间相似度和像素自身特异性计算匹配代价并引入一种自适应多边形支撑区域聚集匹配度。然后采用一种全局意义的简单树形动态规划进行逐点匹配。最后基于左右一致性准则运用一种简单有效的视差校正方法消除误匹配得到最终视差图。实验证明将算法运用于实拍低纹理灰度图像的匹配,得到轮廓光滑清晰的三维点云,说明该方法的适用性。     

Depth Discontinuities by Pixel-to-Pixel Stereo

An algorithm to detect depth discontinuities from a stereo pair of images is presented. The algorithm matches individual pixels in corresponding scanline pairs, while allowing occluded pixels to remain unmatched, then propagates the information between scanlines by means of a fast postprocessor. The algorithm handles large untextured regions, uses a measure of pixel dissimilarity that is insensitive to image sampling, and prunes bad search nodes to increase the speed of dynamic programming. The computation is relatively fast, taking about 600 nanoseconds per pixel per disparity on a personal computer. Approximate disparity maps and precise depth discontinuities (along both horizontal and vertical boundaries) are shown for several stereo image pairs containing textured, untextured, fronto-parallel, and slanted objects in indoor and outdoor scenes.     

Automatic feature matching in uncalibrated stereo vision through the use of color

One difficult problem in stereo vision is how to establish correspondence between features extracted from a pair of images. The difficulty is due to ambiguities or inconsistencies of available information on images. In this paper, we invetigate stereo correspondence problem in the framework of color stereo vision. We propose the use of a matching consistence (MC) constraint in RGB color space and the generalized epipolar geometry to develop an automatic feature matching algorithm.     

基于直线间结构信息的立体视觉图像动态匹配方法

针对立体视觉匹配问题，介绍一种改进的动态规划图像匹配方法，它将边缘直线相似测度分为局部相似测度和全局相似测度，在后者中加入图像边缘直线之问的结构关系信息，并在动态搜索最优匹配路径的过程中利用结构关系约束删除不合理的匹配路径。仿真实验结果证明，采用该方法解决立体视觉中边缘线段的匹配问题，不仅提高了匹配的准确率，而且大大减少了匹配时间。     

基于自适应匹配窗及多特征融合的立体匹配*

针对局部立体匹配方法中存在的匹配窗口大小选择困难、边缘处视差模糊及弱纹理区域、斜面或曲面匹配精度较低等问题,提出基于CIELAB空间下色度分割的自适应窗选取及多特征融合的局部立体匹配算法.首先,在CIELAB空间上对立体图像对进行色度分割,依据同质区域的分布获取初始匹配支持域,同时估计遮挡区域,更新匹配支持域.然后,基于更新后的匹配支持域,采用自适应权值的线性加权多特征融合匹配方法得到初始视差图.最后,利用左右视差一致性检测方法进行误匹配检验,利用基于分割的均值滤波器进行视差优化及细化,得到稠密匹配视差结果.实验表明文中算法有效,匹配精度较高,尤其在弱纹理区域及斜面等情况下匹配效果较好.     

基于Zernike矩的区域匹配方法

在基于区域的立体匹配中,由于遮掩、区域变形及光照条件会对匹配算法造成很大的影响,而传统的顺序性约束、唯一性约束、外极线约束和邻域约束并不能很好地解决这些问题,而近几年提出的相对位置约束虽能解决其中大部分问题,但对于区域的遮掩情况依然效果不佳。为此提出了一种新的基于Zernike矩的区域匹配算法,该算法在相对位置约束的基础上,采用中心距离和Zernike矩构造了新的费用函数,并提出根据匹配区域之间中心距离的大小来动态评判费用函数的权重系数值,从而提高了算法的性能。实验结果表明,该算法优于原方法,且对于区域的遮掩和变形情况都具备更好的识别性能,是一种行之有效的区域匹配算法。     

基于特征约束及区域相关的体视匹配方法

立体匹配是计算机视觉领域的一个关键问题,同时也是难点问题。为了得到准确的高密度视差图,通过对基于区域和基于特征的体视方法的讨论,综合两种方法的优点,提出了基于边缘特征约束及区域相关的立体匹配算法。该方法首先利用基于特征技术来得到边缘特征点,对边缘特征点再做灰度等区域相关匹配处理,然后在匹配的边缘特征点约束下,对非边缘特征点采用区域相关算法进行匹配,得到整体高密度视差图。这样既缩小了匹配搜索空间,又保证了匹配的可靠性。实验结果表明,该算法具有良好的效果和实用价值。     

一种基于特征约束的立体匹配算法

立体匹配一直是计算机视觉领域的一个中心研究问题,为了得到适用于基于图象绘制技术的视图合成高密度视差图,提出了基于边缘特征约束的立体西欧算法,该方法首先利用基于特征技术来得到边缘特征点的准确视差图,然后在边缘特征点视差图的约束下,对非边缘特征点采用区域相关算法进行匹配,这样既缩小了匹配搜索空间,又保证了匹配的可靠性,边缘特征点和边缘特征点的匹配采用双向匹配技术又进一步保证了匹配的可靠性,实验结果表明,该算法效果良好,有实用价值。     

基于损失自注意力机制的立体匹配算法研究

针对现有的立体匹配算法在阴影、物体边缘和光照反射等区域匹配困难且存在大量错误结果的问题,设计了一种可拆卸的损失自注意力网络（loss self-attention net,LSAnet）查找图像中的匹配困难区域。LSAnet的网络各层相互稠密连接,应用了空洞卷积来扩大感受野,并以立体匹配算法生成的损失分布为标签,能够动态地进行有监督训练,最终生成匹配困难区域掩膜辅助立体匹配网络进行更好的优化;同时,改进了立体匹配网络中经典的特征匹配代价卷结构,降低了后续3D卷积的计算负荷,提高了匹配效率。实验结果表明,该算法相比于基准算法精度更高,并且可以提高算法对于匹配困难区域的鲁棒性。     

Implementation of a 3D Vision System on DSPs TMS320C31

In this paper, the implementation of a new stereo vision process on a specialized architecture which comprises of three DSPs TMS320C31 is described. The first step of our stereo vision system is a self-adaptive image segmentation algorithm based on a new concept that we call declivity. The second step is a new and fast stereo matching algorithm based on dynamic programming and using self-adaptive decision parameters. The goal of our work is to develop a stereo vision system that achieves an acceptable level of performance using a modest amount of hardware. This implementation is organized as follows: declivity extraction from the two stereo images is performed in parallel on two DSPs, one for the right image and the other for the left one. Then, the last DSP computes the declivity matching based on our dynamic programming method as well as the 3D maps calculation. Finally, experimental results obtained using real pairs of stereo images on a VME 150/40 Imaging Technology Vision System are presented. They show the feasibility and the effectiveness of our system. These results can surely be improved by using a new generation of DSP in order to consider real-time applications.     

Piecewise-planar reconstruction using two views

The article describes a reconstruction pipeline that generates piecewise-planar models of man-made environments using two calibrated views. The 3D space is sampled by a set of virtual cut planes that intersect the baseline of the stereo rig and implicitly define possible pixel correspondences across views. The likelihood of these correspondences being true matches is measured using signal symmetry analysis [1], which enables to obtain profile contours of the 3D scene that become lines whenever the virtual cut planes intersect planar surfaces. The detection and estimation of these lines cuts is formulated as a global optimization problem over the symmetry matching cost, and pairs of reconstructed lines are used to generate plane hypotheses that serve as input to PEARL clustering [2]. The PEARL algorithm alternates between a discrete optimization step, which merges planar surface hypotheses and discards detections with poor support, and a continuous optimization step, which refines the plane poses taking into account surface slant. The pipeline outputs an accurate semi-dense Piecewise-Planar Reconstruction of the 3D scene. In addition, the input images can be segmented into piecewise-planar regions using a standard labeling formulation for assigning pixels to plane detections. Extensive experiments with both indoor and outdoor stereo pairs show significant improvements over state-of-the-art methods with respect to accuracy and robustness.