Dense disparity estimation with a divide-and-conquer disparityspace image technique

A new divide-and-conquer technique for disparity estimation is proposed in this paper. This technique performs feature matching following the high confidence first principle, starting with the strongest feature point in the stereo pair of scanlines. Once the first matching pair is established, the ordering constraint in disparity estimation allows the original intra-scanline matching problem to be divided into two smaller subproblems. Each subproblem can then be solved recursively until there is no reliable feature point within the subintervals. This technique is very efficient for dense disparity map estimation for stereo images with rich features. For general scenes, this technique can be paired up with the disparity-space image (DSI) technique to compute dense disparity maps with integrated occlusion detection. In this approach, the divide-and-conquer part of the algorithm handles the matching of stronger features and the DSI-based technique handles the matching of pixels in between feature points and the detection of occlusions. An extension to the standard disparity-space technique is also presented to compliment the divide-and-conquer algorithm. Experiments demonstrate the effectiveness of the proposed divide-and-conquer DSI algorithm     

基于无人机双目图像的线目标测量的研究

利用无人机双目图像实现线目标的测量对输电线路巡检具有重要的意义。为提高无人机双目图像下线目标的测量精度,改进Census立体匹配算法,在代价聚合过程中,首先对聚合窗口中的初始匹配代价进行异常筛选,然后计算聚合代价值进而生成视差图,实验证明改进立体匹配算法,提高图像立体匹配精度,且平均误匹配率为5.79%;在线目标测量方面,针对线目标视差图存在的缺陷,提出一种基于目标识别的线目标视差图优化算法,该算法依据目标识别获取线目标视差图,然后根据四个原则进行优化处理,最后将优化后的线目标视差图用于测量,实验证明采用优化后的线目标视差图测量得到结果要优于直接采用视差图得到测量结果。     

基于模糊判别的立体匹配算法

立体视觉一直是计算机视觉领域所研究的一个中心问题,而立体匹配则是立体视觉技术中最关键也是最困难的部分,就得到适用于基于图象绘制技术中视图合成的准确、高密度视差图（Disparity Map）而言,现有的一些方法存在一定的局限性。考虑到立体匹配过程中存在的不确定性和模糊性,本文将已获得广泛应用的模糊理论引入立体匹配领域,提出了基于模糊判别的立体匹配算法,并用实际图象与合成图象进行了实验验证,结果表明该算法效果良好,具有实用价值。     

基于自适应聚合的立体视觉合作算法

提出了一种恢复高质量稠密视差图的立体视觉合作算法.该算法采用基于形态学相似性的自适应加权方法,迭代地进行局部邻域的自适应聚合和抑制放大,实现高效率和高质量稠密视差图计算.将该算法推广到三目摄像机立体匹配系统中,通过重建摄像机坐标系实现图像校正,并根据连续性假设和唯一性假设,建立视差空间中的支持关系和三目摄像机之间的抑制关系.实验结果表明,三目立体合作算法能够得到精确的场景视差映射,并可以实现多基线方向的遮挡检测.该算法特别适用于由多个廉价摄像机组成的立体视觉系统,在几乎不增加软件和硬件资源的情况下,就可以得到高质量的稠密视差图.     

Mutual information based registration of multimodal stereo videos for person tracking

Research presented in this paper deals with the systematic examination, development, and evaluation of a novel multimodal registration approach that can perform accurately and robustly for relatively close range surveillance applications. An analysis of multimodal image registration gives insight into the limitations of assumptions made in current approaches and motivates the methodology of the developed algorithm. Using calibrated stereo imagery, we employ maximization of mutual information in sliding correspondence windows that inform a disparity voting algorithm to demonstrate successful registration of objects in color and thermal imagery. Extensive evaluation of scenes with multiple objects at different depths and levels of occlusion shows high rates of successful registration. Ground truth experiments demonstrate the utility of the disparity voting techniques for multimodal registration by yielding qualitative and quantitative results that outperform approaches that do not consider occlusions. A basic framework for multimodal stereo tracking is investigated and promising experimental studies show the viability of using registration disparity estimates as a tracking feature.     

一种用于立体图像匹配的改进稀疏匹配算法

立体匹配有着广泛的应用前景,是计算机视觉领域的研究热点。立体匹配是立体视觉中最为关键和困难的一步,它的目标是计算标识匹配像素位置的视差图。文中提出的立体匹配算法基于置信传播（Belief Propagation,BP）。左图像首先经过非均匀采样,得到一个内容自适应的网格近似表示。算法的关键是使用基于置信传播的立体匹配算法,匹配稀疏的左图像和右图像得到稀疏视差图。通过左图像得到网格,稀疏视差图可以经过简单的插值得到稠密视差图。实验结果表明,该方法与现有稀疏立体匹配技术相比在视差图质量上平均有40％的提高。     

Dense Features for Semi-Dense Stereo Correspondence

We present a new feature based algorithm for stereo correspondence. Most of the previous feature based methods match sparse features like edge pixels, producing only sparse disparity maps. Our algorithm detects and matches dense features between the left and right images of a stereo pair, producing a semi-dense disparity map. Our dense feature is defined with respect to both images of a stereo pair, and it is computed during the stereo matching process, not a preprocessing step. In essence, a dense feature is a connected set of pixels in the left image and a corresponding set of pixels in the right image such that the intensity edges on the boundary of these sets are stronger than their matching error (which is the difference in intensities between corresponding boundary pixels). Our algorithm produces accurate semi-dense disparity maps, leaving featureless regions in the scene unmatched. It is robust, requires little parameter tuning, can handle brightnessdifferences between images, nonlinear errors, and is fast (linear complexity).     

Real-time joint disparity and disparity flow estimation on programmable graphics hardware

Disparity flow depicts the 3D motion of a scene in the disparity space of a given view and can be considered as view-dependent scene flow. A novel algorithm is presented to compute disparity maps and disparity flow maps in an integrated process. Consequently, the disparity flow maps obtained helps to enforce the temporal consistency between disparity maps of adjacent frames. The disparity maps found also provides the spatial correspondence information that can be used to cross-validate disparity flow maps of different views. Two different optimization approaches are integrated in the presented algorithm for searching optimal disparity values and disparity flows. The local winner-take-all approach runs faster, whereas the global dynamic programming based approach produces better results. All major computations are performed in the image space of the given view, leading to an efficient implementation on programmable graphics hardware. Experimental results on captured stereo sequences demonstrate the algorithm’s capability of estimating both 3D depth and 3D motion in real-time. Quantitative performance evaluation using synthetic data with ground truth is also provided.     

Fast stereo matching using adaptive guided filtering

Dense disparity map is required by many great 3D applications. In this paper, a novel stereo matching algorithm is presented. The main contributions of this work are three-fold. Firstly, a new cost-volume filtering method is proposed. A novel concept named “two-level local adaptation” is introduced to guide the proposed filtering approach. Secondly, a novel post-processing method is proposed to handle both occlusions and textureless regions. Thirdly, a parallel algorithm is proposed to efficiently calculate an integral image on GPU, and it accelerates the whole cost-volume filtering process. The overall stereo matching algorithm generates the state-of-the-art results. At the time of submission, it ranks the 10th among about 152 algorithms on the Middlebury stereo evaluation benchmark, and takes the 1st place in all local methods. By implementing the entire algorithm on the NVIDIA Tesla C2050 GPU, it can achieve over 30 million disparity estimates per second (MDE/s).     

一种室外非理想光照条件下的立体匹配算法

针对室外非理想光照条件和图像低纹理、低对比度造成立体匹配效果较差的问题,提出一种HSL(色相-饱和度-亮度)颜色空间下基于边缘图分割的立体匹配算法.区别于传统的RGB颜色空间下基于像素强度的度量方式,该算法采用一种HSL颜色空间下的像素非相似性度量公式来获得匹配代价,然后基于左右输入图像的边缘检测结果进行图像区域分割和立体匹配.在实验中采用一系列不同光照条件的图片集和具有明显低纹理区域的图片集,对本文算法与现有算法进行对比验证.实验结果证明,该算法能够得到比较理想的视差图,对非理想的光照条件和低纹理图像具有很好的鲁棒性,并且基本达到实时性要求.     

一种基于特征约束的立体匹配算法

立体匹配一直是计算机视觉领域的一个中心研究问题,为了得到适用于基于图象绘制技术的视图合成高密度视差图,提出了基于边缘特征约束的立体西欧算法,该方法首先利用基于特征技术来得到边缘特征点的准确视差图,然后在边缘特征点视差图的约束下,对非边缘特征点采用区域相关算法进行匹配,这样既缩小了匹配搜索空间,又保证了匹配的可靠性,边缘特征点和边缘特征点的匹配采用双向匹配技术又进一步保证了匹配的可靠性,实验结果表明,该算法效果良好,有实用价值。     

基于区域间协同优化的立体匹配算法

提出了一种基于分割区域间协同优化的立体匹配算法. 该算法以图像区域为匹配基元, 利用区域的彩色特征以及相邻区域间应满足的平滑和遮挡关系定义了区域的匹配能量函数, 并引入区域之间的合作竞争机制, 通过协同优化使所定义的匹配能量极小化, 从而得到比较理想的视差结果. 算法首先对参考图像进行分割, 利用相关法得到各分割区域的初始匹配; 然后用平面模型对各区域的视差进行拟合, 得到各区域的视差平面参数; 最后, 基于协同优化的思想, 采用局部优化的方法对各区域的视差平面参数进行迭代优化, 直至得到比较合理的视差图为止. 采用Middlebury test set进行的实验结果表明, 该方法在性能上可以和目前最好的立体匹配算法相媲美, 得到的视差结果接近于真实视差.     

基于视差点的大遮挡检测和立体匹配方法

提出一种基于视差点的方法来解决在高质量立体视差图生成过程中所出现的遮挡问题.首先证明同名核线对应的视差函数曲线可近似为一条分段直线,然后在此基础上引出视差点的概念.在视差点的结构中利用两个参数分别描述左右遮挡量,使得所提出的方法能够很好地解决遮挡问题.通过分析视差点及其邻域的灰度特性,提出一种分层假设证实和Marquardt-Levenberg(M-L)算法相结合的方法从同名核线图像中提取出候选视差点,然后采用不定期的动态规划(dynamic programming,简称DP)算法获得核线最优的视差函数.利用国际标准数据对提出的方法进行了测试,并与其他方法作比较,实验结果表明,它的匹配效果是目前核线最优方法中最好的,仅差于几种优秀的全局最优方法,但其计算复杂度要远低于全局的方法.     

Hierarchical stereo matching with image bit-plane slicing

We propose a new stereo matching framework based on image bit-plane slicing. A pair of image sequences with various intensity quantization levels constructed by taking different bit-rate of the images is used for hierarchical stereo matching. The basic idea is to use the low bit-rate image pairs to compute rough disparity maps. The hierarchical matching strategy is then carried out iteratively to update the low confident disparities with the information provided by extra image bit-planes. It is shown that, depending on the stereo matching algorithms, even the image pairs with low intensity quantization are able to produce fairly good disparity results. Consequently, variate bit-rate matching is performed only regionally in the images for each iteration, and the average image bit-rate for disparity computation is reduced. Our method provides a hierarchical matching framework and can be combined with the existing stereo matching algorithms. Experiments on Middlebury datasets show that the proposed technique gives good results compared to the conventional full bit-rate matching.     

FPGA based disparity map computation with vergence control

Depth estimation in a scene using image pairs acquired by a stereo camera setup, is one of the important tasks of stereo vision systems. The disparity between the stereo images allows for 3D information acquisition which is indispensable in many machine vision applications. Practical stereo vision systems involve wide ranges of disparity levels. Considering that disparity map extraction of an image is a computationally demanding task, practical real-time FPGA based algorithms require increased device utilization resource usage, depending on the disparity levels operational range, which leads to significant power consumption. In this paper a new hardware-efficient real-time disparity map computation module is developed. The module constantly estimates the precisely required range of disparity levels upon a given stereo image set, maintaining this range as low as possible by verging the stereo setup cameras axes. This enables a parallel-pipelined design, for the overall module, realized on a single FPGA device of the Altera Stratix IV family. Accurate disparity maps are computed at a rate of more than 320 frames per second, for a stereo image pair of 640 × 480 pixels spatial resolution with a disparity range of 80 pixels. The presented technique provides very good processing speed at the expense of accuracy, with very good scalability in terms of disparity levels. The proposed method enables a suitable module delivering high performance in real-time stereo vision applications, where space and power are significant concerns.     

基于双序列比对算法的立体图像匹配方法*

在分析现有立体匹配方法的基础上,提出一种基于双序列比对算法的立体图像匹配方法。将立体图像对中同名极线上的像素灰度值看做是一对字符序列,使用基于动态规划思想的双序列比对算法对这些对字符序列进行匹配,以获取立体图像视差。为验证该方法的可行性和适用性,采用人脸立体图像对进行实验。实验结果表明,使用该方法进行立体图像匹配能获得光滑的、稠密的视差图。基于动态规划思想的双序列比对算法能够有效地解决立体图像匹配问题,从而为图像的立体匹配提供了一个实用有效的方法。     

Improving Border Localization of Multi-Baseline Stereo Using Border-Cut

This paper presents a novel algorithm that improves the localization of disparity discontinuities of disparity maps obtained by multi-baseline stereo. Rather than associating a disparity label to every pixel of a disparity map, it associates a position to every disparity discontinuity. This formulation allows us to find an approximate solution to a 2D labeling problem with robust smoothing term by minimizing multiple 1D problems, thus making possible the use of dynamic programming. Dynamic programming allows the efficient computation of the visibility of most of the cameras during the minimization. The proposed algorithm is not a stereo matcher on it own since it requires an initial disparity map. Nevertheless, it is a very effective way of improving the border localization of disparity maps obtained from a large class of stereo matchers. Whilst the proposed minimization strategy is particularly suitable for stereo with occlusion, it may be used for other labeling problems.     

提取轮廓图背景控制点的立体匹配算法

针对动态规划匹配算法的误匹配与狭窄遮挡物问题,提出一种在轮廓图中提取背景控制点的立体匹配算法,在立体图像对的轮廓图中选择背景控制点,利用动态规划在视差空间图像中搜索最优路径,根据视差约束以及狭窄遮挡物的判定公式完善视差图。仿真实验结果表明,该算法能够降低在视差不连续区域匹配上的误匹配率。     

Estimating optimal parameters for MRF stereo from a single image pair

This paper presents a novel approach for estimating the parameters for MRF-based stereo algorithms. This approach is based on a new formulation of stereo as a maximum a posterior (MAP) problem in which both a disparity map and MRF parameters are estimated from the stereo pair itself. We present an iterative algorithm for the MAP estimation that alternates between estimating the parameters while fixing the disparity map and estimating the disparity map while fixing the parameters. The estimated parameters include robust truncation thresholds for both data and neighborhood terms, as well as a regularization weight. The regularization weight can be either a constant for the whole image or spatially-varying, depending on local intensity gradients. In the latter case, the weights for intensity gradients are also estimated. Our approach works as a wrapper for existing stereo algorithms based on graph cuts or belief propagation, automatically tuning their parameters to improve performance without requiring the stereo code to be modified. Experiments demonstrate that our approach moves a baseline belief propagation stereo algorithm up six slots in the Middlebury rankings     

Adaptive disparity computation using local and non-local cost aggregations

A new method is proposed to adaptively compute the disparity of stereo matching by choosing one of the alternative disparities from local and non-local disparity maps. The initial two disparity maps can be obtained from state-of-the-art local and non-local stereo algorithms. Then, the more reasonable disparity is selected. We propose two strategies to select the disparity. One is based on the magnitude of the gradient in the left image, which is simple and fast. The other utilizes the fusion move to combine the two proposal labelings (disparity maps) in a theoretically sound manner, which is more accurate. Finally, we propose a texture-based sub-pixel refinement to refine the disparity map. Experimental results using Middlebury datasets demonstrate that the two proposed selection strategies both perform better than individual local or non-local algorithms. Moreover, the proposed method is compatible with many local and non-local algorithms that are widely used in stereo matching.