基于改进绝对差值代价和动态窗口的立体匹配算法

针对传统的SAD局部立体匹配容易引起幅度失真、存在匹配窗口大小选择困难等问题,提出一种改进SAD局部立体匹配算法。首先在传统的SAD算法的基础上,提出利用像素灰度间欧氏距离的大小关系代替像素差值作为相似度量函数,很好地利用了邻近像素灰度值之间的连续性约束;在极限约束条件下,提出引导滤波器的动态匹配窗口的建立,能够很好地保持边缘特性;最后经过左右一致性检测策略来检测匹配异常点,再进一步平滑去噪,求得最终的视差图。实验结果表明,本文算法效率高、匹配精度高,对光照失真条件和边缘信息较多、深度不连续区域具有更好的鲁棒性。     

基于双目视觉的人脸三维重建

基于双目立体匹配算法PatchMatch算法,提出了一种获取人脸三维点云的算法。该算法对局部立体匹配算法PatchMatch进行了优化。该方法既不需要昂贵的设备,也不需要通用的人脸三维模型,而是结合了人脸的拓扑结构信息以及立体视觉局部优化算法。此方法采用非接触式的双目视觉采集技术获取左右视角的人脸图像,利用回归树集合(ensemble of regression trees,ERT)算法对人脸图像进行关键点定位,恢复人脸稀疏的视差估计,运用线性插值方法初步估计脸部的稠密视差值,并结合局部立体匹配算法对得到的视差结果进行平滑处理,重建人脸的三维点云信息。实验结果表明,这种算法能够还原出光滑的稠密人脸三维点云信息,在人脸Bosphorus数据库上取得了更加准确的人脸重建结果。     

基于自适应匹配窗及多特征融合的立体匹配*

针对局部立体匹配方法中存在的匹配窗口大小选择困难、边缘处视差模糊及弱纹理区域、斜面或曲面匹配精度较低等问题,提出基于CIELAB空间下色度分割的自适应窗选取及多特征融合的局部立体匹配算法.首先,在CIELAB空间上对立体图像对进行色度分割,依据同质区域的分布获取初始匹配支持域,同时估计遮挡区域,更新匹配支持域.然后,基于更新后的匹配支持域,采用自适应权值的线性加权多特征融合匹配方法得到初始视差图.最后,利用左右视差一致性检测方法进行误匹配检验,利用基于分割的均值滤波器进行视差优化及细化,得到稠密匹配视差结果.实验表明文中算法有效,匹配精度较高,尤其在弱纹理区域及斜面等情况下匹配效果较好.     

基于PatchMatch的半全局高效双目立体匹配算法

近年来双目立体匹配技术发展迅速,高精度、高分辨率、大视差的应用需求无疑对该技术的计算效率提出了更高的要求.由于传统立体匹配算法固有的计算复杂度正比于视差范围,已经难以满足高分辨率、大视差的应用场景.因此,从计算复杂度、匹配精度、匹配原理等多方面综合考虑,提出了一种基于PatchMatch的半全局双目立体匹配算法,在路径...     

根据灰度值信息自适应窗口的半全局匹配

目的 立体匹配算法是立体视觉研究的关键点,算法的匹配精度和速度直接影响3维重建的效果。对于传统立体匹配算法来说,弱纹理区域、视差深度不连续区域和被遮挡区域的匹配精度依旧不理想,为此选择具有全局匹配算法和局部匹配算法部分优点、性能介于两种算法之间、且鲁棒性强的半全局立体匹配算法作为研究内容,提出自适应窗口与半全局立体匹配算法相结合的改进方向。方法 以通过AD（absolute difference）算法求匹配代价的半全局立体匹配算法为基础,首先改变算法匹配代价的计算方式,研究窗口大小对算法性能的影响,然后加入自适应窗口算法,研究自适应窗口对算法性能的影响,最后对改进算法进行算法性能评价与比较。结果 实验结果表明,匹配窗口的选择能够影响匹配算法性能、提高算法的适用范围,自适应窗口的加入能够提高算法匹配精度特别是深度不连续区域的匹配精度,并有效降低算法运行时间,对Cones测试图像集,改进的算法较改进前误匹配率在3个测试区域平均减少2.29%;对于所有测试图像集,算法运行时间较加入自适应窗口前平均减少28.5%。结论 加入自适应窗口的半全局立体匹配算法具有更优的算法性能,能够根据应用场景调节算法匹配精度和匹配速度。     

Secrets of adaptive support weight techniques for local stereo matching

In recent years, local stereo matching algorithms have again become very popular in the stereo community. This is mainly due to the introduction of adaptive support weight algorithms that can for the first time produce results that are on par with global stereo methods. The crux in these adaptive support weight methods is to assign an individual weight to each pixel within the support window. Adaptive support weight algorithms differ mainly in the manner in which this weight computation is carried out.In this paper we present an extensive evaluation study. We evaluate the performance of various methods for computing adaptive support weights including the original bilateral filter-based weights, as well as more recent approaches based on geodesic distances or on the guided filter. To obtain reliable findings, we test these different weight functions on a large set of 35 ground truth disparity pairs. We have implemented all approaches on the GPU, which allows for a fair comparison of run time on modern hardware platforms. Apart from the standard local matching using fronto-parallel windows, we also embed the competing weight functions into the recent PatchMatch Stereo approach, which uses slanted sub-pixel windows and represents a state-of-the-art local algorithm. In the final part of the paper, we aim at shedding light on general points of adaptive support weight matching, which, for example, includes a discussion about symmetric versus asymmetric support weight approaches.     

Improving sub-pixel accuracy for long range stereo

Dense stereo algorithms are able to estimate disparities at all pixels including untextured regions. Typically these disparities are evaluated at integer disparity steps. A subsequent sub-pixel interpolation often fails to propagate smoothness constraints on a sub-pixel level.We propose to increase the sub-pixel accuracy in low-textured regions in four possible ways: First, we present an analysis that shows the benefit of evaluating the disparity space at fractional disparities. Second, we introduce a new disparity smoothing algorithm that preserves depth discontinuities and enforces smoothness on a sub-pixel level. Third, we present a novel stereo constraint (gravitational constraint) that assumes sorted disparity values in vertical direction and guides global algorithms to reduce false matches, especially in low-textured regions. Finally, we show how image sequence analysis improves stereo accuracy without explicitly performing tracking. Our goal in this work is to obtain an accurate 3D reconstruction. Large-scale 3D reconstruction will benefit heavily from these sub-pixel refinements.Results based on semi-global matching, obtained with the above mentioned algorithmic extensions are shown for the Middlebury stereo ground truth data sets. The presented improvements, called ImproveSubPix, turn out to be one of the top-performing algorithms when evaluating the set on a sub-pixel level while being computationally efficient. Additional results are presented for urban scenes. The four improvements are independent of the underlying type of stereo algorithm.     

Adaptive disparity computation using local and non-local cost aggregations

A new method is proposed to adaptively compute the disparity of stereo matching by choosing one of the alternative disparities from local and non-local disparity maps. The initial two disparity maps can be obtained from state-of-the-art local and non-local stereo algorithms. Then, the more reasonable disparity is selected. We propose two strategies to select the disparity. One is based on the magnitude of the gradient in the left image, which is simple and fast. The other utilizes the fusion move to combine the two proposal labelings (disparity maps) in a theoretically sound manner, which is more accurate. Finally, we propose a texture-based sub-pixel refinement to refine the disparity map. Experimental results using Middlebury datasets demonstrate that the two proposed selection strategies both perform better than individual local or non-local algorithms. Moreover, the proposed method is compatible with many local and non-local algorithms that are widely used in stereo matching.     

置信度传播和区域边缘构建的立体匹配算法

讨论了区域匹配代价和全局置信度传播算法之间的相互作用,提出一种基于全局置信度传播和区域边缘构建的立体匹配算法。首先,在每个像素的固定邻域内利用二阶微分边缘算子搜索并构建一条虚拟的封闭边缘,形成相应的区域自适应窗口;然后使用自适应窗口内的支持像素计算中心像素之间的匹配代价;最后利用具备加速消息更新机制的置信度传播算法获取视差。实验结果表明,基于区域边缘构建的匹配代价可以较好地适用于全局置信度传播优化算法,提出的立体匹配算法可以在Middlebury标准下获得良好的匹配结果。     

基于mean-shift全局立体匹配方法

针对图像全局立体匹配精度高、计算量大的问题,提出基于mean shift图像分割的全局立体匹配方法。首先,通过mean shift算法对图像进行分割,获取图像同质区域数量和区域的标号。在计算匹配代价时,根据像素所属的分割区域,对像素进行筛选,从而提高匹配代价计算速度;其次,在代价聚合前,将mean shift算法获取的同质区域数K值赋值给K-means聚类算法,对像素再次聚类,提高立体匹配精度和速度;最后通过TRW-S置信传播解决能量最小化问题。实验表明,该算法明显提高了匹配的准确性和速度,与单纯的全局匹配算法相比,具有更大的优势。     

基于颜色与边缘融合的非局部立体匹配算法

针对基于最小生成树的非局部算法在无纹理以及边缘区域出现误匹配的问题,提出了一种改进代价计算和颜色与边缘融合的非局部立体匹配算法。首先重新构造了基于颜色-梯度的代价计算函数,以提高无纹理区域像素对代价聚合的贡献率;其次利用颜色与边缘信息进行融合来构造自适应边权函数,并利用该权重构建树结构进行代价聚合;最后通过视差计算和非局部视差优化得到最终的视差图。在Middlebury数据集上进行了测试,实验结果表明,提出的算法在无纹理及边缘区域都取得了良好的匹配效果,有效地改善了视差。     

PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation

PatchMatch (PM) is a simple, yet very powerful and successful method for optimizing continuous labelling problems. The algorithm has two main ingredients: the update of the solution space by sampling and the use of the spatial neighbourhood to propagate samples. We show how these ingredients are related to steps in a specific form of belief propagation (BP) in the continuous space, called max-product particle BP (MP-PBP). However, MP-PBP has thus far been too slow to allow complex state spaces. In the case where all nodes share a common state space and the smoothness prior favours equal values, we show that unifying the two approaches yields a new algorithm, PMBP, which is more accurate than PM and orders of magnitude faster than MP-PBP. To illustrate the benefits of our PMBP method we have built a new stereo matching algorithm with unary terms which are borrowed from the recent PM Stereo work and novel realistic pairwise terms that provide smoothness. We have experimentally verified that our method is an improvement over state-of-the-art techniques at sub-pixel accuracy level.     

Pyramid Multi‐View Stereo with Local Consistency

In this paper, we propose a PatchMatch‐based Multi‐View Stereo (MVS) algorithm which can efficiently estimate geometry for the textureless area. Conventional PatchMatch‐based MVS algorithms estimate depth and normal hypotheses mainly by optimizing photometric consistency metrics between patch in the reference image and its projection on other images. The photometric consistency works well in textured regions but can not discriminate textureless regions, which makes geometry estimation for textureless regions hard work. To address this issue, we introduce the local consistency. Based on the assumption that neighboring pixels with similar colors likely belong to the same surface and share approximate depth‐normal values, local consistency guides the depth and normal estimation with geometry from neighboring pixels with similar colors. To fasten the convergence of pixelwise local consistency across the image, we further introduce a pyramid architecture similar to previous work which can also provide coarse estimation at upper levels. We validate the effectiveness of our method on the ETH3D benchmark and Tanks and Temples benchmark. Results show that our method outperforms the state‐of‐the‐art.     

改进的跨尺度引导滤波立体匹配算法

作为双目三维重建中的关键步骤,双目立体匹配算法完成了从平面视觉到立体视觉的转化.但如何平衡双目立体匹配算法的运行速度和精度仍然是一个棘手的问题.本文针对现有的局部立体匹配算法在弱纹理、深度不连续等特定区域匹配精度低的问题,并同时考虑到算法实时性,提出了一种改进的跨多尺度引导滤波的立体匹配算法.首先融合AD和Census变换两种代价计算方法,然后采用基于跨尺度的引导滤波进行代价聚合,在进行视差计算时通过制定一个判断准则判断图像中每一个像素点的最小聚合代价对应的视差值是否可靠,当判断对应的视差值不可靠时,对像素点构建基于梯度相似性的自适应窗口,并基于自适应窗口修正该像素点对应的视差值.最后通过视差精化得到最终的视差图.在Middlebury测试平台上对标准立体图像对的实验结果表明,与传统基于引导滤波器的立体匹配算法相比具有更高的精度.     

Progressively weighted affine adaptive correlation matching for quasi-dense 3D reconstruction

Correlation matching has been widely accepted as a rudimentary similarity measure to obtain dense 3D reconstruction from a stereo pair. In particular, given a large overlapping area between images with minimal scale differences, the correlation results followed by a geometrically constrained global optimisation delivers adequately dense and accurate reconstruction results. In order to achieve greater reliability, however, correlation matching should correctly account for the geometrical distortion introduced by the different viewing angles of the stereo or multi-view sensors. Conventional adaptive least squares correlation (ALSC) matching addresses this by modifying the shape of a matching window iteratively, assuming that the distortion can be approximated by an affine transform. Nevertheless, since an image captured from different viewing angle is often not practically identical due to scene occlusions, the matching confidence normally deteriorates. Subsequently, it affects the density of the reconstruction results from ALSC-based stereo region growing algorithms. To address this, we propose an advanced ALSC matching method that can progressively update matching weight for each pixel in an aggregating window using a relaxation labelling technique. The experimental results show that the proposed method can improve matching performance, which consequently enhances the quality of stereo reconstruction. Also, the results demonstrate its ability to refine a scale invariant conjugate point pair to an affine and scale invariant point pair.     

垂直交叉双向搜索策略的自适应窗口匹配算法

目的 基于区域的局部匹配算法是一种简单高效的立体匹配方法.针对局部算法中窗口的抉择问题,提出了基于垂直交叉双向搜索的自适应窗口匹配算法.方法 该算法考虑到局部区域内灰度值与视差值的相关性,通过垂直交叉双向搜索策略自适应地调节窗口的形状和大小,并获得相应掩码窗口;再利用积分图像计算掩码窗口的匹配代价,获取视差图;最后采用米字投票和双边滤波器两个步骤对视差图进行修复.结果 针对不同图像采用提出的自适应窗口算法,得到了适用于各种图像结构的匹配窗口,相较于原始垂直交叉算法的匹配精度提高了约30％ (Teddy),同时两步骤视差后处理较好地保持了图像边缘.结论 实验结果表明,该算法改善了规则窗口产生的视差边缘扩充问题,在提高视差精度的同时提高了算法鲁棒性.     

图像立体匹配研究进展

图像的立体匹配一直是立体视觉的研究重点.首先简要介绍了立体匹配方法及其分类,归纳了立体匹配中的各种约束条件和相似性测度函数;然后总结了局部匹配算法和全局匹配算法的特点,并结合对象的三维重建问题重点分析了全局匹配算法中的动态规划算法、图割法和置信度传播算法;最后对立体匹配研究面临的主要问题给出了一些建议.     

基于改进Census变换的多特性立体匹配算法

针对当前立体匹配算法存在的匹配准确率低，难以达到实用的高精度水平的问题，提出了一种基于改良的Census变换与色彩信息和梯度测度相结合的多特性立体匹配算法，实现高精度的双目立体匹配。算法首先在初始代价匹配阶段，将改进的Census变换、色彩和梯度测度赋权求和得出可靠的初始匹配代价；在聚合阶段，采取高效快捷的最小生成树聚合，获得匹配代价矩阵；最后根据胜者为王法则得到初始视差图，并引入左右一致性检测等策略优化视差图，获得高精度的视差图，实验阶段对源自Middlebury上的标准测试图进行测试验证，实验结果表明，经本文算法处理得到的15组测试数据集的视差图在非遮挡区域的平均误匹配率为6.81%，算法实时响应性优良。     

The non-parametric Parzen''s window in stereo vision matching

This paper presents an approach to the local stereovision matching problem using edge segments as features with four attributes. From these attributes we compute a matching probability between pairs of features of the stereo images. A correspondence is said true when such a probability is maximum. We introduce a nonparametric strategy based on Parzen's window (1962) to estimate a probability density function (PDF) which is used to obtain the matching probability. This is the main finding of the paper. A comparative analysis of other recent matching methods is included to show that this finding can be justified theoretically. A generalization of the proposed method is made in order to give guidelines about its use with the similarity constraint and also in different environments where other features and attributes are more suitable.     

一种改进的区域双目立体匹配方法

双目立体匹配是机器视觉中的热点、难点问题。分析了区域立体匹配方法的优缺点,提出了改进的区域立体匹配方法。首先,采集双目视觉图像对对图像对进行校正、去噪等处理,利用颜色特征进行图像分割,再用一种快速有效的块立体匹配算法对图像进行立体匹配。然后,在匹配过程中使用绝对误差累积(SAD)的小窗口来寻找左右两幅图像之间的匹配点。最后,通过滤波得到最终的视差图。实验表明:该方法能够有效地解决重复区域、低纹理区域、纹理相似区域、遮挡区域等带来的误匹配问题,能得到准确清晰的稠密视差图。