基于图像分割的立体匹配算法

基于马尔可夫随机场（MRF）的立体匹配算法利用MRF模型来对匹配取值进行连续性约束。然而,MRF模型是产生式模型,图像自身特征难以得到准确描述。提出了一种基于图像分割的立体匹配算法SGC。SGC算法预先对图像进行分割,基于图像分割信息建立立体匹配的MRF模型,从而连续性（平滑）约束可以保留视差图中分割的边缘信息;并针对图像的深度连续性约束,定义了一个反映图像自身特征的新能量函数,应用于图割算法,提高了视差计算精度。实验结果表明,与以往算法相比,SGC算法更准确地反映了图像中深度信息,避免了平滑约束所引入的误差,有效提高了视差计算精度。     

基于图像分割和控制点的立体匹配算法研究

提出了一种基于图像分割和地面控制点（GCP）的立体匹配算法。利用Mean-shift算法将参考图像根据彩色信息快速聚类成不同区域,利用像素点的RGB信息与梯度信息相结合计算初始视差;引入地面控制点（GCP）约束,构造能量函数,利用动态规划方法（DP）计算能量函数最小值;在图像分割区域内采用快速投票方式优化初始视差并获得最终视差图。实验结果表明：该算法能有效处理视差不连续和遮挡区域,也解决了DP算法带来的条纹等问题。     

一种利用动态规划和左右一致性的立体匹配算法

立体匹配是计算机视觉领域研究的一个重要课题,为了得到准确、稠密的视差图,提出了一种利用动态规划和左右一致性的立体匹配算法。该算法首先分别以左、右图像为基元图像,计算各自的视差空间图像,在视差空间图像上利用动态规划,计算得到左视差图和右视差图;然后通过使用左右视差图之间的一致性关系,消除误匹配点,得到较为准确的部分视差图;最后利用视差图的顺序约束关系,给出未匹配视差点的搜索空间计算方法,并利用一种简单有效的方法来计算这些点的视差值。在一些标准立体图像对上所做的实验结果表明,该算法效果良好。     

一种对称的基于分割的遮挡处理体视算法

为了解决体视中的遮挡问题,提出一种对称的基于分割的立体匹配算法.首先基于颜色信息对2幅图像进行初步分割;然后利用视差信息和不连续性约束把初始分割块分裂成更小块,使每个块内的像素具有相同的可见性;最后在分割级上构建一种加强可见约束的对称体视模型,利用信任度传播方法交替地估计2幅图像的视差平面和遮挡.实验结果表明:该算法具有很好的性能,尤其在遮挡区域、无纹理区域和视差不连续区域.     

熵率超像素分割一致性检验视差细化算法

针对传统分割一致性检验视差细化算法处理低纹理图片时优化效果较差的问题,提出一种基于熵率超像素分割的改进方法,使用基于熵率的超像素分割算法代替均值漂移（Mean-shift）分割算法。针对参考图像进行超像素分割处理;将每一个分割块进行统计分析,根据集中趋势值筛选可信值与不可信值;进行视差填充处理获得最终优化后的视差图。选取15组Middlebury数据集中的图像对进行视差图获取并检测。实验结果表明,基于熵率超像素分割的改进方法对于低纹理图片和纹理复杂的图片都有着较好的优化效果,该算法平均误匹配率较传统算法最多降低了5.88个百分点。     

一种基于双重分割的立体匹配算法

立体匹配通过计算同一场景不同视点下图像的匹配像素的视差,恢复场景的深度信息.文中对传统的基于分割的立体匹配算法进行改进,提出了一种基于双重分割的立体匹配算法.首先对参考图像进行颜色欠分割,使每个区域包含足够的信息进行平面拟合;然后对初始匹配视差图进行分割,检测颜色分割中的欠分割区域并进行再分割,进而对再分割后的区域进行平面拟合;最后利用合作算法对不可信区域优化,以提高匹配算法的运行效率.Middlebury标准图像测试集上的实验结果表明,相对于传统分割算法,该算法时间开销更少、匹配精度更高.     

基于相位与区域分割的视差估计算法

提出一种结合小波变换与图像分割的立体匹配算法。该算法利用双树复小波多通道提取立体像对带通相位信息作为匹配基元,求取初始视差场。针对单独通道提出一个相位匹配程度方程式,将视差计算归结为求解该方程的极大值。结合图像分割与视差平面拟合的方法对初始视差场进行修正,从而得到精度较高的密集视差图。实验结果表明,该算法结构简单,能快速有效地产生密集视差场。     

基于双随机场能量函数的区域化图像分割*

为了提高图像分割算法的抗噪性,并充分利用特征场和标号场在能量函数分割模型中的作用,提出基于双随机场能量函数的区域化图像分割方法.首先,利用几何划分将图像域划分为一系列子区域.在此基础上,采用多值高斯分布的负对数定义区域化特征场能量函数,用于描述同质区域内像素颜色的统计分布一致性.扩展传统建模邻域像素标号关系的Potts模型至邻域子区域,定义区域化标号场能量函数,用于表征各子区域标号之间的相关性.联合特征场和标号场,采用KL散度定义异质性能量函数,用于刻画同质区域间颜色统计分布异质性.利用非约束吉布斯表达式将定义的特征场和标号场能量函数转换为描述图像分割的概率分布函数.最后,在最大化上述概率分布函数准则下,设计合适的M-H采样算法,获得最优图像分割.在合成图像、遥感图像和自然纹理图像上进行分割实验,验证文中方法的有效性和准确性.     

基于区域间协同优化的立体匹配算法

提出了一种基于分割区域间协同优化的立体匹配算法. 该算法以图像区域为匹配基元, 利用区域的彩色特征以及相邻区域间应满足的平滑和遮挡关系定义了区域的匹配能量函数, 并引入区域之间的合作竞争机制, 通过协同优化使所定义的匹配能量极小化, 从而得到比较理想的视差结果. 算法首先对参考图像进行分割, 利用相关法得到各分割区域的初始匹配; 然后用平面模型对各区域的视差进行拟合, 得到各区域的视差平面参数; 最后, 基于协同优化的思想, 采用局部优化的方法对各区域的视差平面参数进行迭代优化, 直至得到比较合理的视差图为止. 采用Middlebury test set进行的实验结果表明, 该方法在性能上可以和目前最好的立体匹配算法相媲美, 得到的视差结果接近于真实视差.     

流形正则化约束的图像语义分割

目的 在基于深度学习的图像语义分割方法中,损失函数通常只考虑单个像素点的预测值与真实值之间的交叉熵并对其进行简单求和,而引入图像像素间的上下文信息能够有效提高图像的语义分割的精度,但目前引入上下文信息的方法如注意力机制、条件随机场等算法需要高昂的计算成本和空间成本,不能广泛使用。针对这一问题,提出一种流形正则化约束的图像语义分割算法。方法 以经过数据集ImageNet预训练的残差网络(residual network, ResNet)为基础,采用DeepLabV3作为骨架网络,通过骨架网络获得预测分割图像。进行子图像块的划分,将原始图像和分割图像分为若干大小相同的图像块。通过原始图像和分割图像的子图像块,计算输入数据与预测结果所处流形曲面上的潜在几何约束关系。利用流形约束的结果优化分割网络中的参数。结果 通过加入流形正则化约束,捕获图像中上下文信息,降低了网络前向计算过程中造成的本征结构的损失,提高了算法精度。为验证所提方法的有效性,实验在Cityscapes和PASCAL VOC 2012(pattern analysis, statistical modeling and computational learning visual object classes)两个数据集上进行。在Cityscapes数据集中,精度值为78.0%,相比原始网络提高了0.5%;在PASCAL VOC 2012数据集中,精度值为69.5%,相比原始网络提高了2.1%。同时,在Cityscapes数据集中进行对比实验,验证了算法的有效性,对比实验结果证明提出的算法改善了语义分割的效果。结论 本文提出的语义分割算法在不提高推理网络计算复杂度的前提下,取得了较好的分割精度,具有极大的实用价值。     

基于可变特征基元的视差估计新方法

视差估计是立体视频和多视点视频信号处理中的一个关键问题。选择可变长线段作为特征基元,提出一种基于特征匹配的视差估计新方法。算法结合唯一性和顺序性约束条件来增强视察估计的可靠性与准确性。在详细介绍了算法的基本原理后进行了实验仿真,实验结果表明新算法能获得较为准确、可靠、亚像素精度的密集视差场,其性能优于固定块匹配（Fixed Size Block Matching,FSBM）、可变块匹配（Variable Size Block Matching,VSBM）等传统的视差估计算法。     

基于纹理分析的视差估计算法

针对传统方法难以可靠估计图像中纹理单一像素点视差的问题,提出一种新的基于纹理分析的视差估计算法。与已有方法不同,在以极线约束计算像素点视差时,将极线上纹理单一且近似的像素点合并成直线段,根据连续性和唯一性约束对直线段进行整体匹配,采用直线段的视差得到纹理单一区域的稠密视差图。利用直线段进行整体匹配,提高比较基元包含的信息量,减少扫描范围,从而降低误匹配产生的概率和算法时间复杂度。实验结果表明,该方法能提高纹理单一区域稠密视差图的精度,匹配速度快,具有实用价值。     

Differential Geometric Inference in Surface Stereo

Many traditional two-view stereo algorithms explicitly or implicitly use the frontal parallel plane assumption when exploiting contextual information since, e.g., the smoothness prior biases toward constant disparity (depth) over a neighborhood. This introduces systematic errors to the matching process for slanted or curved surfaces. These errors are nonnegligible for detailed geometric modeling of natural objects such as a human face. We show how to use contextual information geometrically to avoid such errors. A differential geometric study of smooth surfaces allows contextual information to be encoded in Cartan's moving frame model over local quadratic approximations, providing a framework of geometric consistency for both depth and surface normals; the accuracy of our reconstructions argues for the sufficiency of the approximation. In effect, Cartan's model provides the additional constraint necessary to move beyond the frontal parallel plane assumption in stereo reconstruction. It also suggests how geometry can extend surfaces to account for unmatched points due to partial occlusion.     

基于信任度传播的体视算法

针对信任度传播算法计算量大及误匹配率高的问题,提出一种高效的计算稠密视差图的全局优化算法。首先,根据像素匹配代价的特点、视差不连续亮度变化的特征,定义具有适应性的数据约束和平滑约束,并对平滑约束进行分层调节后执行消息的传输。其次,讨论消息传输迭代过程中的冗余计算问题,通过检测消息的收敛性减少运行时间。最后,分析信任度传播算法中的误匹配问题,通过匹配的对称性检测遮挡,并提出重建数据项后,利用贪婪迭代法优化所得视差图,将图像中可靠像素的视差向不可靠像素扩散。实验结果表明,该算法能以较快的速度计算出更理想的视差图。     

Depth Discontinuities by Pixel-to-Pixel Stereo

An algorithm to detect depth discontinuities from a stereo pair of images is presented. The algorithm matches individual pixels in corresponding scanline pairs, while allowing occluded pixels to remain unmatched, then propagates the information between scanlines by means of a fast postprocessor. The algorithm handles large untextured regions, uses a measure of pixel dissimilarity that is insensitive to image sampling, and prunes bad search nodes to increase the speed of dynamic programming. The computation is relatively fast, taking about 600 nanoseconds per pixel per disparity on a personal computer. Approximate disparity maps and precise depth discontinuities (along both horizontal and vertical boundaries) are shown for several stereo image pairs containing textured, untextured, fronto-parallel, and slanted objects in indoor and outdoor scenes.     

Stereo using monocular cues within the tensor voting framework

We address the fundamental problem of matching in two static images. The remaining challenges are related to occlusion and lack of texture. Our approach addresses these difficulties within a perceptual organization framework, considering both binocular and monocular cues. Initially, matching candidates for all pixels are generated by a combination of matching techniques. The matching candidates are then embedded in disparity space, where perceptual organization takes place in 3D neighborhoods and, thus, does not suffer from problems associated with scanline or image neighborhoods. The assumption is that correct matches produce salient, coherent surfaces, while wrong ones do not. Matching candidates that are consistent with the surfaces are kept and grouped into smooth layers. Thus, we achieve surface segmentation based on geometric and not photometric properties. Surface overextensions, which are due to occlusion, can be corrected by removing matches whose projections are not consistent in color with their neighbors of the same surface in both images. Finally, the projections of the refined surfaces on both images are used to obtain disparity hypotheses for unmatched pixels. The final disparities are selected after a second tensor voting stage, during which information is propagated from more reliable pixels to less reliable ones. We present results on widely used benchmark stereo pairs.     

Hierarchical adaptive stereo matching algorithm for obstacle detection with dynamic programming

An adaptive weighted stereo matching algorithm with multilevel and bidirectional dynamic programming based on ground control points （GCPs） is presented. To decrease time complexity without losing matching precision, using a multilevel search scheme, the coarse matching is processed in typical disparity space image, while the fine matching is processed in disparity-offset space image. In the upper level, GCPs are obtained by enhanced volumetric iterative algorithm enforcing the mutual constraint and the threshold constraint. Under the supervision of the highly reliable GCPs, bidirectional dynamic programming framework is employed to solve the inconsistency in the optimization path. In the lower level, to reduce running time, disparity-offset space is proposed to efficiently achieve the dense disparity image. In addition, an adaptive dual support-weight strategy is presented to aggregate matching cost, which considers photometric and geometric information. Further, post-processing algorithm can ameliorate disparity results in areas with depth discontinuities and related by occlusions using dual threshold algorithm, where missing stereo information is substituted from surrounding regions. To demonstrate the effectiveness of the algorithm, we present the two groups of experimental results for four widely used standard stereo data sets, including discussion on performance and comparison with other methods, which show that the algorithm has not only a fast speed, but also significantly improves the efficiency of holistic optimization.     

Disparity Estimation by Pooling Evidence From Energy Neurons

In this paper, we propose an algorithm for disparity estimation from disparity energy neurons that seeks to maintain simplicity and biological plausibility, while also being based upon a formulation that enables us to interpret the model outputs probabilistically. We use the Bayes factor from statistical hypothesis testing to show that, in contradiction to the implicit assumption of many previously proposed biologically plausible models, a larger response from a disparity energy neuron does not imply more evidence for the hypothesis that the input disparity is close to the preferred disparity of the neuron. However, we find that the normalized response can be interpreted as evidence, and that information from different orientation channels can be combined by pooling the normalized responses. Based on this insight, we propose an algorithm for disparity estimation constructed out of biologically plausible operations. Our experimental results on real stereograms show that the algorithm outperforms a previously proposed coarse-to-fine model. In addition, because its outputs can be interpreted probabilistically, the model also enables us to identify occluded pixels or pixels with incorrect disparity estimates.