Efficient local stereo matching algorithm based on fast gradient domain guided image filtering

Guided image filtering (GIF) based cost aggregation or disparity refinement stereo matching algorithms are studied extensively owing to the edge-aware preserved smoothing property. However, GIF suffers from halo artifacts in sharp edges and shows high computational costs on high-resolution images. The performance of GIF in stereo matching would be limited by the above two defects. To solve these problems, a novel fast gradient domain guided image filtering (F-GDGIF) is proposed. To be specific, halo artifacts are effectively alleviated by incorporating an efficient multi-scale edge-aware weighting into GIF. With this multi-scale weighting, edges can be preserved much better. In addition, high computational costs are cut down by sub-sampling strategy, which decreases the computational complexity from O(N) to O(N/s²) (s: sub-sampling ratio) To verify the effectiveness of the algorithm, F-GDGIF is applied to cost aggregation and disparity refinement in stereo matching algorithms respectively. Experiments on the Middlebury evaluation benchmark demonstrate that F-GDGIF based stereo matching method can generate more accuracy disparity maps with low computational cost compared to other GIF based methods.     

Stacking learning with coalesced cost filtering for accurate stereo matching

Deep learning based stereo matching algorithms have produced impressive disparity estimation for recent years; and the success of them has once overshadowed the conventional ones. In this paper, we intend to reverse this inferiority, by leveraging Stacking Learning with Coalesced Cost Filtering to make the conventional algorithms achieve or even surpass the results of deep learning ones. Four classical and Discriminative Dictionary Learning (DDL) algorithms are adopted as base-models for Stacking. For the former ones, four classical stereo matching algorithms are employed and regarded as ‘Coalesced Cost Filtering Module’; for the latter supervised learning one, we utilize the Discriminative Dictionary Learning (DDL) stereo matching algorithm. Then three categories of features are extracted from the predictions of base-models to train the meta-model. For the meta-model (final classifier) of Stacking, the Random Forest (RF) classifier is selected. In addition, we also employ an advanced one-view disparity refinement strategy to compute the final refined results more efficiently. Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed algorithm outperforms other four most challenging stereo matching algorithms. Besides, the submitted online results even show better results than deep learning ones.     

光栅式双目立体视觉传感器的立体匹配方法

光栅式双目立体视觉传感器的难点之一在于立体匹配问题,为此,提出了一种基于极线约束和空间点最小距离搜索的立体匹配方法.该方法将光栅式双目立体视觉传感器看作两个光栅结构光传感器,分别标定后可测定光条中心点关于某个结构光模型的三维坐标,若两点匹配,则其三维坐标间的距离理论上为零.引入极线约束,在左摄像机成像光条上找一个特征点,在右摄像机所成像中便可计算出一条极线与之对应,在极线与各光条中心的交点中寻找匹配点.该方法在三维空间进行匹配,计算量小,能够实现点与点的唯一匹配.仿真实验表明了该方法的有效性.     

Convolutional neural network based deep conditional random fields for stereo matching

Stereo matching has been studied for many years and is still a challenge problem. The Markov Random Fields (MRF) model and the Conditional Random Fields (CRF) model based methods have achieved good performance recently. Based on these pioneer works, a deep conditional random fields based stereo matching algorithm is proposed in this paper, which draws a connection between the Convolutional Neural Network (CNN) and CRF. The object knowledge is used as a soft constraint, which can effectively improve the depth estimation accuracy. Moreover, we proposed a CNN potential function that learns the potentials of CRF in a CNN framework. The inference of the CRF model is formulated as a Recurrent Neural Network (RNN). A variety of experiments have been conducted on KITTI and Middlebury benchmark. The results show that the proposed algorithm can produce state-of-the-art results and outperform other MRF-based or CRF-based methods.     

Dense-CNN: Dense convolutional neural network for stereo matching using multiscale feature connection

In spite of the fact that convolutional neural network-based stereo matching models have shown good performance in both accuracy and robustness, the issue of image feature loss in regions of texture-less, complex scenes and occlusions remains. In this paper, we present a dense convolutional neural network-based stereo matching method with multiscale feature connection, named Dense-CNN. First, we construct a novel densely connected network with multiscale convolutional layers to extract rich image features, in which the merged multiscale features with context information are utilized to estimate the cost volume for stereo matching. Second, we plan a novel loss-function strategy to learn the network parameters more reasonably, which can develop the performance of the proposed Dense-CNN model on disparity computation. Finally, we run our Dense-CNN model on the Middlebury and KITTI databases to conduct a comprehensive comparison with several state-of-the-art approaches. The experimental results demonstrate that the proposed method achieved superior performance on computational accuracy and robustness of disparity estimation, especially achieving the significant benefit of feature preservation in ill-posed regions.     

A new parallel hardware architecture for high-performance stereo matching calculation

In this paper, we propose a hardware (H/W) architecture to find disparities for stereo matching in real time. After analyzing the arithmetic characteristic of stereo matching, we propose a new calculating method that reuses the intermediate results to minimize the calculation load and memory access. From this, we propose a stereo matching calculation cell and a new H/W architecture. Finally, we propose a new stereo matching processor. The implemented H/W can operate at the clock frequency of 250 MHz at least in the FPGA (field programmable gate array) environment and produce about 120 disparity images per second for HD stereo images.     

Novel shrinking residual convolutional neural network for efficient accurate stereo matching

For stereo matching based on patch comparing using convolutional neural networks (CNNs), the matching cost estimation is highly dependent on the network structure, and the patch comparing is time consuming for traditional CNNs. Accordingly, we propose a stereo matching method based on a novel shrinking residual CNN, which consists of convolutional layers and skip-connection layers, and the size of the fully connected layers decreases progressively. Firstly, a layer-by-layer shrinking size model is adopted for the full-connection layers to greatly increase the running speed. Secondly, the convolutional layer and the residual structure are fused to improve patch comparing. Finally, the Loss function is re-designed to give higher weights to hard-classified examples compared with the standard cross entropy loss. Experimental results on KITTI2012 and KITTI2015 demonstrate that the proposed method can improve the operation speed while maintaining high accuracy.     

Stereo matching cost computation based on nonsubsampled contourlet transform

A new matching cost computation method based on nonsubsampled contourlet transform (NSCT) for stereo image matching is proposed in this paper. Firstly, stereo image is decomposed into high frequency sub-band images at different scales and along different directions by NSCT. Secondly, by utilizing coefficients in high frequency domain and grayscales in RGB color space, the computation model of weighted matching cost between two pixels is designed based on the gestalt laws. Lastly, two types of experiments are carried out with standard stereopairs in the Middlebury benchmark. One of the experiments is to confirm optimum values of NSCT scale and direction parameters, and the other is to compare proposed matching cost with nine known matching costs. Experimental results show that the optimum values of scale and direction parameters are respectively 2 and 3, and the matching accuracy of the proposed matching cost is twice higher than that of traditional NCC cost.     

用于自由曲面视觉测量的立体精匹配方法

根据立体视觉原理,针对自由曲面视觉测量的实际情况,文中将多种方法融合为一体用于立体匹配,提出一种亚像素级的立体精匹配方法。首先利用投影光栅,在曲面上形成变形条纹,采用小波变换检测出像素级的离散边缘点;在此基础上,提出搜索式无监督聚类拟合匹配算法,将边缘点按实际边缘情况分为不同组别,并用三次B样条将离散的边缘点拟合成连续曲线;根据对极约束特性,实现亚像素级的立体精匹配,解决了交向摆放姿态的双目立体视觉系统的匹配问题。     

Stereo matching algorithm based on per pixel difference adjustment,iterative guided filter and graph segmentation

Stereo matching process is a difficult and challenging task due to many uncontrollable factors that affect the results. These factors include the radiometric variations and illumination inconsistence. The absolute differences (AD) algorithms work fast, but they are too sensitive to noise and low textured areas. Therefore, this paper proposes an improved algorithm to overcome these limitations. First, the proposed algorithm utilizes per-pixel difference adjustment for AD and gradient matching to reduce the radiometric distortions. Then, both differences are combined with census transform to reduce the effect of illumination variations. Second, a new approach of iterative guided filter is introduced at cost aggregation to preserve and improve the object boundaries. The undirected graph segmentation is used at the last stage in order to smoothen the low textured areas. The experimental results on the standard indoor and outdoor datasets show that the proposed algorithm produces smooth disparity maps and accurate results.     

一种边缘保持立体匹配方法

针对边缘处前景和背景视差易混淆问题,提出一种边缘保持立体匹配方法.在代价匹配阶段,采用级联Census变换增强代价的抗噪特性.在代价聚集阶段,引入SLIC超像素分割信息进行快速边缘保持代价聚集.在视差后处理阶段,通过导向十字滤波器进一步优化边缘视差.实验结果表明,文中提出的立体匹配方法在Middlebury测试集以及实际场景获得高质量视差效果,并在边缘处的视差较以往非局部立体匹配方法有所提升.实验还发现在点云上采样时,引入本文所提的导向十字滤波器,可以解决点云在边缘处的过渡.     

Stereo matching based on multi-direction polynomial model

This paper presents a segmentation based stereo matching algorithm. For the purposes of both preserving the shape of object surfaces and being robust to under segmentations, we introduce a new scene formulation where the reference image is divided into overlapping lines. The disparity value and the index of pixels on lines are modeled by polynomial functions. Polynomial functions are propagated among lines to obtain smooth surfaces via solving energy minimizing problems. Finally, the disparity of pixels is estimated from the disparity fields provided by lines. Because lines in multiple directions implicitly segment different objects in an under segmentation region, our method is robust for under segmented regions where it is usually difficult for conventional region based methods to produce satisfactory results. Experimental results demonstrate that the proposed method has an outstanding performance compared with the current state-of-the-art methods. The scene representation method in this work is also a powerful approach to surface based scene representations.     

二次加权引导滤波融合双色彩空间模型的立体匹配

立体匹配一直以来都是双目视觉领域中研究的重点 和难点。针对现有立体匹配算法边 缘保持性差、匹配精度低等问题,提出了一种二次加权引导滤波融合双色彩空间模型的立体 匹配算法(Secondary Weighted Guided Filtering fusion double color model,SWGF)。首 先在代价计算阶段提出了一种双色彩空间模型,从两个颜色空间进行像素颜色匹配代价计算 ,增强像素在低纹理区域的特征;然后在代价聚合阶段基于HSV颜色空间利用不同窗口中像 素纹理不同加入一个边缘保持项,从而使正则化参数进行自适应调整。在一次引导滤波之后 ,我们使用Census变换得到的汉明距离和初始视差完成一次代价更新,再次对其进行代价聚 合,随后计算视差并对视差进行左右一致性检测、空洞填充和加权中值滤波等优化,最后获 得视差图。本文算法在Middlebury测试平台上测试结果表明SWGF算法误匹配率仅为 4.61%,可以大幅提升立体匹配的精度,同时增强其边缘保持性。     

双目视觉中立体匹配算法的研究与比较

在基于双目立体视觉三维重建的研究中,立体匹配是其中最重要的部分,它的准确性影响着最后的重建结果.本文主要讲述了常用的立体匹配算法,并详细介绍了两种算法分类中的代表性算法的实现步骤,即局域算法中的基于特征点匹配的算法和全局算法中的基于图割法的匹配算法,并对算法从运算速度和误配率两方面进行了比较,总结了两种算法的优缺点,比...     

Stereo matching using cost volume watershed and region merging

Segment based disparity estimation methods have been proposed in many different ways. Most of these studies are built upon the hypothesis that no large disparity jump exists within a segment. When this hypothesis does not hold, it is difficult for these methods to estimate disparities correctly. Therefore, these methods work well only when the images are initially over segmented but do not work well for under segmented cases. To solve this problem, we present a new segment based stereo matching method which consists of two algorithms: a cost volume watershed algorithm (CVW) and a region merging (RM) algorithm. For incorrectly under segmented regions where pixels on different objects are grouped into one segment, the CVW algorithm regroups the pixels on different objects into different segments and provides disparity estimation to the pixels in different segments accordingly. For unreliable and occluded regions, we merge them into neighboring reliable segments for robust disparity estimation. The comparison between our method and the current state-of-the-art methods shows that our method is very competitive and is robust particularly when the images are initially under segmented.     

Reliable fusion of ToF and stereo data based on joint depth filter

To obtain reliable depth images with high resolution, a novel method is proposed in this study that fuses data acquired from time-of-flight (ToF) and stereo cameras, through which the advantages of both active and passive sensing are utilised. Based on the classic error model of the ToF, gradient information is introduced to establish the likelihood distribution for all disparity candidates. The stereo likelihood is estimated in parallel based on a 3D adaptive support-weight approach. The two independent likelihoods are unified using a maximum likelihood estimation, a process also referred to as a joint depth filter herein. Conventional post-processing methods such as a mutual consistency check are also used after applying a joint depth filter. We also propose a novel hole-filling method based on the seed-growing algorithm to retrieve missing disparities. Experiment results show that the proposed fusion method can produce reliable high-resolution depth maps and outperforms other compared methods.     

User assisted disparity remapping for stereo images

This study concentrates on user assisted disparity remapping for stereo image footage, i.e. the disparity of an object of interest is altered while leaving the remaining scene unattended. This application is useful in the sense that it provides a method for emphasizing/de-emphasizing an object on the scene by adjusting its depth with respect to the camera. The proposed technique can also be used as a post-processing step for retargeting stereoscopic footage on different display sizes and resolutions. The proposed technique involves an MRF-based energy minimization step for interactive stereo image segmentation, for which user assistance on only one of the stereo pairs is required for determining the location of stereo object pair. A key contribution of the proposed study is elimination of dense disparity estimation step from the pipeline. This step is realized through a sparse feature matching technique between the stereo pairs. Moreover, by the help of the proposed technique, novel disparity adjusted views are synthesized using the produced stereo object segments and background information for the images. Qualitative and quantitative evaluation of the generated segments and the disparity adjusted images prove the functionality and superiority of the proposed technique.     

虚拟计算环境下面向应用的基于信任的资源匹配模型

为了解决虚拟计算环境中的资源合理调度问题,提出了一种基于信任的资源匹配模型--“资源滑动窗口”模型。首先对资源的静态属性进行分类,然后依据基于时间窗的贝叶斯信任模型对资源节点进行评价,同时考虑资源的负载,动态划分其实时性能。最后综合评估静态和动态属性,确定调度资源分配。该模型为不同任务和属性的资源调度策略奠定了基础,实现了“合适的资源服务于合适的任务”的目的。仿真实验表明所提的模型相对传统的调度算法,具有更高的系统任务执行成功率和资源利用率。     

基于Census变换和多扫描线优化的半全局立体匹配算法

为了解决局部匹配算法误匹配率高的问题,提出 一种基于AD-Census变换和多扫描线优化的半全局匹配算 法。首先,通过绝对差AD算法与Census变换相结合作为相似性度量函数计算初始匹配代价, 并构建动态交叉域聚合 匹配代价;然后在代价聚合计算阶段,将一维动态规划的代价聚合推广到多扫描线优化,利 用上下左右四个方向逐 次扫描进行匹配代价聚合的计算,并引入正则化约束以确保匹配代价聚合的一致性,大大减 少初始代价中的匹配异 常点;最后,运用简单高效的胜者为王策略选出像素点在代价聚合最小时对应的视差,并在 视差细化阶段,采用左 右一致性检测和抛物线拟合方法进行后续处理以提高立体匹配的正确率。实验结果证明,该 算法可获得高匹配率的视差图并且耗时较少。