Fast panoramic stereo matching using cylindrical maximum surfaces.

This paper presents a fast panoramic stereo matching algorithm using a cylindrical maximum surface technique. The disparity for a pair of panoramic images is found in a cylindrical shaped correlation coefficient volume by obtaining the maximum surface rather than simply choosing a position that gives the maximum correlation coefficient value. The use of our cylindrical maximum surface technique ensures that the disparities obtained at the left and the right columns of the panoramic stereo images are properly constrained. Typical running time for a pair of 1324 x 120 images is about 0.33 s on a 1.7-GHz PC. A variety of real images have been tested, and good results have been obtained.     

基于单目和双目视觉信息的全参考立体图像质量评价模型

在立体图像质量评价领域,有效地模拟人类视觉系统对图像质量进行评价具有重要意义,考虑到人眼的视觉感知特性,基于单目和双目视觉信息构建一种立体图像质量评价模型MB-FR-SIQA。采用基于结构相似性的立体视差算法得到参考和失真立体图像的视差矩阵,结合Gabor能量响应图、显著性图和视差矩阵生成中间视图,并优化左右眼加权系数计算方法,以提高生成中间视图的准确性。分别利用单目图像和中间视图提取单目和双目视觉信息,计算单目质量分数和双目质量分数,并融合得到立体图像的质量分数,达到评价立体图像质量的目的。实验结果表明,MB-FR-SIQA模型在LIVE-I数据库上具有较高的预测精度,其斯皮尔曼等级相关系数、皮尔森线性相关系数、均方根误差分别为0.945、0.951、5.318,且预测的质量分数符合人类主观评估。     

A real-time fuzzy hardware structure for disparity map computation

Stereo images acquired by a stereo camera setup provide depth estimation of a scene. Numerous machine vision applications deal with retrieval of 3D information. Disparity map recovery from a stereo image pair involves computationally complex algorithms. Previous methods of disparity map computation are mainly restricted to software-based techniques on general-purpose architectures, presenting relatively high execution time. In this paper, a new hardware-implemented real-time disparity map computation module is realized. This enables a hardware-based fuzzy inference system parallel-pipelined design, for the overall module, implemented on a single FPGA device with a typical operating frequency of 138 MHz. This provides accurate disparity map computation at a rate of nearly 440 frames per second, given a stereo image pair with a disparity range of 80 pixels and 640 × 480 pixels spatial resolution. The proposed method allows a fast disparity map computational module to be built, enabling a suitable module for real-time stereo vision applications.     

一种改进的区域双目立体匹配方法

双目立体匹配是机器视觉中的热点、难点问题。分析了区域立体匹配方法的优缺点,提出了改进的区域立体匹配方法。首先,采集双目视觉图像对对图像对进行校正、去噪等处理,利用颜色特征进行图像分割,再用一种快速有效的块立体匹配算法对图像进行立体匹配。然后,在匹配过程中使用绝对误差累积(SAD)的小窗口来寻找左右两幅图像之间的匹配点。最后,通过滤波得到最终的视差图。实验表明:该方法能够有效地解决重复区域、低纹理区域、纹理相似区域、遮挡区域等带来的误匹配问题,能得到准确清晰的稠密视差图。     

FPGA based disparity map computation with vergence control

Depth estimation in a scene using image pairs acquired by a stereo camera setup, is one of the important tasks of stereo vision systems. The disparity between the stereo images allows for 3D information acquisition which is indispensable in many machine vision applications. Practical stereo vision systems involve wide ranges of disparity levels. Considering that disparity map extraction of an image is a computationally demanding task, practical real-time FPGA based algorithms require increased device utilization resource usage, depending on the disparity levels operational range, which leads to significant power consumption. In this paper a new hardware-efficient real-time disparity map computation module is developed. The module constantly estimates the precisely required range of disparity levels upon a given stereo image set, maintaining this range as low as possible by verging the stereo setup cameras axes. This enables a parallel-pipelined design, for the overall module, realized on a single FPGA device of the Altera Stratix IV family. Accurate disparity maps are computed at a rate of more than 320 frames per second, for a stereo image pair of 640 × 480 pixels spatial resolution with a disparity range of 80 pixels. The presented technique provides very good processing speed at the expense of accuracy, with very good scalability in terms of disparity levels. The proposed method enables a suitable module delivering high performance in real-time stereo vision applications, where space and power are significant concerns.     

基于视差平面分割的移动机器人障碍物地图构建方法

作为自主移动机器人地表障碍物探测（GPOD）技术的一部分,提出了一种利用双目摄像机的视差图像 获取信息来构建机器人前方障碍物栅格地图的方法. 该方法融合了3 维立体视觉技术以及2 维图像处理技术,前者 依据视差图的直方图信息对视差图像进行自适应平面分割,把每个平面看作是3 维场景中的实物切片进而提取障碍 物3 维信息,后者通过计算各平面上的障碍物信息曲线来提取障碍物信息,把立体视觉数据从视差图像空间变换到 2 维的障碍物地图空间. 给出了该方法构建障碍物地图的整体过程,试验结果证明了该算法的有效性和精确性.     

Optimizing Disparity for Motion in Depth

Beyond the careful design of stereo acquisition equipment and rendering algorithms, disparity post‐processing has recently received much attention, where one of the key tasks is to compress the originally large disparity range to avoid viewing discomfort. The perception of dynamic stereo content however, relies on reproducing the full disparity‐time volume that a scene point undergoes in motion. This volume can be strongly distorted in manipulation, which is only concerned with changing disparity at one instant in time, even if the temporal coherence of that change is maintained. We propose an optimization to preserve stereo motion of content that was subject to an arbitrary disparity manipulation, based on a perceptual model of temporal disparity changes. Furthermore, we introduce a novel 3D warping technique to create stereo image pairs that conform to this optimized disparity map. The paper concludes with perceptual studies of motion reproduction quality and task performance in a simple game, showing how our optimization can achieve both viewing comfort and faithful stereo motion.     

室外崎岖地形下基于视差图的无人自主车障碍物识别

对于地面无人车和室外非结构化环境, 本文介绍了我们开发的基于立体视觉的障碍物快速识别系统. 为了使无人地面车适应于较复杂的地形, 根据V视差图, 我们提出了一种新的地面主视差图的估计方法. 通过地面主视差图和局部的三维重建, 本文给出了一种由粗到精的障碍物识别与定位方法. 在我们的无人地面车平台上, 我们对这一障碍物自动识别系统进行了相应的实际试验. 其试验结果验证了该系统的有效性.     

面向复杂背景中彩色立体图像的障碍物检测

目标识别技术作为图像理解领域的核心研究内容之一,一直以来是文献中讨论的热门话题.随着计算机科学的发展以及彩色成像设备的普及,越来越多的彩色立体图像在实际生产、生活中得到应用,面向彩色立体图像的目标识别技术逐渐成为吸引学者注意力的新方向.本文基于立体视觉信息,对安装在行进车辆上的两个摄像机在一段时间内获取的立体视频信息进行分析,提取车辆前方可能对行车安全造成威胁的障碍物.在V视差理论的基础上引入两个改进算法首先对原始立体图像进行数据净化和前景/背景分割,使后续算法不受复杂、多变背景的干扰,同时减少数据处理的冗余.然后使用绝对值差(SAD)[9]法直接计算彩色立体图像的浓密视差图,而无需彩色到灰度图像的转换.实验证明,本文提出的改进算法针对实际立体图像可以准确、快速地提取障碍物信息.     

基于FPGA的双目立体视觉系统

立体视觉的目的之一就是为了获得周围场景的3维信息,其关键在于匹配算法。然而即便是使用目前先进的通用处理器,其计算致密视差图所需的时间仍无法满足高速自主导航的需求。为了解决这个问题,提出了一种基于现场可编程门阵列(FPGA)的双目立体视觉系统的设计方案,同时介绍了系统的硬件结构,并在讨论区域匹配的快速算法的基础上,提出了基于FPGA的像素序列和并行窗口算法框架,用以实现零均值像素灰度差平方和(ZSSD)的匹配算法。该算法是先将视频信号经解码芯片生成场景立体图像对,并由FPGA来完成立体图像对的几何校正和ZSSD匹配算法,然后将获得的致密视差图通过PC I总线发送至上位机。实践表明,该算法效果好、速度快,不仅具有较强的鲁棒性,并且硬件系统性能稳定、可靠。此外,该方案还适用于像素灰度差的绝对值和(SAD)和像素灰度差的平方和(SSD)等多种传统区域匹配算法的快速实现和实时处理。     

A 1D approach to correlation-based stereo matching

In stereovision, indices allowing pixels of the left and right images to be matched are basically one-dimensional features of the epipolar lines. In some situations, these features are not significant or cannot be extracted from the single epipolar line. Therefore, many techniques use 2D neighbourhoods to increase the available information. In this paper, we discuss the systematic use of 2D neighbourhoods for stereo matching. We propose an alternative approach to stereo matching using multiple 1D correlation windows, which yields a semi-dense disparity map and an associated confidence map. A particular technique derived from this approach — using fuzzy filtering and a basic decision rule — is compared to about 80 other methods on the Middlebury image datasets [1]. Results are first presented in the framework of the Middlebury website, then on the Receiver Operating Characteristics (ROC) evaluation [2] and, finally, on stereo image pairs of slanted surfaces. We show that a 1D correlation window is sufficient to provide correct matchings in most cases.     

新的单目立体视觉的视差图的获得方法

在立体视觉中,视差间接反映物体的深度信息,视差计算是深度计算的基础。常见的视差计算方法研究都是面向双目立体视觉,而双焦单目立体视觉的视差分布不同于双目视差,具有沿极线辐射的特点。针对双焦单目立体视觉的特点,提出了一种单目立体视差的计算方法。对于计算到的初步视差图,把视差点分类为匹配计算点和误匹配点。通过均值偏移向量（Mean Shift）算法,实现了对误匹配点依赖于匹配点和图像分割的视差估计,最终得到致密准确的视差图。实验证明,这种方法可以通过双焦立体图像对高效地获得场景的视差图。     

基于偏微分方程的稠密视差图获取方法

立体匹配是计算机视觉领域中的一个重要的热门研究课题,为了获得性能更优的稠密视差图,通过把偏微分方程理论运用于机器视觉中,提出了一种新的基于能量函数获取稠密视差图(disparity map)的方法,并首先分析了匹配点对在不同相对位置下对匹配项产生的影响;接着提出了适用于视差图的各向异性的热扩散方程,它不仅继承了Alvarez定义的正则项对初始视差图内部平滑和保持边缘不连续的特性,还通过引入图像的噪声屏蔽函数和二阶方向导数来分别控制对应视差图中不同区域的扩散速度和角点处的扩散方向;最后通过定义的正则项和匹配项来构造新的能量函数,并把基于区域匹配算法得到的视差图作为初始值,再利用最速下降法求解相应的最小能量泛函。实验结果表明,无论从视觉效果上,还是重构深度图的判别上,该新算法都取得了更优的性能。     

Dense Features for Semi-Dense Stereo Correspondence

We present a new feature based algorithm for stereo correspondence. Most of the previous feature based methods match sparse features like edge pixels, producing only sparse disparity maps. Our algorithm detects and matches dense features between the left and right images of a stereo pair, producing a semi-dense disparity map. Our dense feature is defined with respect to both images of a stereo pair, and it is computed during the stereo matching process, not a preprocessing step. In essence, a dense feature is a connected set of pixels in the left image and a corresponding set of pixels in the right image such that the intensity edges on the boundary of these sets are stronger than their matching error (which is the difference in intensities between corresponding boundary pixels). Our algorithm produces accurate semi-dense disparity maps, leaving featureless regions in the scene unmatched. It is robust, requires little parameter tuning, can handle brightnessdifferences between images, nonlinear errors, and is fast (linear complexity).     

基于双目视觉的不规则堆料体积测量研究

针对不规则堆料的体积难以直接测量的问题,提出一种基于双目视觉的不规则堆料体积测量方法。首先,使用双目相机采集堆料的俯视图像,对图像进行预处理;接着通过立体匹配求出堆料图像的视差图,并采用中值滤波和最邻近插值算法对于视差图中的空洞进行填充,同时计算出视差图对应的三维点云坐标;再采用改进的OTSU阈值分割算法将视差图中的堆料区域提取出来,计算出堆料区域的平均高度和面积,最后计算出堆料的体积。实验结果表明,体积测量误差小于5%,可以快速有效地测量出堆料的体积,能够满足工业检测领域的需求,同时为准确快速地测量堆料体积提供了一种新的思路。     

基于标签化匹配区域校正的双目立体匹配算法

双目立体视觉是一种精准有效的测量方法,由此文中提出基于标签化匹配区域校正的双目立体匹配算法.以常规图割算法为基础,通过匹配块的空间几何信息校正匹配区域,获得更高亚像素级精度的匹配视差.首先,根据标签及空间几何信息确定校正变换,充分利用左右两图匹配区域的像素信息.然后,不断更新获选标签,找到使全局能量最小的标签.最后,依据左右一致性准则及均值滤波细化视差图.实验表明,文中算法有利于寻找良好、平滑的分段线性视差图,在处理边缘区域和遮挡区域时精确度较高.     

2D/3D information fusion for building extraction from high-resolution satellite stereo images using kernel graph cuts

This study presents a building extraction strategy from High-resolution satellite stereo images (HRSSI) using 2D and 3D information fusion. In the 2D processing strategy, a visible vegetation index (VVI) is generated. In the 3D processing, a disparity map is generated using semi-global matching (SGM). To remove defects from the disparity map, an object-based approach is proposed by using mean-shift image segmentation and extracting rectangles. By removing terrain effects, a normalized disparity map (nDM) is produced. In the next step, vegetation pixels are removed from nDM and an initial building mask is generated. As nDM does not have precise building boundaries, hybrid segmentation by the kernel graph cut (KGC) is applied to the feature space including the RGB, nDM, and VVI and the results are used in a decision level fusion step. By this methodology, segments that are highly intersected with initial building mask are classified as buildings. Finally, a building boundary refinement (BBR) algorithm is applied to buildings for removing the remaining defects. The proposed method is applied to two pairs of GeoEye-1 stereo images including residential and industrial test areas. Evaluation results show the completeness and correctness level of higher than 90% for the two test areas. Further evaluations show that the quality metric has significantly changed after decision level fusion using the KGC.     

Genetic-Based Stereo Algorithm and Disparity Map Evaluation

In this paper, a new genetic-based stereo algorithm is presented. Our motivation is to improve the accuracy of the disparity map by removing the mismatches caused by both occlusions and false targets. In our approach, the stereo matching problem is considered as an optimization problem. The algorithm first takes advantage of multi-view stereo images to detect occlusions, and therefore, removes mismatches caused by visibility problems. By optimizing the compatibility between corresponding points and the continuity of the disparity map using a genetic algorithm, mismatches caused by false targets are removed. The quadtree structure is used to implement the multi-resolution framework. Since nodes at different level of the quadtree cover different number of pixels, selecting nodes at different levels gives a similar effect as adjusting the window size at different locations of the image. The experimental results show that our approach can generate more accurate disparity maps than two existing approaches. In addition, we introduce a new disparity map evaluation technique, which is developed based on a similar technique employed in the image segmentation area. Comparing with two existing evaluation approaches, the new technique can evaluate the disparity maps generated without additional knowledge of the scene, such as the correct depth information or novel views.     

自适应窗口的时间规整立体匹配算法

针对立体视觉中图像对应点的误匹配问题,以时间规整算法(DTW)为基础,提出了自适应窗口的立体匹配算法．根据外极线的约束,在自适应窗口内采用灰度相关技术得到长度不相等的两个灰度段作为相容的匹配序列;利用动态规划法及连续性约束寻找一条最佳的匹配路径．根据回溯得到的匹配路径及其坐标值得到高密度视差图．实验结果表明,该算法具有较高的运行效率和良好的匹配效果．