Fast Stereo Matching Using Rectangular Subregioning and 3D Maximum-Surface Techniques

This paper presents a fast and reliable stereo matching algorithm which produces a dense disparity map by using fast cross correlation, rectangular subregioning (RSR) and 3D maximum-surface techniques in a coarse-to-fine scheme. Fast correlation is achieved by using the box-filtering technique whose speed is invariant to the size of the correlation window and by segmenting the stereo images into rectangular subimages at different levels of the pyramid. By working with rectangular subimages, not only can the speed of the correlation be further increased, the intermediate memory storage requirement can also be reduced. The disparity map for the stereo images is found in the 3D correlation coefficient volume by obtaining the global 3D maximum-surface rather than simply choosing the position that gives the local maximum correlation coefficient value for each pixel. The 3D maximum-surface is obtained using our new two-stage dynamic programming (TSDP) technique. There are two original contributions in this paper: (1) development of the RSR technique for fast similarity measure; and (2) development of the TSDP technique for efficiently obtaining 3D maximum-surface in a 3D volume. Typical running time of our algorithm implemented in the C language on a 512 × 512 image is in the order of a few seconds on a 500 MHz PC. A variety of synthetic and real images have been tested, and good results have been obtained.     

Panoramic Depth Imaging: Single Standard Camera Approach

In this paper we present a panoramic depth imaging system. The system is mosaic-based which means that we use a single rotating camera and assemble the captured images in a mosaic. Due to a setoff of the camera's optical center from the rotational center of the system we are able to capture the motion parallax effect which enables stereo reconstruction. The camera is rotating on a circular path with a step defined by the angle, equivalent to one pixel column of the captured image. The equation for depth estimation can be easily extracted from the system geometry. To find the corresponding points on a stereo pair of panoramic images the epipolar geometry needs to be determined. It can be shown that the epipolar geometry is very simple if we are doing the reconstruction based on a symmetric pair of stereo panoramic images. We get a symmetric pair of stereo panoramic images when we take symmetric pixel columns on the left and on the right side from the captured image center column. Epipolar lines of the symmetrical pair of panoramic images are image rows. The search space on the epipolar line can be additionaly constrained. The focus of the paper is mainly on the system analysis. Results of the stereo reconstruction procedure and quality evaluation of generated depth images are quite promissing. The system performs well for reconstruction of small indoor spaces. Our finall goal is to develop a system for automatic navigation of a mobile robot in a room.     

Omnistereo: panoramic stereo imaging

An omnistereo panorama consists of a pair of panoramic images, where one panorama is for the left eye and another panorama is for the right eye. The panoramic stereo pair provides a stereo sensation up to a full 360 degrees. Omnistereo panoramas can be constructed by mosaicing images from a single rotating camera. This approach also enables the control of stereo disparity, giving larger baselines for faraway scenes, and a smaller baseline for closer scenes. Capturing panoramic omnistereo images with a rotating camera makes it impossible to capture dynamic scenes at video rates and limits omnistereo imaging to stationary scenes. We present two possibilities for capturing omnistereo panoramas using optics without any moving parts. A special mirror is introduced such that viewing the scene through this mirror creates the same rays as those used with the rotating cameras. The lens used for omnistereo panorama is also introduced, together with the design of the mirror. Omnistereo panoramas can also be rendered by computer graphics methods to represent virtual environments     

自适应窗口的时间规整立体匹配算法

针对立体视觉中图像对应点的误匹配问题,以时间规整算法(DTW)为基础,提出了自适应窗口的立体匹配算法．根据外极线的约束,在自适应窗口内采用灰度相关技术得到长度不相等的两个灰度段作为相容的匹配序列;利用动态规划法及连续性约束寻找一条最佳的匹配路径．根据回溯得到的匹配路径及其坐标值得到高密度视差图．实验结果表明,该算法具有较高的运行效率和良好的匹配效果．     

3D reconstruction for a scanning electron microscope

An algorithm for the reconstruction of the 3D shape of the surface of a micro-object from a stereo pair of images obtained on a raster electron microscope (REM) has been considered. A model of building an image in REM has been presented. The SIFT algorithm was used for determination of the correspondence points. The correspondence points are used to determine the mutual position of the object in a stereo pair by the RANSAC method. A set of points is created in the 3D space, which is later interpolated to reconstruct the 3D surface.     

Fast optical flow using 3D shortest path techniques

Optical flow or image motion estimation is important in the area of computer vision. This paper presents a fast and reliable optical flow algorithm which produces a dense optical flow map by using fast cross correlation and 3D shortest path techniques. Fast correlation is achieved by using the box-filtering technique which is invariant to the size of the correlation window. The motion for each scanline or each column of the input image is obtained from the correlation coefficient volume by finding the best 3D path using dynamic programming techniques rather than simply choosing the position that gives the maximum cross correlation coefficient. Sub-pixel accuracy is achieved by fitting the local correlation coefficients to a quadratic surface. Typical running time for a 256×256 image is in the order of a few seconds on a 85 MHz computer. A variety of synthetic and real images have been tested, and good results have been obtained.     

A multicamera setup for generating stereo panoramic video

Traditional visual communication systems convey only two-dimensional (2-D) fixed field-of-view (FOV) video information. The viewer is presented with a series of flat, nonstereoscopic images, which fail to provide a realistic sense of depth. Furthermore, traditional video is restricted to only a small part of the scene, based on the director's discretion and the user is not allowed to "look around" in an environment. The objective of this work is to address both of these issues and develop new techniques for creating stereo panoramic video sequences. A stereo panoramic video sequence should be able to provide the viewer with stereo vision at any direction (complete 360-degree FOV) at video rates. In this paper, we propose a new technique for creating stereo panoramic video using a multicamera approach, thus creating a high-resolution output. We present a setup that is an extension of a previously known approach, developed for the generation of still stereo panoramas, and demonstrate that it is capable of creating high-resolution stereo panoramic video sequences. We further explore the limitations involved in a practical implementation of the setup, namely the limited number of cameras and the nonzero physical size of real cameras. The relevant tradeoffs are identified and studied.     

基于双目视觉的基准差梯度立体匹配法􀀂

因灰度相关只是从一个侧面来描述左右图像特征点区域之间的灰度相似性,没有考虑特征点之间的空间相关性,因此利用灰度间的相似性作为测量标准进行匹配,不可避免地出现误匹配,提出了在进行双目视觉立体匹配时,采用灰度相关匹配技术,提取复峰特征点作为初始匹配集,采用视差梯度有限约束优化初始匹配集.利用左右图像一对已知对应基准点,通过计算基准点与复峰集各点间的基准差梯度,采用基准差梯度极小化评判标准,确定唯一匹配,并将匹配结果确定为新的基准点以不断更新基准点,直至左(右)图像特征点匹配完毕.通过分别对一幅弱纹理实际自然图像及已知三维坐标标准件的三维重建,证实了所提方法的有效性和可靠性.     

Stereo reconstruction from multiperspective panoramas

A new approach to computing a panoramic (360 degrees) depth map is presented in this paper. Our approach uses a large collection of images taken by a camera whose motion has been constrained to planar concentric circles. We resample regular perspective images to produce a set of multiperspective panoramas and then compute depth maps directly from these resampled panoramas. Our panoramas sample uniformly in three dimensions: rotation angle, inverse radial distance, and vertical elevation. The use of multiperspective panoramas eliminates the limited overlap present in the original input images and, thus, problems as in conventional multibaseline stereo can be avoided. Our approach differs from stereo matching of single-perspective panoramic images taken from different locations, where the epipolar constraints are sine curves. For our multiperspective panoramas, the epipolar geometry, to the first order approximation, consists of horizontal lines. Therefore, any traditional stereo algorithm can be applied to multiperspective panoramas with little modification. In this paper, we describe two reconstruction algorithms. The first is a cylinder sweep algorithm that uses a small number of resampled multiperspective panoramas to obtain dense 3D reconstruction. The second algorithm, in contrast, uses a large number of multiperspective panoramas and takes advantage of the approximate horizontal epipolar geometry inherent in multiperspective panoramas. It comprises a novel and efficient 1D multibaseline matching technique, followed by tensor voting to extract the depth surface. Experiments show that our algorithms are capable of producing comparable high quality depth maps which can be used for applications such as view interpolation.     

Relative DEM production from SPOT data without GCP

Abstract

This paper describes a method to produce a relative digital elevation model (DEM) from SPOT level 1A stereo pair or triplet images without the use of ground control points. The proposed method consists of 1. stereo matching by correlation, 2. calculation of the intersection point of pair or triplet viewing lines and 3. interpolation from random DEM to grid DEM. The emphasis is laid on how to restrict a search area for stereo matching using approximate epipolar constraint.

Relative error (standard deviation of error) of 80 by 50poinis DEM produced from stereo pair images with the convergence angle of 32° was found to be less than 30 m. Relative error from stereo triplet images was less than 26 m. The obtained relative errors suggest the potential of SPOT data to produce a relative DEM in the area where ground control points are not available.     

A setup for panoramic stereo imaging

In this paper, based on the proposed basic implementation of circular projection, a multi-camera setup which can be used for panoramic stereo imaging is presented. Firstly, we get multiple stereo pairs instantly by fixing multiple cameras on the proposed setup, which is an approximate implementation of circular projection. We then prove that the influence on the resultant stereo pairs generated by this approximate setup is slight based on similar triangles and structural similarity (SSIM). With these stereo pairs, the automatic panoramic image stitching algorithm is applied to generate panoramas for both eyes. In addition, with reasonable distance between cameras and the scene, physical dimensions of this setup are given. Finally, experimental results demonstrate that our scheme can be used to generate cylindrical stereo panoramic image with proper scene depth, which can provide viewers with distinguishing stereoscopic experience. The device can be installed on the mobile equipment and captures panoramic stereo image in a moment without stopping. In the future, the proposed setup can be used in the field of network navigation, video monitor and virtual reality.     

Progressively weighted affine adaptive correlation matching for quasi-dense 3D reconstruction

Correlation matching has been widely accepted as a rudimentary similarity measure to obtain dense 3D reconstruction from a stereo pair. In particular, given a large overlapping area between images with minimal scale differences, the correlation results followed by a geometrically constrained global optimisation delivers adequately dense and accurate reconstruction results. In order to achieve greater reliability, however, correlation matching should correctly account for the geometrical distortion introduced by the different viewing angles of the stereo or multi-view sensors. Conventional adaptive least squares correlation (ALSC) matching addresses this by modifying the shape of a matching window iteratively, assuming that the distortion can be approximated by an affine transform. Nevertheless, since an image captured from different viewing angle is often not practically identical due to scene occlusions, the matching confidence normally deteriorates. Subsequently, it affects the density of the reconstruction results from ALSC-based stereo region growing algorithms. To address this, we propose an advanced ALSC matching method that can progressively update matching weight for each pixel in an aggregating window using a relaxation labelling technique. The experimental results show that the proposed method can improve matching performance, which consequently enhances the quality of stereo reconstruction. Also, the results demonstrate its ability to refine a scale invariant conjugate point pair to an affine and scale invariant point pair.     

基于区域增长的立体像对稠密匹配算法

该文提出了一种新的基于区域增长的立体像对稠密匹配算法，该算法适用于多种图像对，包括存在较大视差的未经校准的图像对和其中某些纹理稀疏的区域．首先用新的两层算法匹配图像对中的种子点，匹配关系再根据两种策略由这些种子点向图像的其余部分传播．区域增长过程中以新的加权差值平方和准则作为目标函数，模板窗的大小根据其中包含的纹理数量动态变化，而搜索窗的大小与可信系数成反比．对真实立体像对的稠密匹配结果表明了该算法具有良好的性能．     

一种自动全景拼图的快速算法

基于柱面坐标的自动拼图方法计算简单快速,被广泛采用,但是该方法存在的两个问题：一是基于微分迭代的参数估计方法依赖于初值,容易落入局部极小;二是未考虑摄像机绕米轴的小角度旋转,导致拼接不准,提出一个改进的自动全景拼图鲁棒性快速算法,对于第1个问题,初始支 用重采样的小图像进行相关匹配,可以迅速得到接近最优欠妥 的参数初值,然后再用基于微分迭代的参数估计方法求精,对于第2个问题,在用基于微分迭代的参数估计方法求精时,增加对摄像机绕 光轴的小角度旋转参数估计,该算法简单快速,经实验适用于手持相机拍摄的图像。     

Analysis of depth estimation error for cylindrical stereo imaging

Conventional stereo imaging uses area CCDs for which depth perception error has been analyzed in our past research (Comput. Vision Image understand. 63 (3) (1996) 447). In this work we analyze the depth perception error for a stereo system that uses two rotating linear CCD cameras to create cylindrical stereo images. Theoretical analysis shows certain advantages to cylindrical stereo imaging over conventional methods. Initial experimental results are presented to validate the theoretical results. Unlike normal area CCDs, cylindrical images captured by rotating linear CCD cameras can produce very high-resolution images thereby substantially reducing the error in 3D estimation.     

全景图像质量评价方法最新进展

随着虚拟现实技术的快速发展,全景视频图像已成为一种新的媒体展示形式被投入使用。全景图像的质量评价方法对促进虚拟现实技术具有重要的现实意义。研究了近五年来全景图像质量评价新方法。分析了全景图像客观评价指标,包括基于峰值信噪比的改进方法和基于结构相似性的改进方法。归纳总结了基于深度学习的全景图像质量评价方法以及其特有的缝合失真和投影失真的评价方法。搜集整理了常用的全景图像质量评价公共数据集与专有自建数据集。采用皮尔森相关系数、斯皮尔曼秩相关系数与均方根误差作为评价指标,根据客观评分与人眼主观感受的相关程度对现有全景图像质量评价方法进行了对比分析。对全景图像质量评价方法的发展方向进行展望。     

Characterization of Errors in Compositing Panoramic Images

A panoramic image has a 360° horizontal field of view, and it can provide the viewer the impression of being immersed in the scene. A panorama is created by first taking a sequence of images while rotating the camera about a vertical axis. These images are then projected onto a cylindrical surface before being seamlessly composited. The cross-sectional circumference of the cylindrical panorama is called thecompositing length. This work characterizes the error in compositing panoramic images due to errors in some of the intrinsic parameters. The intrinsic camera parameters that are considered are the camera focal length and the radial distortion coefficient. We show that the error in the compositing length is more sensitive to the error in the camera focal length. Especially important is the discovery that the relative error in compositing length is always smaller than the relative error in the focal length. This means that the error in focal length can be corrected by iteratively using the composited length to compute a new and more correct focal length. Thiscompositing approach to camera calibrationhas the advantages of not requiring both feature detection and separate prior calibration.     

基于形变模型由立体序列图象恢复物体的3D形状

结合立体视觉和形变模型提出了一种新的物体3D形状的恢复方法。采用立体视觉方法导出物体表面的3D坐标;利用光流模型估计物体的3D运动,根据此运动移动形变模型,使其对准物体的表面块;由形变模型将由各幅图象得到的离散的3D点融为一起,得到物体的表面形状。实验结果表明该方法能用于形状复杂的物体恢复。     

Obstacle-Free Pathway Detection by Means of Depth Maps

The detection of surrounding obstacle-free space is an essential task for many intelligent automotive and robotic applications. In this paper we present a method to detect obstacle-free pathways in real-time using depth maps from a pair of stereo images. Depth maps are obtained by processing the disparity between left and right images from a stereo-vision system. The proposed technique assumes that depth of pixels in obstacle-free pathways should increase slightly and linearly from the bottom of the image to the top. The proposed real-time detection checks whether the depth of groups of image columns matches a linear model. Only pixels fulfilling the matching requirements are identified as obstacle-free pathways. Experimental results with real outdoor stereo images show that the method performance is promising.