Shape-indifferent stereo disparity based on disparity gradient estimation

A new algorithm capable of estimating disparity gradients to produce accurate dense disparities is proposed. Such a disparity gradient plays a critical role in acquiring accurate disparities for scenes with many different object shapes. The target is a road traffic scene because it contains various objects, including the road surface, vehicles, pedestrians, sidewalks, and walls. In this paper, we adopt several methods, such as initial matching cost computation, scanline optimization, left/right consistency check, and cost aggregation. However, disparity accuracy is slightly improved by the simple organization of such methods. Disparity quality decisively relies on the application of disparity gradients. Accordingly, in the proposed algorithm, cost aggregation is performed along the direction of the estimated disparity gradient in a disparity space image. This approach improves disparity quality significantly. However, this cost aggregation is time consuming. To reduce the time required, we designed a new 2D integral cost technique. The robustness of the proposed algorithm is demonstrated through the disparity maps obtained from standard images on the Web, indoor images, and outdoor images of various road traffic scenes.     

Accurate real-time neural disparity MAP estimation with FPGA

We propose in this paper a new method for real-time dense disparity map computing using a stereo pair of rectified images. Based on the neural network and Disparity Space Image (DSI) data structure, the disparity map computing consists of two main steps: initial disparity map estimation by combining the neuronal network and the DSI structure, and its refinement. Four improvements are introduced so that an accurate and fast result will be reached. The first one concerns the proposition of a new strategy in order to optimize the computation time of the initial disparity map. In the second one, a specific treatment is proposed in order to obtain more accurate disparity for the neighboring pixels to boundaries. The third one, it concerns the pixel similarity measure for matching score computation and it consists of using in addition to the traditional pixel intensities, the magnitude and orientation of the gradients providing more accuracy. Finally, the processing time of the method has been decreased consequently to our implementation of some critical steps on FPGAs. Experimental results on real datasets are conducted and a comparative evaluation of the obtained results relative to the state-of-art methods is presented.     

YCbCr空间分治的双分支低照度图像增强网络

目的 现有的低照度图像增强算法通常在RGB颜色空间采用先增强后去噪的方式提升对比度并抑制噪声,由于亮度失真和噪声在RGB颜色空间存在复杂的耦合关系,往往导致增强结果不理想。先增强后去噪的方式也放大了原本隐藏在黑暗中的噪声,使去噪变得困难。为有效处理亮度失真并抑制噪声,提出了一个基于YCbCr颜色空间的双分支低照度图像增强网络,以获得正常亮度和具有低噪声水平的增强图像。方法 由于YCbCr颜色空间可以分离亮度信息与色度信息,实现亮度失真和噪声的解耦,首先将低照度图像由RGB颜色空间变换至YCbCr颜色空间,然后设计一个双分支增强网络,该网络包含亮度增强模块和噪声去除模块,分别对亮度信息和色度信息进行对比度增强和噪声去除,最后使用亮度监督模块和色度监督模块强化亮度增强模块和噪声去除模块的功能,确保有效地提升对比度和去除噪声。结果 在多个公开可用的低照度图像增强数据集上测试本文方法的有效性,对比经典的低照度图像增强算法,本文方法生成的增强图像细节更加丰富、颜色更加真实,并且含有更少噪声,在LOL（low-light dataset）数据集上,相比经典的KinD++（kindling the darkness）,峰值信噪比（peak signal-to-noise ratio,PSNR）提高了3.09 dB,相比URetinex（Retinex-based deep unfolding network）,PSNR提高了2.74 dB。结论 本文提出的空间解耦方法能够有效地分离亮度失真与噪声,设计的双分支网络分别用于增强亮度和去除噪声,能够有效地解决低照度图像中亮度与噪声的复杂耦合问题,获取低噪声水平的亮度增强图像。     

Sampling the disparity space image

A central issue in stereo algorithm design is the choice of matching cost. Many algorithms simply use squared or absolute intensity differences based on integer disparity steps. In this paper, we address potential problems with such approaches. We begin with a careful analysis of the properties of the continuous disparity space image (DSI) and propose several new matching cost variants based on symmetrically matching interpolated image signals. Using stereo images with ground truth, we empirically evaluate the performance of the different cost variants and show that proper sampling can yield improved matching performance.     

基于视差矢量特点的视差估计快速算法

视差估计是立体视频压缩中的一项关键技术。针对极线校正算法存在的不足,提出了一种基于视差矢量特点的快速视差估计算法。算法分析了平行摄像机与会聚摄像机系统中视差矢量特点,并根据它们的特点通过三步搜索来确定最佳匹配块。分别在分辨率640×480与1280×720两种素材中进行了实验,实验结果表明,与JMVC中的TZ搜索算法相比,所提算法能够在保证图像质量与压缩效率基本不变的前提下,有效缩短编码时间,提高编码效率。由于所提算法不再进行极线校正,所以不会产生极线校正算法存在的问题。     

Parallax correction via disparity estimation in a multi-aperture camera

In this paper, an image fusion algorithm is proposed for a multi-aperture camera. Such camera is a feasible alternative to traditional Bayer filter camera in terms of image quality, camera size and camera features. The camera consists of several camera units, each having dedicated optics and color filter. The main challenge of a multi-aperture camera arises from the fact that each camera unit has a slightly different viewpoint. Our image fusion algorithm corrects the parallax error between the sub-images using a disparity map, which is estimated from the single-spectral images. We improve the disparity estimation by combining matching costs over multiple views using trifocal tensors. Images are matched using two alternative matching costs, mutual information and Census transform. We also compare two different disparity estimation methods, graph cuts and semi-global matching. The results show that the overall quality of the fused images is near the reference images.     

Real-time joint disparity and disparity flow estimation on programmable graphics hardware

Disparity flow depicts the 3D motion of a scene in the disparity space of a given view and can be considered as view-dependent scene flow. A novel algorithm is presented to compute disparity maps and disparity flow maps in an integrated process. Consequently, the disparity flow maps obtained helps to enforce the temporal consistency between disparity maps of adjacent frames. The disparity maps found also provides the spatial correspondence information that can be used to cross-validate disparity flow maps of different views. Two different optimization approaches are integrated in the presented algorithm for searching optimal disparity values and disparity flows. The local winner-take-all approach runs faster, whereas the global dynamic programming based approach produces better results. All major computations are performed in the image space of the given view, leading to an efficient implementation on programmable graphics hardware. Experimental results on captured stereo sequences demonstrate the algorithm’s capability of estimating both 3D depth and 3D motion in real-time. Quantitative performance evaluation using synthetic data with ground truth is also provided.     

Dense estimation of fluid flows

In this paper, we address the problem of estimating and analyzing the motion of fluids in image sequences. Due to the great deal of spatial and temporal distortions that intensity patterns exhibit in images of fluids, the standard techniques from computer vision, originally designed for quasi-rigid motions with stable salient features, are not well adapted in this context. We thus investigate a dedicated minimization-based motion estimator. The cost function to be minimized includes a novel data term relying on an integrated version of the continuity equation of fluid mechanics, which is compatible with large displacements. This term is associated with an original second-order div-curl regularization which prevents the washing out of the salient vorticity and divergence structures. The performance of the resulting fluid flow estimator is demonstrated on meteorological satellite images. In addition, we show how the sequences of dense motion fields we estimate can be reliably used to reconstruct trajectories and to extract the regions of high vorticity and divergence     

基于视差空间采样的区域增长稠密立体匹配算法

通过对现有区域增长算法的研究,提出了改进的区域增长算法,该算法克服了原有算法依赖种子点精度,在平滑区域停止传播,在纹理区域易产生误匹配的缺陷,得到了良好的视差图质量.首先通过对提取的特征点进行匹配,实现对视差空间的采样.然后在视差空间中建立种子点新的传播方式.实验结果证明,该算法能遍历整个视差空间,在整个传播过程中能自动从匹配错误中恢复,在平滑区域和重复纹理区域也能得到良好的匹配效果.     

Performance of phase-based algorithms for disparity estimation

Stereoscopic depth analysis by means of disparity estimation has been a classical topic of computer vision, from the biological models of stereopsis [1] to the widely used techniques based on correlation or sum of squared differences [2]. Most of the recent work on this topic has been devoted to the phase-based techniques, developed because of their superior performance and better theoretical grounding [3, 4]. In this article we characterize the performance of phase-based disparity estimators, giving quantitative measures of their precision and their limits, and how changes in contrast, imbalance, and noise in the two stereo images modify the attainable accuracy. We find that the theoretical range of measurable disparities, one period of the modulation of the filter, is not attainable: the actual range is approx. two-thirds of this value. We show that the phase-based disparity estimators are robust to changes in contrast of 100% or more and well tolerate imbalances of luminosity of 400% between the images composing the stereo pair. Clearing the Gabor filter of its DC component has been often advocated as a means to improve the accuracy of the results. We give a quantitative measure of this improvement and show that using a DC-free Gabor filter leads to disparity estimators nearly insensitive to contrast and imbalance. Our tests show that the most critical source of error is noise: the error increases linearly with the increase in noise level. We conclude by studying the influence of the spectra and the luminosity of the input images on the error surface, for both artificial and natural images, showing that the spectral structure of the images has little influence on the results, changing only the form of the error surface near the limits of the detectable disparity range. In conclusion, this study allows estimation of the expected accuracy of custom-designed phase-based stereo analyzers for a combination of the most common error sources.     

A generalized depth estimation algorithm with a single image

A depth estimation algorithm proposed by A.P. Pentland (1987) is generalized. In the proposed algorithm, the raw image data in the vicinity of the edge is used to estimate the depth from defocus. Since no differentiation operation on the image data is required before the optimization process, the method is less sensitive to the noise disturbance of measurements. Furthermore, the edge orientation that was critical in Pentland's approach will not be required in the case. This algorithm is then applied to synthetic images containing various amounts of noise to test its performance. Experimental results indicate that the depth estimation errors are kept within 5% of true values on the average when it is applied to real images     

A new image motion estimation algorithm based on the EM technique

This paper focuses on the presentation and implementation of a new iterative algorithm for image motion coefficient estimation from noisy measurements based on the expectation-maximization (EM) technique. We also compare this algorithm with two other robust iterative algorithms. We represent the motion field by a (unitary) series expansion to obtain the motion coefficients, and show this characterization to have several virtues. First, an inherent property of motion, referred to as smoothness, is imposed. Second, the nonuniform motion estimation is reduced to the estimation of a few coefficients using the low-pass property of the motion. Finally, the motion estimation can be accomplished without the need for a motion model; in the events for which the motion model is completely unknown, the DCT representation is shown to be very effective in describing the true motion     

Land cover estimation with ALOS satellite image using a neural-network

On May 12, 2008, a large earthquake occurred in Sichuan, China. We analyzed the damage caused by this disaster using satellite images from ALOS, a Japanese satellite. The land cover classification is operated by images captured on AVNIR-2. Frequently, the AVNIR-2 images cannot be monitored because of the cloud cover and solar irradiation. The area near the center of the earthquake area is covered with clouds. The goal of this article is to classify the land cover using PALSAR images. PALSAR can observe over a 350-km-wide area independently of the weather. The PALSAR is a single-band sensor, and the inputs consist of many pixels by using the nearest pixel values, and the supervisor signal is the classes estimated by AVNIR-2.     

Defocus map estimation from a single image

In this paper, we address the challenging problem of recovering the defocus map from a single image. We present a simple yet effective approach to estimate the amount of spatially varying defocus blur at edge locations. The input defocused image is re-blurred using a Gaussian kernel and the defocus blur amount can be obtained from the ratio between the gradients of input and re-blurred images. By propagating the blur amount at edge locations to the entire image, a full defocus map can be obtained. Experimental results on synthetic and real images demonstrate the effectiveness of our method in providing a reliable estimation of the defocus map.     

局部双目视差回归的目标距离估计

目的 双目视觉是目标距离估计问题的一个很好的解决方案。现有的双目目标距离估计方法存在估计精度较低或数据准备较繁琐的问题,为此需要一个可以兼顾精度和数据准备便利性的双目目标距离估计算法。方法 提出一个基于R-CNN（region convolutional neural network）结构的网络,该网络可以实现同时进行目标检测与目标距离估计。双目图像输入网络后,通过主干网络提取特征,通过双目候选框提取网络以同时得到左右图像中相同目标的包围框,将成对的目标框内的局部特征输入目标视差估计分支以估计目标的距离。为了同时得到左右图像中相同目标的包围框,使用双目候选框提取网络代替原有的候选框提取网络,并提出了双目包围框分支以同时进行双目包围框的回归;为了提升视差估计的精度,借鉴双目视差图估计网络的结构,提出了一个基于组相关和3维卷积的视差估计分支。结果 在KITTI（Karlsruhe Institute of Technology and Toyota Technological Institute）数据集上进行验证实验,与同类算法比较,本文算法平均相对误差值约为3.2%,远小于基于双目视差图估计算法（11.3%）,与基于3维目标检测的算法接近（约为3.9%）。另外,提出的视差估计分支改进对精度有明显的提升效果,平均相对误差值从5.1%下降到3.2%。通过在另外采集并标注的行人监控数据集上进行类似实验,实验结果平均相对误差值约为4.6%,表明本文方法可以有效应用于监控场景。结论 提出的双目目标距离估计网络结合了目标检测与双目视差估计的优势,具有较高的精度。该网络可以有效运用于车载相机及监控场景,并有希望运用于其他安装有双目相机的场景。     

Fast motion and disparity estimation for multiview video coding

Multiview video involves a huge amount of data, and as such, efficiently encoding each view is a critical issue for its wider application. In this paper, a fast motion and disparity estimation algorithm is proposed, utilizing the close correlation between temporal and interview reference frames. First, a reliable predictor is found according to the correlation of motion and disparity vectors. Second, an iterative search process is carried out to find the optimal motion and disparity vectors. The proposed algorithm makes use of the prediction vector obtained in the previous motion estimation for the next disparity estimation and achieves both optimal motion and disparity vectors jointly. Experimental results demonstrate that the proposed algorithm can successfully save an average of 86% of computational time with a negligible quality drop when compared to the joint multiview video model (JMVM) full search algorithm. Furthermore, in comparison with the conventional simulcast coding, the proposed algorithm enhances the video quality and also greatly increases coding speed.     

Fast motion and disparity estimation for multiview video coding

Multiview video involves a huge amount of data, and as such, efficiently encoding each view is a critical issue for its wider application. In this paper, a fast motion and disparity estimation algorithm is proposed, utilizing the close correlation between temporal and inter-view reference frames. First, a reliable predictor is found according to the correlation of motion and disparity vectors. Second, an iterative search process is carried out to find the optimal motion and disparity vectors. The proposed algorithm makes use of the prediction vector obtained in the previous motion estimation for the next disparity estimation and achieves both optimal motion and disparity vectors jointly. Experimental results demonstrate that the proposed algorithm can successfully save an average of 86% of computational time with a negligible quality drop when compared to the joint multiview video model (JMVM) full search algorithm. Furthermore, in comparison with the conventional simulcast coding, the proposed algorithm enhances the video quality and also greatly increases coding speed.