Consideration of illumination effects and optimization of window size for accurate calculation of depth map for 3D shape recovery

Obtaining an accurate and precise depth map is the ultimate goal for 3D shape recovery. For depth map estimation, one of the most vital parts is the initial selection of the focus measure and processing the images with the selected focus measure. Although, many focus measures have been proposed in the literature but not much attention has been paid to the factors affecting those focus measures as well as the manner the images are processed with those focus measures. In this paper, for accurate calculation of depth map, we consider the effects of illumination on the depth map as well as the selection of the window size for application of the focus measures. The resulting depth map can further be used in techniques and algorithms leading to recovery of three-dimensional structure of the object which is required in many high-level vision applications. It is shown that the illumination effects can directly result in incorrect estimation of depth map if proper window size is not selected during focus measure computation. Further, it is shown that the images need some kind of pre-processing to enhance the dark regions and shadows in the image. For this purpose, an adaptive enhancement algorithm is proposed for pre-processing. In this paper, we prove that without such pre-processing for image enhancement and without the use of proper window size for the estimation of depth maps, it is not possible to obtain the accurate depth map.     

Optimal depth estimation by combining focus measures using genetic programming

Three-dimensional (3D) shape reconstruction is a fundamental problem in machine vision applications. Shape From Focus (SFF) is one of the passive optical methods for 3D shape recovery that uses degree of focus as a cue to estimate 3D shape. In this approach, usually a single focus measure operator is applied to measure the focus quality of each pixel in the image sequence. However, the applicability of a single focus measure is limited to estimate accurately the depth map for diverse type of real objects. To address this problem, we develop Optimal Composite Depth (OCD) function through genetic programming (GP) for accurate depth estimation. The OCD function is constructed by optimally combining the primary information extracted using one/or more focus measures. The genetically developed composite function is then used to compute the optimal depth map of objects. The performance of the developed nonlinear function is investigated using both the synthetic and the real world image sequences. Experimental results demonstrate that the proposed estimator is more useful in computing accurate depth maps as compared to the existing SFF methods. Moreover, it is found that the heterogeneous function is more effective than homogeneous function.     

基于加权匹配代价的初始视差估计算法

随着计算机图形学的发展,立体匹配技术已经成为三维场景恢复中一项重要的手段,视差估计是立体匹配中的关键基础技术。为了能够更好进行三维场景恢复,改善视差的工作便迫在眉睫。主要研究了在基于模板的可信传播立体匹配算法中改善视差初始值的算法：通过引入梯度差算子与绝对差和算子加权的匹配代价,运用交叉检验估计及WTA优化初始视差矩阵,进而提高初始视差值的准确性,从而改善最终的视差结果。经实验证明,本方法能够很有效地去除噪点,进而获得较高质量的视差结果。     

遮挡场景的光场图像深度估计方法

光场相机通过单次拍摄可获取立体空间中的4维光场数据,利用光场的多视角特性可从中提取全光场图像的深度信息.然而,现有深度估计方法很少考虑场景中存在遮挡的情况,当场景中有遮挡时,提取深度信息的精度会明显降低.对此,提出一种新的基于多线索融合的光场图像深度提取方法以获取高精度的深度信息.首先分别利用自适应散焦算法和自适应匹配算法提取场景的深度信息;然后用峰值比作为置信以加权融合两种算法获取的深度;最后,用具有结构一致性的交互结构联合滤波器对融合深度图进行滤波,得到高精度深度图.合成数据集和真实数据集的实验结果表明,与其他先进算法相比,所提出的算法获取的深度图精度更高、噪声更少、图像边缘保持效果更好.     

Robust structure and motion recovery for monocular vision systems with noisy measurements

This study proposes a novel complete-order nonlinear structure and motion observer for monocular vision systems subjected to significant measurement noise. In contrast with previous studies that assume noise-free measurements, and require prior knowledge of either the relative motion of the camera or scene geometry, the proposed scheme assumes a single component of linear velocity as known. Under a persistency of excitation condition, the observer then relies on filtered estimates of optical flow to yield exponentially convergent estimates of the unknown motion parameters and feature depth that converge to a uniform, ultimate bound in the presence of measurement noise. The unknown linear and angular velocities are assumed to be generated using an imperfectly known model that incorporates a bounded uncertainty, and optical flow estimation is accomplished using a robust differentiator that is based on the sliding-mode technique. Numerical results are used to validate and demonstrate superior observer performance compared to an alternative leading design in the presence of model uncertainty and measurement noise.     

A real-time fuzzy hardware structure for disparity map computation

Stereo images acquired by a stereo camera setup provide depth estimation of a scene. Numerous machine vision applications deal with retrieval of 3D information. Disparity map recovery from a stereo image pair involves computationally complex algorithms. Previous methods of disparity map computation are mainly restricted to software-based techniques on general-purpose architectures, presenting relatively high execution time. In this paper, a new hardware-implemented real-time disparity map computation module is realized. This enables a hardware-based fuzzy inference system parallel-pipelined design, for the overall module, implemented on a single FPGA device with a typical operating frequency of 138 MHz. This provides accurate disparity map computation at a rate of nearly 440 frames per second, given a stereo image pair with a disparity range of 80 pixels and 640 × 480 pixels spatial resolution. The proposed method allows a fast disparity map computational module to be built, enabling a suitable module for real-time stereo vision applications.     

A temporal-analysis-based pitch estimation system for noisy speech with a comparative study of performance of recent systems

In this paper, a new system of pitch estimation is presented. The system is designed to be robust to challenging noise conditions. This robustness to the presence of noise in the signal is achieved by developing a new representation of the speech signal, based on the operation of damped harmonic oscillators (DHOs), and temporal mode analysis of their output. The resulting representation is shown to possess qualities that are only gradually degraded in the presence of noise. A harmonic grouping based system is used to estimate the pitch frequency. This method is easily extended to simultaneously track the pitch of more than one speaker. In a series of experiments the accuracy and noise robustness of the proposed system was compared with that of a number of prominent pitch estimation and tracking systems. The results show that the proposed system's overall performance is much better than any of the other systems tested, especially in the presence of very large amounts of noise. Furthermore, the proposed system is comparatively inexpensive in terms of processing and memory requirements.     

Optic flow-based vision system for autonomous 3D localization and control of small aerial vehicles

The problem considered in this paper involves the design of a vision-based autopilot for small and micro Unmanned Aerial Vehicles (UAVs). The proposed autopilot is based on an optic flow-based vision system for autonomous localization and scene mapping, and a nonlinear control system for flight control and guidance. This paper focusses on the development of a real-time 3D vision algorithm for estimating optic flow, aircraft self-motion and depth map, using a low-resolution onboard camera and a low-cost Inertial Measurement Unit (IMU). Our implementation is based on 3 Nested Kalman Filters (3NKF) and results in an efficient and robust estimation process. The vision and control algorithms have been implemented on a quadrotor UAV, and demonstrated in real-time flight tests. Experimental results show that the proposed vision-based autopilot enabled a small rotorcraft to achieve fully-autonomous flight using information extracted from optic flow.     

Digital shallow depth-of-field adapter for photographs

This paper proposes a novel method to synthesize shallow depth-of-field images from two input photographs taken with different aperture values. The basic approach is to estimate the depth map of a given scene using a DFD (depth-from-defocus) algorithm and blur an input image according to the estimated depth map. The depth information estimated by DFD contains much noise and error, while the estimation is rather accurate along the edges of the image. To overcome the limitation, we propose a depth map filling algorithm using a set of initial depth maps and a segmented image. After depth map filling, the depth map can be fine tuned by applying segment clustering and user interaction. Since our method blurs an input image according to the estimated depth information, it generates physically plausible result images with shallow depth-of-field. In addition to depth-of-field control, the proposed method can be utilized for digital refocusing and detail control in image stylization.     

基于不确定度评价的Kinect深度图预处理

针对应用在机器人三维(3D)场景感知测量中,Kinect深度图的联合双边滤波(JBF)存在降低原始场景深度信息精确度的制约性问题,提出一种新的预处理算法。首先,通过构建深度图的测量和采样模型,得到深度图的蒙特卡罗不确定度评价模型;其次,依据该模型计算得到深度值估计区间,实现噪声点与非噪声点的判定及滤除;最后,利用估计区间均值完成噪声点的修复。实验结果表明,该算法在噪声滤波的同时保证了非噪声的不变性;非噪声的不变性以及基于估计均值的噪声修复使原始深度梯度具有不变性;与联合彩色深度图的双边滤波相比,预处理结果图物体边缘轮廓清晰不变且其均方误差降低了15.25%~28.79%。因此,该预处理算法达到了提高三维场景深度信息精确度的目的。     

A generalized least absolute deviation method for parameter estimation of autoregressive signals

This paper proposes a generalized least absolute deviation (GLAD) method for parameter estimation of autoregressive (AR) signals under non-Gaussian noise environments. The proposed GLAD method can improve the accuracy of the estimation of the conventional least absolute deviation (LAD) method by minimizing a new cost function with parameter variables and noise error variables. Compared with second- and high-order statistical methods, the proposed GLAD method can obtain robustly an optimal AR parameter estimation without requiring the measurement noise to be Gaussian. Moreover, the proposed GLAD method can be implemented by a cooperative neural network (NN) which is shown to converge globally to the optimal AR parameter estimation within a finite time. Simulation results show that the proposed GLAD method can obtain more accurate estimates than several well-known estimation methods in the presence of different noise distributions.     

基于多孔卷积神经网络的图像深度估计模型

针对在传统机器学习方法下单幅图像深度估计效果差、深度值获取不准确的问题，提出了一种基于多孔卷积神经网络（ACNN）的深度估计模型。首先，利用卷积神经网络（CNN）逐层提取原始图像的特征图；其次，利用多孔卷积结构，将原始图像中的空间信息与提取到的底层图像特征相互融合，得到初始深度图；最后，将初始深度图送入条件随机场（CRF），联合图像的像素空间位置、灰度及其梯度信息对所得深度图进行优化处理，得到最终深度图。在客观数据集上完成了模型可用性验证及误差估计，实验结果表明，该算法获得了更低的误差值和更高的准确率，均方根误差（RMSE）比基于机器学习的算法平均降低了30.86%，而准确率比基于深度学习的算法提高了14.5%，所提算法在误差数据和视觉效果方面都有较大提升，表明该模型能够在图像深度估计中获得更好的效果。     

基于高斯-柯西混合模型的单幅散焦图像深度恢复方法

单幅图像场景深度的获取一直是计算机视觉领域的一个难题。使用高斯分布函数或柯西分布函数近似点扩散函数模型(PSF),再根据图像边缘处散焦模糊量的大小与场景深度之间的关系估算出深度信息,是一种常用的方法。真实世界中图像模糊的缘由千变万化,高斯分布函数以及柯西分布函数并不一定是最佳的近似模型,并且传统的方法对于图像存在阴影、边缘不明显以及深度变化比较细微的区域的深度恢复结果不够准确。为了提取更为精确的深度信息,提出一种利用高斯-柯西混合模型近似PSF的方法;然后对散焦图像进行再模糊处理,得到两幅散焦程度不同的图像;再通过计算两幅散焦图像边缘处梯度的比值估算出图像边缘处的散焦模糊量,从而得到稀疏深度图;最后使用深度扩展法得到场景的全景深度图。通过大量真实图像的测试,说明新方法能够从单幅散焦图像中恢复出完整、可靠的深度信息,并且其结果优于目前常用的两种方法。     

Depth image‐based rendering for multiview generation

Abstract— Techniques for 3‐D display have evolved from stereoscopic 3‐D systems to multiview 3‐D systems, which provide images corresponding to different viewpoints. Currently, new technology is required for application in multiview display systems that use input‐source formats such as 2‐D images to generate virtual‐view images of multiple viewpoints. Due to the changes in viewpoints, occlusion regions of the original image become disoccluded, resulting in problems related to the restoration of output image information that is not contained in the input image. In this paper, a method for generating multiview images through a two‐step process is proposed: (1) depth‐map refinement and (2) disoccluded‐area estimation and restoration. The first step, depth‐map processing, removes depth‐map noise, compensates for mismatches between RGB and depth, and preserves the boundaries and object shapes. The second step, disoccluded‐area estimation and restoration, predicts the disoccluded area by using disparity and restores information about the area by using information about neighboring frames that are most similar to the occlusion area. Finally, multiview rendering generates virtual‐view images by using a directional rendering algorithm with boundary blending.     

深度学习单目深度估计研究进展

单目深度估计是从单幅图像中获取场景深度信息的重要技术,在智能汽车和机器人定位等领域应用广泛,具有重要的研究价值。随着深度学习技术的发展,涌现出许多基于深度学习的单目深度估计研究,单目深度估计性能也取得了很大进展。本文按照单目深度估计模型采用的训练数据的类型,从3个方面综述了近年来基于深度学习的单目深度估计方法：基于单图像训练的模型、基于多图像训练的模型和基于辅助信息优化训练的单目深度估计模型。同时,本文在综述了单目深度估计研究常用数据集和性能指标基础上,对经典的单目深度估计模型进行了性能比较分析。以单幅图像作为训练数据的模型具有网络结构简单的特点,但泛化性能较差。采用多图像训练的深度估计网络有更强的泛化性,但网络的参数量大、网络收敛速度慢、训练耗时长。引入辅助信息的深度估计网络的深度估计精度得到了进一步提升,但辅助信息的引入会造成网络结构复杂、收敛速度慢等问题。单目深度估计研究还存在许多的难题和挑战。利用多图像输入中包含的潜在信息和特定领域的约束信息,来提高单目深度估计的性能,逐渐成为了单目深度估计研究的趋势。     

Heteroscedastic Regression in Computer Vision: Problems with Bilinear Constraint

We present an algorithm to estimate the parameters of a linear model in the presence of heteroscedastic noise, i.e., each data point having a different covariance matrix. The algorithm is motivated by the recovery of bilinear forms, one of the fundamental problems in computer vision which appears whenever the epipolar constraint is imposed, or a conic is fit to noisy data points. We employ the errors-in-variables (EIV) model and show why already at moderate noise levels most available methods fail to provide a satisfactory solution. The improved behavior of the new algorithm is due to two factors: taking into account the heteroscedastic nature of the errors arising from the linearization of the bilinear form, and the use of generalized singular value decomposition (GSVD) in the computations. The performance of the algorithm is compared with several methods proposed in the literature for ellipse fitting and estimation of the fundamental matrix. It is shown that the algorithm achieves the accuracy of nonlinear optimization techniques at much less computational cost.     

新的单目立体视觉的视差图的获得方法

在立体视觉中,视差间接反映物体的深度信息,视差计算是深度计算的基础。常见的视差计算方法研究都是面向双目立体视觉,而双焦单目立体视觉的视差分布不同于双目视差,具有沿极线辐射的特点。针对双焦单目立体视觉的特点,提出了一种单目立体视差的计算方法。对于计算到的初步视差图,把视差点分类为匹配计算点和误匹配点。通过均值偏移向量（Mean Shift）算法,实现了对误匹配点依赖于匹配点和图像分割的视差估计,最终得到致密准确的视差图。实验证明,这种方法可以通过双焦立体图像对高效地获得场景的视差图。     

Blind adaptive carrier phase recovery for QAM systems

This paper provides a study of adaptive phase recovery in quadrature amplitude modulation (QAM) based communication systems. Here, we modify the traditional fourth-power phase recovery algorithm (FP-PRA) to propose three improved algorithms, and analyze their performances in the presence of additive white Gaussian noise, phase noise, frequency-offset, and inter-symbol interference. We demonstrate that it is possible to obtain the optimal values of step-sizes (or loop-gains) in closed-form in the presence of phase noise and/or frequency-offset. In particular, we discuss two methods to improve FP-PRA. The first method involves utilizing the idea of partitioning the QAM constellation into QPSK-like and not-QPSK-like annular regions. The phase synchronizer is allowed to update only when a derotated QAM symbol lies in QPSK-like region; otherwise, the update process is stopped. The second method exploits an evolving idea of QAM-to-QPSK transformation, and uses transformed symbols to estimate phase mismatch. We provide a new interpretation of this transformation method and relate it to the quadrant-wise centroid of the rotated constellation. Furthermore, we discuss the feasibility of this method for both square and cross-QAM, and, identify and verify numerically the existence of false-locks in the case of cross-QAM. To the best of our knowledge, the ideas of constellation partitioning and constellation transformation have not appeared earlier in the context of adaptive phase recovery. We discuss adaptive blind estimation of optimal step-sizes in the presence of phase noise and frequency-offset. Finally, we discuss the modification of the proposed stochastic gradient methods to transform them into batch processing algorithms so as to make them more suitable for higher data rate systems. Numerical experiments indicate that the proposed algorithms can outperform the traditional FP-PRA algorithm for a number of practical QAM sizes under different mismatch conditions, and that our analytical findings are in close agreement with the simulation results.     

SVD域的图像高斯噪声强度估计

精确估计图像或视频中的噪声强度对于后续的信号处理是至关重要的先决条件。通过对含噪图像的奇异值特性的研究,提出一种精确的SVD域的图像噪声强度估计算法。该算法对噪声强度估计提出了创新的解决方法:1)利用奇异值的尾部数据进行噪声强度估计,这样达到尽可能地降低图像信息对噪声估计的干扰;2)对含噪图像加入已知强度的高斯白噪声,以计算噪声估计时需要设置的与图像内容相关的参数,因此该算法可以自适应图像的结构,能够广泛地适应各种类型的图片。实验结果表明SVD域噪声强度估计算法适用于各种图片类型,而且在极大的噪声强度范围内都能够稳定精确地估计噪声强度。     

双尺度顺序填充的深度图像修复

目的 深度图像作为一种重要的视觉感知数据,其质量对于3维视觉系统至关重要。由于传统方法获取的深度图像大多有使用场景的限制,容易受到噪声和环境影响,导致深度图像缺失部分深度信息,使得修复深度图像仍然是一个值得研究并有待解决的问题。对此,本文提出一种用于深度图像修复的双尺度顺序填充框架。方法 首先,提出基于条件熵快速逼近的填充优先级估计算法。其次,采用最大似然估计实现缺失深度值的最优预测。最后,在像素和超像素两个尺度上对修复结果进行整合,准确实现了深度图像孔洞填充。结果 本文方法在主流数据集MB (Middlebury)上与7种方法进行比较,平均峰值信噪比(peak signal-to-noise ratio,PSNR)和平均结构相似性指数(structural similarity index,SSIM)分别为47.955 dB和0.998 2;在手工填充的数据集MB+中,本文方法的PSNR平均值为34.697 dB,SSIM平均值为0.978 5,对比其他算法,本文深度修复效果有较大优势。在时间效率对比实验中,本文方法也表现优异,具有较高的效率。在消融实验部分,对本文提出的填充优先级估计、深度值预测和双尺度改进分别进行评估,验证了本文创新点的有效性。结论 实验结果表明,本文方法在鲁棒性、精确度和效率方面相较于现有方法具有比较明显的优势。