3D shape recovery from image focus using kernel regression in eigenspace

Shape from focus (SFF) is one of the optical passive methods for three dimensional (3D) shape recovery of an object from its two dimensional (2D) images. The focus measure plays important role in SFF algorithms. Mostly, conventional focus measures are based on gradient, so their performance is restricted under noisy conditions. Moreover, SFF methods also suffer from loss of focus information due to discreteness. This paper introduces a new SFF method based on principal component analysis (PCA) and kernel regression. The focus values are computed through PCA by considering a sequence of small 3D neighborhood for each object point. We apply unsupervised regression through Nadaraya and Watson Estimate (NWE) on depth values to get a refined 3D shape of the object. It reduces the effect of noise within a small surface area as well as approximates the accurate 3D shape by exploiting the depth dependencies in the neighborhood. Performance of the proposed scheme is investigated in the presence of different types of noises and textured areas. Experimental results demonstrate effectiveness of the proposed approach.     

Optimal depth estimation by combining focus measures using genetic programming

Three-dimensional (3D) shape reconstruction is a fundamental problem in machine vision applications. Shape From Focus (SFF) is one of the passive optical methods for 3D shape recovery that uses degree of focus as a cue to estimate 3D shape. In this approach, usually a single focus measure operator is applied to measure the focus quality of each pixel in the image sequence. However, the applicability of a single focus measure is limited to estimate accurately the depth map for diverse type of real objects. To address this problem, we develop Optimal Composite Depth (OCD) function through genetic programming (GP) for accurate depth estimation. The OCD function is constructed by optimally combining the primary information extracted using one/or more focus measures. The genetically developed composite function is then used to compute the optimal depth map of objects. The performance of the developed nonlinear function is investigated using both the synthetic and the real world image sequences. Experimental results demonstrate that the proposed estimator is more useful in computing accurate depth maps as compared to the existing SFF methods. Moreover, it is found that the heterogeneous function is more effective than homogeneous function.     

基于场景对象注意与深度图融合的深度估计

现有单目深度估计算法主要从单幅图像中获取立体信息,存在相邻深度边缘细节模糊、明显的对象缺失问题。提出一种基于场景对象注意机制与加权深度图融合的单目深度估计算法。通过特征矩阵相乘的方式计算特征图任意两个位置之间的相似特征向量,以快速捕获长距离依赖关系,增强用于估计相似深度区域的上下文信息,从而解决自然场景中对象深度信息不完整的问题。基于多尺度特征图融合的优点,设计加权深度图融合模块,为具有不同深度信息的多视觉粒度的深度图赋予不同的权值并进行融合,融合后的深度图包含深度信息和丰富的场景对象信息,有效地解决细节模糊问题。在KITTI数据集上的实验结果表明,该算法对目标图像预估时σ<1.25的准确率为0.879,绝对相对误差、平方相对误差和对数均方根误差分别为0.110、0.765和0.185,预测得到的深度图具有更加完整的场景对象轮廓和精确的深度信息。     

融合双目多维感知特征的立体视频显著性检测

目的 立体视频能提供身临其境的逼真感而越来越受到人们的喜爱,而视觉显著性检测可以自动预测、定位和挖掘重要视觉信息,可以帮助机器对海量多媒体信息进行有效筛选。为了提高立体视频中的显著区域检测性能,提出了一种融合双目多维感知特性的立体视频显著性检测模型。方法 从立体视频的空域、深度以及时域3个不同维度出发进行显著性计算。首先,基于图像的空间特征利用贝叶斯模型计算2D图像显著图;接着,根据双目感知特征获取立体视频图像的深度显著图;然后,利用Lucas-Kanade光流法计算帧间局部区域的运动特征,获取时域显著图;最后,将3种不同维度的显著图采用一种基于全局-区域差异度大小的融合方法进行相互融合,获得最终的立体视频显著区域分布模型。结果 在不同类型的立体视频序列中的实验结果表明,本文模型获得了80%的准确率和72%的召回率,且保持了相对较低的计算复杂度,优于现有的显著性检测模型。结论 本文的显著性检测模型能有效地获取立体视频中的显著区域,可应用于立体视频/图像编码、立体视频/图像质量评价等领域。     

Optimizing image focus for 3D shape recovery through genetic algorithm

Three-dimensional information of objects is advantageous and widely used in multimedia systems and applications. Shape form focus (SFF) is a passive optical technique that reconstructs 3D shape of an object using a sequence of images with varying focus settings. In this paper, we propose an optimization of the focus measure. First, Wiener filter is applied for noise reduction from the image sequence. At the second stage, genetic algorithm (GA) is applied for focus measure optimization. GA finds the maximum focus measurement under a fitness criterion. Finally, 3D shape of the object is determined by maximizing focus measure along the optical direction. The proposed method is tested with image sequences of simulated and real objects. The performance of the proposed technique is analyzed through statistical criteria such as root mean square error (RMSE) and correlation. Comparative analysis shows the effectiveness of the proposed method.     

Fast intra prediction algorithm based on texture analysis for 3D-HEVC encoders

The advances in display technologies and the growing popularity of 3D video systems have attracted more consumers for 3D viewing experiences, and, consequently, the demand for storage and transmission of 3D video content is increasing. To cope with this demand, a 3D video extension of high-efficiency video coding (HEVC) standard is being developed and near the final standardization stage. The upcoming 3D-HEVC standard is expected to provide higher encoding efficiency than its predecessors, supporting multiple views with high resolution, at a cost of considerable increase in computational complexity, which can be an obstacle to its use in real-time applications. This article proposes a novel complexity reduction algorithm developed to optimize the 3D-HEVC intra mode decision targeting real-time video processing for consumer devices with limited computational power, such as 3D camcorders and smartphones equipped with multiple cameras and depth acquisition capabilities. The proposed algorithm analyzes the texture frames and depth maps to estimate the orientation of edges present in the prediction unit data, speeding up the intra prediction process and reducing the 3D-HEVC encoding processing time. Experimental results demonstrate that the proposed algorithm can save 26 % in computational complexity on average with negligible loss of encoding efficiency. This solution contributes to make more feasible the compression of 3D videos targeting real-time applications in power-constrained devices.     

采用空间聚类和平面或连通的彩色图像分割

提出了一种RGB空间聚类和平面区域生长相结合的彩色图像分割方法。聚类时采用重心和空间棋盘距离进行计算,主要为加法、减法运算;区域生长时提出了一种用或连通代替八邻域来判断连通性的新方法,主要为简单的二进制位运算。总体计算复杂度低,从而有效而且快速地实现了彩色图像的目标分割。     

结合暗通道原理和双边滤波的遥感图像增强

目的 在遥感应用如目视解译等任务中,需要提高遥感影像的视觉质量,为此提出一种基于暗通道原理和双边滤波的遥感图像增强算法。方法 由于暗通道模型的softmatting过程计算复杂性高,故使用双边滤波估计大气光幕,进而获得优化透射图,代替He算法中softmatting过程,提高了计算效率。针对将暗通道原理应用于遥感图像增强时所产生的色彩失真现象,提出透射图的改进算法,提高景深图像的取值,同时约束其最大值不大于1。最后,基于景深图像和暗通道原理获得增强后的遥感图像。结果 实验结果表明,本文算法能够有效地增加图像的对比度。与基于双边滤波单尺度Retinex图像增强、四尺度Retinex增强、直方图均衡化及MSRCR增强的结果进行了比较,实验结果验证了算法的有效性。结论 本文模型能够使处理后的遥感图像更符合视觉特性,以便于目视解译与分析。该算法适用于遥感图像的可视化增强。     

Consideration of illumination effects and optimization of window size for accurate calculation of depth map for 3D shape recovery

Obtaining an accurate and precise depth map is the ultimate goal for 3D shape recovery. For depth map estimation, one of the most vital parts is the initial selection of the focus measure and processing the images with the selected focus measure. Although, many focus measures have been proposed in the literature but not much attention has been paid to the factors affecting those focus measures as well as the manner the images are processed with those focus measures. In this paper, for accurate calculation of depth map, we consider the effects of illumination on the depth map as well as the selection of the window size for application of the focus measures. The resulting depth map can further be used in techniques and algorithms leading to recovery of three-dimensional structure of the object which is required in many high-level vision applications. It is shown that the illumination effects can directly result in incorrect estimation of depth map if proper window size is not selected during focus measure computation. Further, it is shown that the images need some kind of pre-processing to enhance the dark regions and shadows in the image. For this purpose, an adaptive enhancement algorithm is proposed for pre-processing. In this paper, we prove that without such pre-processing for image enhancement and without the use of proper window size for the estimation of depth maps, it is not possible to obtain the accurate depth map.     

超分辨率图像复原中的快速L-曲线估计

讨论了从一组低采样降质的视频图像重建超分辨率图像中未知参数的估计问题．使用L-Curve标准来估计正则化参数，然而，L-Curve的计算代价十分昂贵．它需要计算正则化近似解和残差的范式．为此提出一种基于Lanczos算法和Gauss积分理论的算法，在超分辨率图像重建中的参数估计中可以减少L-Curve的计算代价．     

深度时空能量特征表示下的人体行为识别

目的 利用深度图序列进行人体行为识别是机器视觉和人工智能中的一个重要研究领域,现有研究中存在深度图序列冗余信息过多以及生成的特征图中时序信息缺失等问题。针对深度图序列中冗余信息过多的问题,提出一种关键帧算法,该算法提高了人体行为识别算法的运算效率;针对时序信息缺失的问题,提出了一种新的深度图序列特征表示方法,即深度时空能量图（depth spatial-temporal energy map,DSTEM）,该算法突出了人体行为特征的时序性。方法 关键帧算法根据差分图像序列的冗余系数剔除深度图序列的冗余帧,得到足以表述人体行为的关键帧序列。DSTEM算法根据人体外形及运动特点建立能量场,获得人体能量信息,再将能量信息投影到3个正交轴获得DSTEM。结果 在MSR_Action3D数据集上的实验结果表明,关键帧算法减少冗余量,各算法在关键帧算法处理后运算效率提高了20% 30%。对DSTEM提取的方向梯度直方图（histogram of oriented gradient,HOG）特征,不仅在只有正序行为的数据库上识别准确率达到95.54%,而且在同时具有正序和反序行为的数据库上也能保持82.14%的识别准确率。结论 关键帧算法减少了深度图序列中的冗余信息,提高了特征图提取速率;DSTEM不仅保留了经过能量场突出的人体行为的空间信息,而且完整地记录了人体行为的时序信息,在带有时序信息的行为数据上依然保持较高的识别准确率。     

Reliability measure for shape-from-focus

Shape-from-focus (SFF) is a passive technique widely used in image processing for obtaining depth-maps. This technique is attractive since it only requires a single monocular camera with focus control, thus avoiding correspondence problems typically found in stereo, as well as more expensive capturing devices. However, one of its main drawbacks is its poor performance when the change in the focus level is difficult to detect. Most research in SFF has focused on improving the accuracy of the depth estimation. Less attention has been paid to the problem of providing quality measures in order to predict the performance of SFF without prior knowledge of the recovered scene. This paper proposes a reliability measure aimed at assessing the quality of the depth-map obtained using SFF. The proposed reliability measure (the R-measure) analyzes the shape of the focus measure function and estimates the likelihood of obtaining an accurate depth estimation without any previous knowledge of the recovered scene. The proposed R-measure is then applied for determining the image regions where SFF will not perform correctly in order to discard them. Experiments with both synthetic and real scenes are presented.     

基于自适应标记提取的分水岭彩图分割算法

针对分水岭算法过分割问题,提出一种基于自适应提取标记的改进算法。该算法结合极小值深度和汇水盆地尺度信息提取与物体相关的极小值标记,根据梯度图像中极值点的统计信息自适应设定标记提取的阈值。提取到的标记采用形态学极小值标定技术强制作为原始梯度图像的极小值,在修改过的梯度图像上进行分水岭分割。仿真结果表明,该算法能有效解决分水岭算法的过分割问题,具有更强的抗噪性能和边缘定位能力,且计算复杂度较小。     

Realtime Object-aware Monocular Depth Estimation in Onboard Systems

This paper proposes the object depth estimation in real-time, using only a monocular camera in an onboard computer with a low-cost GPU. Our algorithm estimates scene depth from a sparse feature-based visual odometry algorithm and detects/tracks objects’ bounding box by utilizing the existing object detection algorithm in parallel. Both algorithms share their results, i.e., feature, motion, and bounding boxes, to handle static and dynamic objects in the scene. We validate the scene depth accuracy of sparse features with KITTI and its ground-truth depth map made from LiDAR observations quantitatively, and the depth of detected object with the Hyundai driving datasets and satellite maps qualitatively. We compare the depth map of our algorithm with the result of (un-) supervised monocular depth estimation algorithms. The validation shows that our performance is comparable to that of monocular depth estimation algorithms which train depth indirectly (or directly) from stereo image pairs (or depth image), and better than that of algorithms trained with monocular images only, in terms of the error and the accuracy. Also, we confirm that our computational load is much lighter than the learning-based methods, while showing comparable performance.

     

基于主成分分析方向深度梯度直方图的立体视觉深度图特征提取

针对立体视觉深度图特征提取精确度低、复杂度高的问题,提出了一种基于主成分分析方向深度梯度直方图(PCA-HODG)的特征提取算法。首先,对双目立体视觉图像进行视差计算和深度图提取,获取高质量深度图;然后,基于预设大小窗口对所获取的深度图进行边缘检测和梯度计算,获得区域形状直方图特征并量化;同时运用主成分分析(PCA)进行降维;最后,为实现特征获取的精确性和完整性,采用滑动窗口检测方法实现整幅深度图的特征提取,并再次降维。在特征匹配分类实验中,对于Street测试序列帧,该算法比距离样本深度特征(RSDF)算法平均分类准确率提高了1.15%,而对于Tanks、Tunnel、Temple测试序列帧,该算法比测度不变特征(GIF)算法平均分类准确率分别提高了0.69%、1.95%、0.49%;同时与方向深度直方图(HOD)、RSDF、GIF算法相比,平均运行时间分别降低了71.65%、78.05%、80.06%。实验结果表明,该算法不仅能够更精确地检测和提取深度图特征,而且通过降低维数复杂度大大减少了运行时间;同时算法具有较好的鲁棒性。     

一种新的基于散焦图像的深度恢复算法

由散焦图像求深度是计算机视觉中一个非常重要的课题。散焦图像中点的模糊程度随物体的深度而变化,因此可以利用散焦图像估计物体的深度信息,该方法不存在立体视觉和运动视觉中对应点的匹配问题,具有很好的应用前景。研究了一种基于散焦图像空间的深度估计算法：将散焦成像描述成热扩散过程,借助形变函数将两幅散焦图像扩张成一个散焦空间,再估计出形变参数,进而恢复物体的深度信息。最后利用实验验证了算法的有效性。     

一种基于暗通道优先和双边滤波的遥感图像 增强方法

提出了一种基于暗通道原理和双边滤波的遥感图像增强算法。由于暗通道模型的softmatting过程计算复杂性高,本文使用双边滤波代替softmatting以用于透射图的优化,提高了计算效率。针对将暗通道原理应用于遥感图像增强时所产生的色彩失真现象,提出了透射图的改进算法,提高景深图像的取值,同时约束其最大值不大于1。实验结果表明,本文的算法能够有效地增加图像对比度,使图像更符合视觉特性,适用于遥感图像的可视化增强。     

Pose estimation from multiple cameras based on Sylvester’s equation

In this paper, we introduce a method to estimate the object’s pose from multiple cameras. We focus on direct estimation of the 3D object pose from 2D image sequences. Scale-Invariant Feature Transform (SIFT) is used to extract corresponding feature points from adjacent images in the video sequence. We first demonstrate that centralized pose estimation from the collection of corresponding feature points in the 2D images from all cameras can be obtained as a solution to a generalized Sylvester’s equation. We subsequently derive a distributed solution to pose estimation from multiple cameras and show that it is equivalent to the solution of the centralized pose estimation based on Sylvester’s equation. Specifically, we rely on collaboration among the multiple cameras to provide an iterative refinement of the independent solution to pose estimation obtained for each camera based on Sylvester’s equation. The proposed approach to pose estimation from multiple cameras relies on all of the information available from all cameras to obtain an estimate at each camera even when the image features are not visible to some of the cameras. The resulting pose estimation technique is therefore robust to occlusion and sensor errors from specific camera views. Moreover, the proposed approach does not require matching feature points among images from different camera views nor does it demand reconstruction of 3D points. Furthermore, the computational complexity of the proposed solution grows linearly with the number of cameras. Finally, computer simulation experiments demonstrate the accuracy and speed of our approach to pose estimation from multiple cameras.     

Digital shallow depth-of-field adapter for photographs

This paper proposes a novel method to synthesize shallow depth-of-field images from two input photographs taken with different aperture values. The basic approach is to estimate the depth map of a given scene using a DFD (depth-from-defocus) algorithm and blur an input image according to the estimated depth map. The depth information estimated by DFD contains much noise and error, while the estimation is rather accurate along the edges of the image. To overcome the limitation, we propose a depth map filling algorithm using a set of initial depth maps and a segmented image. After depth map filling, the depth map can be fine tuned by applying segment clustering and user interaction. Since our method blurs an input image according to the estimated depth information, it generates physically plausible result images with shallow depth-of-field. In addition to depth-of-field control, the proposed method can be utilized for digital refocusing and detail control in image stylization.     

基于双重循环交替迭代的湍流退化图像复原

湍流退化图像复原是一个富有挑战性的世界性难题,在航天光电成像领域具有广泛的应用前景。为了从少数几帧湍流退化图像中将目标图像有效地恢复出来,提出一种新颖的基于双重循环交替迭代的湍流退化图像复原算法,建立了基于内外循环交替求解目标图像及各帧点扩展函数的迭代关系。在微机上进行了一些复原实验,实验结果表明该算法能用少数几帧图像获取目标图像和点扩展函数的最佳联合估计,证实了算法的可行性和实用价值。