A novel iterative shape from focus algorithm based on combinatorial optimization

Shape from focus (SFF) is a technique to estimate the depth and 3D shape of an object from a sequence of images obtained at different focus settings. In this paper, the SFF is presented as a combinatorial optimization problem. The proposed algorithm tries to find the combination of pixel frames which produces maximum focus measure computed over pixels lying on those frames. To reduce the high computational complexity, a local search method is proposed. After the estimate of the initial depth map solution of an object, the neighborhood is defined, and an intermediate image volume is generated from the neighborhood. The updated depth map solution is found from the intermediate image volume. This update process of the depth map solution continues until the amount of improvement is negligible. The results of the proposed SFF algorithm have shown significant improvements in both the accuracy of the depth map estimation and the computational complexity, with respect to the existing SFF methods.     

Optimal depth estimation by combining focus measures using genetic programming

Three-dimensional (3D) shape reconstruction is a fundamental problem in machine vision applications. Shape From Focus (SFF) is one of the passive optical methods for 3D shape recovery that uses degree of focus as a cue to estimate 3D shape. In this approach, usually a single focus measure operator is applied to measure the focus quality of each pixel in the image sequence. However, the applicability of a single focus measure is limited to estimate accurately the depth map for diverse type of real objects. To address this problem, we develop Optimal Composite Depth (OCD) function through genetic programming (GP) for accurate depth estimation. The OCD function is constructed by optimally combining the primary information extracted using one/or more focus measures. The genetically developed composite function is then used to compute the optimal depth map of objects. The performance of the developed nonlinear function is investigated using both the synthetic and the real world image sequences. Experimental results demonstrate that the proposed estimator is more useful in computing accurate depth maps as compared to the existing SFF methods. Moreover, it is found that the heterogeneous function is more effective than homogeneous function.     

Optimizing image focus for 3D shape recovery through genetic algorithm

Three-dimensional information of objects is advantageous and widely used in multimedia systems and applications. Shape form focus (SFF) is a passive optical technique that reconstructs 3D shape of an object using a sequence of images with varying focus settings. In this paper, we propose an optimization of the focus measure. First, Wiener filter is applied for noise reduction from the image sequence. At the second stage, genetic algorithm (GA) is applied for focus measure optimization. GA finds the maximum focus measurement under a fitness criterion. Finally, 3D shape of the object is determined by maximizing focus measure along the optical direction. The proposed method is tested with image sequences of simulated and real objects. The performance of the proposed technique is analyzed through statistical criteria such as root mean square error (RMSE) and correlation. Comparative analysis shows the effectiveness of the proposed method.     

Shape from focus

The shape from focus method presented here uses different focus levels to obtain a sequence of object images. The sum-modified-Laplacian (SML) operator is developed to provide local measures of the quality of image focus. The operator is applied to the image sequence to determine a set of focus measures at each image point. A depth estimation algorithm interpolates a small number of focus measure values to obtain accurate depth estimates. A fully automated shape from focus system has been implemented using an optical microscope and tested on a variety of industrial samples. Experimental results are presented that demonstrate the accuracy and robustness of the proposed method. These results suggest shape from focus to be an effective approach for a variety of challenging visual inspection tasks     

多模式3维视频形状编码

目的 具有立体感和高端真实感的3D视频正越来越受到学术界和产业界的关注和重视,未来在3D影视、机器视觉、远程医疗、军事航天等领域将有着广泛的应用前景。对象基3D视频是未来3D视频技术的重要发展趋势,其中高效形状编码是对象基3D视频应用中的关键问题。但现有形状编码方法主要针对图像和视频对象,面向3D视频的形状编码算法还很少。为此,基于对象基3D视频的应用需求,提出一种基于轮廓和链码表示的高效多模式3D视频形状编码方法。方法 对于给定的3D视频形状序列逐帧进行对象轮廓提取并预处理后,进行对象轮廓活动性分析,将形状图像分成帧内模式编码图像和帧间预测模式编码图像。对于帧内编码图像,基于轮廓内链码方向约束和线性特征进行高效编码。对于帧间编码图像,采用基于链码表示的轮廓基运动补偿预测、视差补偿预测、联合运动与视差补偿预测等多种模式进行编码,以充分利用视点内对象轮廓的帧间时域相关性和视点间对象轮廓的空域相关性,从而达到高效编码的目的。结果 实验仿真结果显示所提算法性能优于经典和现有的最新同类方法,压缩效率平均能提高9.3%到64.8%不等。结论 提出的多模式3D视频形状编码方法可以有效去除对象轮廓的帧间和视点间冗余,能够进行高效编码压缩,性能优于现有同类方法,可广泛应用于对象基编码、对象基检索、对象基内容分析与理解等。     

Equal distance offset approach to representing and process planning for solid freeform fabrication of functionally graded materials

This paper deals with the representation and process planning for solid freeform fabrication (SFF) of 3D functionally graded material (FGM) objects. A novel approach of representation and process planning for SFF of FGM objects, termed as equal distance offset (EDO), is proposed. In EDO, a neutral arbitrary 3D CAD model is adaptively sliced into a series of 2D layers. Within each layer, 2D material gradients are designed and represented via dividing the 2D shape into several sub-regions enclosed by iso-composition contours. If needed, the material composition gradient within each of the sub-regions can be further specified by applying the equal distance offset algorithm to each sub-region. Using this approach, an arbitrary-shaped 3D FGM object with linear or non-linear composition gradients can be represented and fabricated via suitable SFF machines.     

Reliability measure for shape-from-focus

Shape-from-focus (SFF) is a passive technique widely used in image processing for obtaining depth-maps. This technique is attractive since it only requires a single monocular camera with focus control, thus avoiding correspondence problems typically found in stereo, as well as more expensive capturing devices. However, one of its main drawbacks is its poor performance when the change in the focus level is difficult to detect. Most research in SFF has focused on improving the accuracy of the depth estimation. Less attention has been paid to the problem of providing quality measures in order to predict the performance of SFF without prior knowledge of the recovered scene. This paper proposes a reliability measure aimed at assessing the quality of the depth-map obtained using SFF. The proposed reliability measure (the R-measure) analyzes the shape of the focus measure function and estimates the likelihood of obtaining an accurate depth estimation without any previous knowledge of the recovered scene. The proposed R-measure is then applied for determining the image regions where SFF will not perform correctly in order to discard them. Experiments with both synthetic and real scenes are presented.     

Hallucinating Stereoscopy from a Single Image

We introduce a novel method for enabling stereoscopic viewing of a scene from a single pre‐segmented image. Rather than attempting full 3D reconstruction or accurate depth map recovery, we hallucinate a rough approximation of the scene's 3D model using a number of simple depth and occlusion cues and shape priors. We begin by depth‐sorting the segments, each of which is assumed to represent a separate object in the scene, resulting in a collection of depth layers. The shapes and textures of the partially occluded segments are then completed using symmetry and convexity priors. Next, each completed segment is converted to a union of generalized cylinders yielding a rough 3D model for each object. Finally, the object depths are refined using an iterative ground fitting process. The hallucinated 3D model of the scene may then be used to generate a stereoscopic image pair, or to produce images from novel viewpoints within a small neighborhood of the original view. Despite the simplicity of our approach, we show that it compares favorably with state‐of‐the‐art depth ordering methods. A user study was conducted showing that our method produces more convincing stereoscopic images than existing semi‐interactive and automatic single image depth recovery methods.     

距离图像局部特征提取方法综述

基于距离图像的三维目标识别是计算机视觉领域的研究热点,而局部特征提取则是实现遮挡和复杂场景下三维目标识别的关键。文中首先介绍距离图像及其表示形式,详细分析法向量、曲率和形状索引等微分几何属性。进而将局部特征检测方法分类为固定尺度和自适应尺度方法,将局部特征描述方法分类为基于深度信息、基于点云空间分布和基于几何属性分布的方法,并对各种具体算法进行阐述、分析和定性评价。最后对现有方法进行归纳总结,并指出所面临的挑战及进一步研究的方向。     

学习点云邻域信息的三维物体形状补全

在现实世界中,点云数据的采集方式有激光雷达、双目相机和深度相机,但是在机器人采集过程中由于设备分辨率、周围环境等因素的影响,收集到的点云数据通常是非完整的。为了解决物体形状缺失的问题,提出了一种使用局部邻域信息的三维物体形状自动补全的网络架构。该架构包括点云特征提取网络模块和点云生成网络模块,输入为缺失的点云形状,输出为缺失部分的点云形状,将输入与输出点云形状进行合并完成物体的形状补全。采用倒角距离和测地距离进行评估,实验结果表明,在ShapeNet数据集上,平均倒角距离和平均测地距离均小于多层感知机特征提取网络与PCN网络的值,两值分别为0.000 84和0.028。对于现实中扫描的点云数据进行补全处理也达到了预期效果,说明该网络有较强的泛化性,可以修复不同类别的物体。     

结合三维几何形状信息和二维纹理的3D目标匹配

为了实现场景中三维目标与模型之间的匹配,提出了一种结合三维几何形状信息和二维纹理的三维目标匹配方法。首先提取场景中深度图像的尺度不变特征变换(SIFT)特征,用SIFT算法与三维模型重建时所用到的一系列2.5维深度图像进行一一匹配,找到与场景中目标姿态最为相似的深度图像,提取此深度图像的三维几何形状特征与模型进行匹配,实现模型的初始化,即将模型重置到与场景目标相接近的姿态。最后用融合二维纹理信息的迭代就近点（ICP）算法实现场景中目标与模型之间的匹配,从而得到场景中三维目标的准确姿态。实验结果验证了方法的可行性与精确性。     

基于目标区域特性的海面小目标检测快速算法

A fast object detection method based on object region dissimilarity and 1-D AGADM (one dimensional average gray absolute difference maximum) between object and background is proposed for real-time defection of small offshore targets. Then computational complexity, antinoise performance, the signal-to-noise ratio (SNR) gain between original images and their results as a function of SNR of original images and receiver operating characteristic (ROC) curve are analyzed and compared with those existing methods of small target detection such as two dimensional average gray absolute difference maximum (2-D AGADM), median contrast filter algorithm and multi-level filter algorithm. Experimental results and theoretical analysis have shown that the proposed method has faster speed and more adaptability to small object shape and also yields improved SNR performance.     

Plane-based optimization for 3D object reconstruction from single line drawings

In previous optimization-based methods of 3D planar-faced object reconstruction from single 2D line drawings, the missing depths of the vertices of a line drawing (and other parameters in some methods) are used as the variables of the objective functions. A 3D object with planar faces is derived by finding values for these variables that minimize the objective functions. These methods work well for simple objects with a small number N of variables. As N grows, however, it is very difficult for them to find expected objects. This is because with the nonlinear objective functions in a space of large dimension N, the search for optimal solutions can easily get trapped into local minima. In this paper, we use the parameters of the planes that pass through the planar faces of an object as the variables of the objective function. This leads to a set of linear constraints on the planes of the object, resulting in a much lower dimensional nullspace where optimization is easier to achieve. We prove that the dimension of this nullspace is exactly equal to the minimum number of vertex depths which define the 3D object. Since a practical line drawing is usually not an exact projection of a 3D object, we expand the nullspace to a larger space based on the singular value decomposition of the projection matrix of the line drawing. In this space, robust 3D reconstruction can be achieved. Compared with two most related methods, our method not only can reconstruct more complex 3D objects from 2D line drawings, but also is computationally more efficient.     

Object detection,shape recovery,and 3D modelling by depth-encoded hough voting

Detecting objects, estimating their pose, and recovering their 3D shape are critical problems in many vision and robotics applications. This paper addresses the above needs using a two stages approach. In the first stage, we propose a new method called DEHV – Depth-Encoded Hough Voting. DEHV jointly detects objects, infers their categories, estimates their pose, and infers/decodes objects depth maps from either a single image (when no depth maps are available in testing) or a single image augmented with depth map (when this is available in testing). Inspired by the Hough voting scheme introduced in [1], DEHV incorporates depth information into the process of learning distributions of image features (patches) representing an object category. DEHV takes advantage of the interplay between the scale of each object patch in the image and its distance (depth) from the corresponding physical patch attached to the 3D object. Once the depth map is given, a full reconstruction is achieved in a second (3D modelling) stage, where modified or state-of-the-art 3D shape and texture completion techniques are used to recover the complete 3D model. Extensive quantitative and qualitative experimental analysis on existing datasets [2], [3], [4] and a newly proposed 3D table-top object category dataset shows that our DEHV scheme obtains competitive detection and pose estimation results. Finally, the quality of 3D modelling in terms of both shape completion and texture completion is evaluated on a 3D modelling dataset containing both in-door and out-door object categories. We demonstrate that our overall algorithm can obtain convincing 3D shape reconstruction from just one single uncalibrated image.     

Fast Small Offshore Target Detection Based on Object Region Characteristic

A fast object detection method based on object region dissimilarity and 1-D AGADM (one dimensional average gray absolute difference maximum) between object and background is proposed for real-time defection of small offshore targets.Then computational complexity,antinoise performance,the signal-to-noise ratio (SNR) gain between original images and their results as a function of SNR of original images and receiver operating characteristic (ROC) curve are analyzed and compared with those existing methods of small target detection such as two dimensional average gray absolute difference maximum (2-D AGADM),median contrast filter algorithm and multi-level filter algorithm.Experimental results and theoretical analysis have shown that the proposed method has faster speed and more adaptability to small object shape and also yields improved SNR performance.     

基于相关型图像传感器3维人脸成像的3维AAMs人脸识别方法的研究

主动外观模型是基于统计分析建立物体2维模型的有效方法,它融合了目标的形状和纹理信息。在基于相关型图像传感器3维人脸成像的基础上,提出了一种建立3维人脸模型的方法,该方法利用由相关型图像传感器得到的深度信息和与之对应的亮度信息将2维AAMs扩展为3维AAMs,融合人脸的形状,纹理和深度信息来构建3维人脸模型。人脸识别实验结果表明,该方法在不同人脸姿态,表情和光照条件下识别效果要优于Eigenface和2维AAMs。     

多方法相融合的复杂物体深度信息的恢复

在恢复图象深度信息的方法之中,利用立体视觉的偏差来精确地定位物体的深度,是行之有效的,但只能适用于可匹配的特征点,如何建立左右图象中对应点的匹配是该方法的主要障碍;Ｓｈａｐｅ Ｆｒｏｍ Ｓｈａｄｉｎｇ方法是利用单幅图象的灰度信息获取物体表面的形状信息（表面的方向）,而不能获得其深度信息,其约束条件是表面的光滑性。在此用神经网络方法将二者融合起来,形成优势互补,用来获取物体的深度信息,通过对合成图象     

多尺度Transformer激光雷达点云3D物体检测

激光雷达点云3D物体检测,对于小物体如行人、自行车的检测精度较低,容易漏检误检,提出一种多尺度Transformer激光雷达点云3D物体检测方法MSPT-RCNN(multi-scale point transformer-RCNN),提高点云3D物体检测精度.该方法包含两个阶段,即第一阶段(RPN)和第二阶段(RCN...     

Shape recovery from equal thickness contours

A unique imaging modality based on equal thickness contours (ETC) has introduced a new opportunity for 3D shape reconstruction from multiple views. These ETC can be generated through an interference between transmitted and diffracted beams. We present a computational framework for representing each view of an object in terms of its object thickness and then integrating these representations into a 3D surface by algebraic reconstruction. In this framework, the object thickness is first derived from ideal contours and then extended to real data. For real data, the object thickness is inferred by grouping curve segments that correspond to points of second derivative maxima. At each step of the process, we use some form of regularization to ensure closeness to the original features as well as neighborhood continuity. We apply our approach to images of a submicron crystal structure obtained through a holographic process