Estimating the principal curvatures and the Darboux frame from real 3-D range data

Principal curvatures and the local Darboux frame are natural tools to be used during processes which involve extraction of geometric properties from three-dimensional (3-D) range data. As second-order features their estimations are highly sensitive to noise and therefore, until recent years, it was almost impractical to extract reliable results from real 3-D data. Since the use of more accurate 3-D range imaging equipment has become more popular, as well as the use of polyhedral meshes to approximate surfaces, evaluation of existing algorithms for curvature estimation is again relevant. The work presented here, makes some subtle but very important modifications to two such algorithms, originally suggested by Taubin (1995) and Chen and Schmitt (1992). The algorithms have been adjusted to deal with real discrete noisy range data, given as a cloud of sampled points, lying on surfaces of free-form objects. The results of this linear time (and space) complexity implementation were evaluated in a series of tests on synthetic and real input. We also present one of many possible uses for these extracted features in an efficient and robust application for the recovery of 3-D geometric primitives from range data of complex scenes. The application combines the segmentation, classification and fitting processes in a single process which advances monotonously through the recovery procedure. It is also very robust and does not use any least-squares fittings. The conclusion of this study is that with current scanning technology and the algorithms presented here, reliable estimates of the principal curvatures and Darboux frame can be extracted from real data and used in a large variety of tasks.     

Estimating differential quantities from point cloud based on a linear fitting of normal vectors

Estimation of differential geometric properties on a discrete surface is a fundamental work in computer graphics and computer vision. In this paper, we present an accurate and robust method for estimating differential quantities from unorganized point cloud. The principal curvatures and principal directions at each point are computed with the help of partial derivatives of the unit normal vector at that point, where the normal derivatives are estimated by fitting a linear function to each component of the normal vectors in a neighborhood. This method takes into account the normal information of all neighboring points and computes curvatures directly from the variation of unit normal vectors, which improves the accuracy and robustness of curvature estimation on irregular sampled noisy data. The main advantage of our approach is that the estimation of curvatures at a point does not rely on the accuracy of the normal vector at that point, and the normal vectors can be refined in the process of curvature estimation. Compared with the state of the art methods for estimating curvatures and Darboux frames on both synthetic and real point clouds, the approach is shown to be more accurate and robust for noisy and unorganized point cloud data. Supported in part by the National Natural Science Foundation of China (Grant Nos. 60672148, 60872120), the National High-Tech Research & Development Program of China (Grant Nos. 2006AA01Z301, 2008AA01Z301), and Beijing Municipal Natural Science Foundation (Grant No. 4062033)     

Constrained Structure and Motion From Multiple Uncalibrated Views of a Piecewise Planar Scene

This paper is about multi-view modeling of a rigid scene. We merge the traditional approaches of reconstructing image-extractable features and of modeling via user-provided geometry. We use features to obtain a first guess for structure and motion, fit geometric primitives, correct the structure so that reconstructed features lie exactly on geometric primitives and optimize both structure and motion in a bundle adjustment manner while enforcing the underlying constraints. We specialize this general scheme to the point features and the plane geometric primitives. The underlying geometric relationships are described by multi-coplanarity constraints. We propose a minimal parameterization of the structure enforcing these constraints and use it to devise the corresponding maximum likelihood estimator. The recovered primitives are then textured from the input images. The result is an accurate and photorealistic model.Experimental results using simulated data confirm that the accuracy of the model using the constrained methods is of clearly superior quality compared to that of traditional methods and that our approach performs better than existing ones, for various scene configurations. In addition, we observe that the method still performs better in a number of configurations when the observed surfaces are not exactly planar. We also validate our method using real images.     

Efficient three-dimensional metric object modeling from uncalibrated image sequences

This paper presents a scheme that addresses the practical issues associated with producing a geometric model of a scene using a passive sensing technique. The proposed image-based scheme comprises a recursive structure recovery method and a recursive surface reconstruction technique. The former method employs a robust corner-tracking algorithm that copes with the appearance and disappearance of features and a corner-based structure and motion estimation algorithm that handles the inclusion and expiration of features. The novel formulation and dual extended Kalman filter computational framework of the estimation algorithm provide an efficient approach to metric structure recovery that does not require any prior knowledge about the camera or scene. The newly developed surface reconstruction technique employs a visibility constraint to iteratively refine and ultimately yield a triangulated surface that envelops the recovered scene structure and can produce views consistent with those of the original image sequence. Results on simulated data and synthetic and real imagery illustrate that the proposed scheme is robust, accurate, and has good numerical stability, even when features are repeatedly absent or their image locations are affected by extreme levels of noise.     

Camera calibration in sport event scenarios

The main goal of this paper is the design of a novel and robust methodology for calibrating cameras from a single image in sport scenarios, such as a soccer field, or a basketball or tennis court. In these sport scenarios, the only references we use to calibrate the camera are the lines and circles delimiting the different regions. The first problem we address is the extraction of image primitives, including the challenging problems of shaded regions and lens distortion. From these primitives, we automatically recognise the location of the sport court in the scene by estimating the homography which matches the actual court with its projection onto the image. This is achieved even when only a few primitives are available. Finally, from this homography, we recover the camera calibration parameters. In particular, we estimate the focal length as well as the position and orientation in the 3D space. We present some experiments on models and real courts which illustrate the accuracy of the proposed methodology.     

Robust segmentation of primitives from range data in the presenceof geometric degeneracy

This paper addresses a common problem in the segmentation of range images. We present methods for the least-squares fitting of spheres, cylinders, cones, and tori to 3D point data, and their application within a segmentation framework. Least-squares fitting of surfaces other than planes, even of simple geometric type, has rarely been studied. Our main application areas of this research are reverse engineering of solid models from depth-maps and automated 3D inspection where reliable extraction of these surfaces is essential. Our fitting method has the particular advantage of being robust in the presence of geometric degeneracy, i.e., as the principal curvatures of the surfaces being fitted decrease, the results returned naturally become closer and closer to those surfaces of “simpler type”, i.e., planes, cylinders, cones, or spheres, which best describe the data. Many other methods diverge because, in such cases, various parameters or their combination become infinite     

Good match exploration using triangle constraint

This paper presents a novel method for addressing the problem of finding more good feature pairs between images, which is one of the most fundamental tasks in computer vision and pattern recognition. We first select matched features by Bi-matching as seed points, then organize these seed points by adopting the Delaunay triangulation algorithm. Finally, triangle constraint is used to explore good matches. The experimental evaluation shows that our method is robust to most geometric and photometric transformations including rotation, scale change, blur, viewpoint change, JPEG compression and illumination change, and significantly improves both the number of correct matches and the matching score. And the application on estimating the fundamental matrix for a pair of images is also shown. Both the experiments and the application demonstrate the robust performance of our method.     

A geometric interpretation of weak-perspective motion

We present a geometric interpretation of the problem of motion recovery from three weak-perspective images. Our interpretation is based on reducing the problem of estimating the motion to a problem of finding triangles on a sphere whose angles are known. Using this geometric interpretation, a simple method to completely recover the motion parameters using three images is developed. The results of running the algorithm on real images are presented. In addition, we describe which of the various motion parameters can be recovered already from two images     

Axis estimation and grouping of rotationally symmetric object segments

This paper proposes a new method for estimating the symmetric axis of a pottery from its small fragment using surface geometry. Also, it provides a scheme for grouping such fragments into shape categories using distribution of surface curvature. For automatic assembly of pot from broken sherds, axis estimation is an important task and when a fragment is small, it is difficult to estimate axis orientation since it looks like a patch of a sphere and conventional methods mostly fail. But the proposed method provides fast and robust axis estimation by using multiple constraints. The computational cost is also too lowered. To estimate the symmetric axis, the proposed algorithm uses three constraints: (1) The curvature is constant on a circumference C_H. (2) The curvature is invariant in any scale. (3) Also the principal curvatures does not vary on C_H. C_H is a planar circle which is one of all the possible circumferences of a pottery or sherd. A hypothesis test for axis is performed using maximum likelihood. The variance of curvature, multi-scale curvature and principal curvatures is computed in the likelihood function. We also show that the principal curvatures can be used for grouping of sherds. The grouping of sherds will reduce the computation significantly by omitting impossible configurations in broken pottery assembly process.     

基尼指数成本量指导的抗遮挡光场深度估计方法

光场相机能够实现一次拍摄即获得三维场景的多视角信息,在深度估计领域中具有独特优势.但是,当场景中存在复杂遮挡时,现有深度估计方法提取深度信息的精度会明显降低.针对该问题,设计一种基尼指数成本量指导下的抗遮挡光场深度估计方法.首先,利用光场重聚焦方法获得焦栈图像;然后,构造中心视角与其他视角的基尼指数成本量,并根据成本最小原则计算得到初始深度图;最后,结合彩色图进行联合引导滤波,获得最终的高精度的深度图像.实验结果表明,所提方法对复杂场景更加鲁棒,能够在较小的算法复杂度下获取更好的深度估计结果.相比于其他先进方法,所提方法获取的深度图精度更高,图像边缘保留效果更好,在HCI数据集上的MSE₁₀₀指标平均降低约7.8%.     

Fast segmentation of range images into planar regions by scan line grouping

A novel technique is presented for rapid partitioning of surfaces in range images into planar patches. The method extends and improves Pavlidis' algorithm (1976), proposed for segmenting images from electron microscopes. The new method is based on region growing where the segmentation primitives are scan line grouping features instead of individual pixels. We use a noise variance estimation to automatically set thresholds so that the algorithm can adapt to the noise conditions of different range images. The proposed algorithm has been tested on real range images acquired by two different range sensors. Experimental results show that the proposed algorithm is fast and robust.     

Structure from Motion: Beyond the Epipolar Constraint

The classic approach to structure from motion entails a clear separation between motion estimation and structure estimation and between two-dimensional (2D) and three-dimensional (3D) information. For the recovery of the rigid transformation between different views only 2D image measurements are used. To have available enough information, most existing techniques are based on the intermediate computation of optical flow which, however, poses a problem at the locations of depth discontinuities. If we knew where depth discontinuities were, we could (using a multitude of approaches based on smoothness constraints) accurately estimate flow values for image patches corresponding to smooth scene patches; but to know the discontinuities requires solving the structure from motion problem first. This paper introduces a novel approach to structure from motion which addresses the processes of smoothing, 3D motion and structure estimation in a synergistic manner. It provides an algorithm for estimating the transformation between two views obtained by either a calibrated or uncalibrated camera. The results of the estimation are then utilized to perform a reconstruction of the scene from a short sequence of images.The technique is based on constraints on image derivatives which involve the 3D motion and shape of the scene, leading to a geometric and statistical estimation problem. The interaction between 3D motion and shape allows us to estimate the 3D motion while at the same time segmenting the scene. If we use a wrong 3D motion estimate to compute depth, we obtain a distorted version of the depth function. The distortion, however, is such that the worse the motion estimate, the more likely we are to obtain depth estimates that vary locally more than the correct ones. Since local variability of depth is due either to the existence of a discontinuity or to a wrong 3D motion estimate, being able to differentiate between these two cases provides the correct motion, which yields the least varying estimated depth as well as the image locations of scene discontinuities. We analyze the new constraints, show their relationship to the minimization of the epipolar constraint, and present experimental results using real image sequences that indicate the robustness of the method.     

结合MSCRs与MSERs的自然场景文本检测

目的 目前,基于MSERs(maximally stable extremal regions)的文本检测方法是自然场景图像文本检测的主流方法。但是自然场景图像中部分文本的背景复杂多变,MSERs算法无法将其准确提取出来,降低了该类方法的鲁棒性。本文针对自然场景图像文本背景复杂多变的特点,将MSCRs(maximally stable color regions)算法用于自然场景文本检测,提出一种结合MSCRs与MSERs的自然场景文本检测方法。方法 首先采用MSCRs算法与MSERs算法提取候选字符区域;然后利用候选字符区域的纹理特征训练随机森林字符分类器,对候选字符区域进行分类,从而得到字符区域;最后,依据字符区域的彩色一致性和几何邻接关系对字符进行合并,得到最终文本检测结果。结果 本文方法在ICDAR 2013上的召回率、准确率和F值分别为71.9%、84.1%和77.5%,相对于其他方法的召回率和F值均有所提高。结论 本文方法对自然场景图像文本检测具有较强的鲁棒性,实验结果验证了本文方法的有效性。     

单幅自然场景深度恢复

离焦测距算法是一种用于恢复场景深度信息的常用算法。传统的离焦测距算法通常需要采集多幅离焦图像,实际应用中具有很大的制约性。文中基于局部模糊估计提出单幅离焦图像深度恢复算法。基于局部模糊一致性的假设,本文采用简单而有效的两步法恢复输入图像的深度信息：1)通过求取输入离焦图和利用已知高斯核再次模糊图之间的梯度比得到边缘处稀疏模糊图 2)将边缘位置模糊值扩离至全部图像,完整的相对深度信息即可恢复。为了获得准确的场景深度信息,本文加入几何条件约束、天空区域提取策略来消除颜色、纹理以及焦点平面歧义性带来的影响,文中对各种类型的图片进行对比实验,结果表明该算法能在恢复深度信息的同时有效抑制图像中的歧义性。     

Hybrid architecture for shape reconstruction and object recognition

The proposed architecture is aimed to recover 3-D shape information from gray-level images of a scene: to build a geometric representation of the scene in terms of geometric primitives; and to reason about the scene. The novelty of the architecture is in fact the integration of different approaches: symbolic reasoning techniques typical of knowledge representation in artificial intelligence, algorithmic capabilities typical of artificial vision schemes, and analogue techniques typical of artificial neural networks. Experimental results obtained by means of an implemented version of the proposed architecture acting on real scene images are reported to illustrate the system capabilities. © 1996 John Wiley & Sons, Inc.     

高度场八叉树的体特征表达算法

体特征表达对用户理解和认知虚拟环境有着至关重要的作用。当前的体特征表达算法由于存储量大且不易于在GPU中加速等问题,渲染效率低下,难以满足场景可视化的实时性需求。针对这一问题,提出了一种高效的高度场八叉树体特征表达算法,不仅解决了传统高度场仅能表达2.5维模型,无法表达真三维模型的问题,而且为体特征表达提供了一种新的可行途径。算法使用八叉树结构生成三维模型的高度场表示,将传统的z向高度场扩展到x,y,z三个方向的高度场。首先,提出了三角面片预处理方法,保证模型精度和数据的完整性;其次,提出了基于投影变换的高度场表示判断及栅格化方法,将几何图元转换成二维空间的高度场数据;最后,提出了基于高度场八叉树的光线投射算法。实验结果表明,算法能极大地减少存储量,具有较高的光线投射效率,表达三维模型时取得较好效果。     

Probabilistic visibility evaluation using geometry proxies

Evaluating the visibility between two points is a fundamental problem for ray‐tracing and path‐tracing algorithms. Ideally, visibility computations are organized such that a minimum number of geometric primitives need to be checked for each ray. Replacing complex geometric shapes by a simpler set of primitives is one strategy to control the amount of intersection calculations. However, approximating the original geometry introduces inaccuracies in e.g. shadow regions when shadow rays are intersected with the approximate geometry. This paper presents a theoretical framework for probabilistic visibility evaluation. When intersecting a shadow ray with the scene, we randomly select the original geometry, the approximated geometry, or one of several correction terms, to be tested. Not all shadow rays will therefore intersect the original geometry, but our method is able to produce unbiased images that converge to the correct solution. Although probabilistic visibility evaluation is an experimental idea, we show several example scenes that highlight the potential for future improvements.