Robust Shape Recovery from Occluding Contours Using a Linear Smoother

Recovering the shape of an object from two views fails at occluding contours of smooth objects because the extremal contours are view dependent. For three or more views, shape recovery is possible, and several algorithms have recently been developed for this purpose. We present a new approach to the multiframe stereo problem that does not depend on differential measurements in the image, which may be noise sensitive. Instead, we use a linear smoother to optimally combine all of the measurements available at the contours (and other edges) in all of the images. This allows us to extract a robust and reasonably dense estimate of surface shape, and to integrate shape information from both surface markings and occluding contours. Results are presented, which empirically show that in the presence of noise, smoothing over more than three views reduces the error even when the epipolar curve is nonplanar.     

Generalised Epipolar Constraints

In this paper we will discuss structure and motion problems for curved surfaces. These will be studied using the silhouettes or apparent contours in the images. The problem of determining camera motion from the apparent contours of curved three-dimensional surfaces, is studied. It will be shown how special points, called epipolar tangency points or frontier points, can be used to solve this problem. A generalised epipolar constraint is introduced, which applies to points, curves, as well as to apparent contours of surfaces. The theory is developed for both continuous and discrete motion, known and unknown orientation, calibrated and uncalibrated, perspective, weak perspective and orthographic cameras. Results of an iterative scheme to recover the epipolar line structure from real image sequences using only the outlines of curved surfaces, is presented. A statistical evaluation is performed to estimate the stability of the solution. It is also shown how the motion of the camera from a sequence of images can be obtained from the relative motion between image pairs.     

Structure and motion estimation from apparent contours under circular motion

In this paper, we address the problem of recovering structure and motion from the apparent contours of a smooth surface. Fixed image features under circular motion and their relationships with the intrinsic parameters of the camera are exploited to provide a simple parameterization of the fundamental matrix relating any pair of views in the sequence. Such a parameterization allows a trivial initialization of the motion parameters, which all bear physical meaning. It also greatly reduces the dimension of the search space for the optimization problem, which can now be solved using only two epipolar tangents. In contrast to previous methods, the motion estimation algorithm introduced here can cope with incomplete circular motion and more widely spaced images. Existing techniques for model reconstruction from apparent contours are then reviewed and compared. Experiment on real data has been carried out and the 3D model reconstructed from the estimated motion is presented.     

三维场景建模的全景外极面图像分析方法

文章提出由非精确摄像机运动下的图像序列建立3D环境模型的全景外极面图像方法,实现了 无特征提取的时空纹理方向精确估计、深度边界确定和遮挡恢复算法.该方法推广并结合了 外极面图像方法和全景图像方法,避免了现有运动分层方法迭代过程中的局部最小化问题,具 有计算和存储效率高、适应性强、算法鲁棒性好的优点.建立的自然景物的真实感三维环境 模型,可用于机器人全局定位的自然路标提取和真实环境虚拟再现的图像合成.     

Motion estimation in image sequences using the deformation ofapparent contours

The problem of determining the camera motion from apparent contours or silhouettes of a priori unknown curved 3D surfaces is considered. In a sequence of images, it is shown how to use the generalized epipolar constraint on apparent contours. One such constraint is obtained for each epipolar tangency point in each image pair. An accurate algorithm for computing the motion is presented based on a maximum likelihood estimate. It is shown how to generate initial estimates on the camera motion using only the tracked contours. It is also shown that in theory the motion can be calculated from the deformation of a single contour. The algorithm has been tested on several real image sequences, for both Euclidean and projective reconstruction. The resulting motion estimate is compared to motion estimates calculated independently using standard feature-based methods. The motion estimate is also used to classify the silhouettes as curves or apparent contours. The statistical evaluation shows that the technique gives accurate and stable results     

Efficient Fourier-Based Approach for Detecting Orientations and Occlusions in Epipolar Plane Images for 3D Scene Modeling

This paper presents a Fourier-based approach for automatically constructing a 3D panoramic model of a natural scene from a video sequence. The video sequences could be captured by an unstabilized camera mounted on a moving platform on a common road surface. As the input of the algorithms, seamles panoramic view images (PVIs) and epipolar plane images (EPIs) are generated after image stabilization if the camera is unstabilized. A novel panoramic EPI analysis method is proposed that combines the advantages of both PVIs and EPIs efficiently in three important steps: locus orientation detection in the Fourier frequency domain, motion boundary localization in the spatio-temporal domain, and occlusion/resolution recovery only at motion boundaries. The Fourier energy-based approaches in literature were usually for low-level local motion analysis and are therefore not accurate for 3D reconstruction and are also computationally expensive. Our panoramic EPI analysis approach is both accurate and efficient for 3D reconstruction. Examples of layered panoramic representations for large-scale 3D scenes from real world video sequences are given.     

3D Shape Recovery of Smooth Surfaces: Dropping the Fixed-Viewpoint Assumption

We present a new method for recovering the 3D shape of a featureless smooth surface from three or more calibrated images illuminated by different light sources (three of them are independent). This method is unique in its ability to handle images taken from unconstrained perspective viewpoints and unconstrained illumination directions. The correspondence between such images is hard to compute and no other known method can handle this problem locally from a small number of images. Our method combines geometric and photometric information in order to recover dense correspondence between the images and accurately computes the 3D shape. Only a single pass starting at one point and local computation are used. This is in contrast to methods that use the occluding contours recovered from many images to initialize and constrain an optimization process. The output of our method can be used to initialize such processes. In the special case of fixed viewpoint, the proposed method becomes a new perspective photometric stereo algorithm. Nevertheless, the introduction of the multiview setup, self-occlusions, and regions close to the occluding boundaries are better handled, and the method is more robust to noise than photometric stereo. Experimental results are presented for simulated and real images.     

Invariant-Based Recognition of Complex Curved 3D Objects from Image Contours

This paper addresses the problem of recognizing three-dimensional objects bounded by smooth curved surfaces from image contours found in a single photograph. The proposed approach is based on a viewpoint-invariant relationship between object geometry and certain image features under weak perspective projection. The image features themselves are viewpoint-dependent. Concretely, the set of all possible silhouette bitangents, along with the contour points sharing the same tangent direction, is the projection of a one-dimensional set of surface points where each point lies on the occluding contour for a five-parameter family of viewpoints. These image features form a one-parameter family of equivalence classes, and it is shown that each class can be characterized by a set of numerical attributes that remain constant across the corresponding five-dimensional set of viewpoints. This is the basis for describing objects by “invariant” curves embedded in high-dimensional spaces. Modeling is achieved by moving an object in front of a camera and does not require knowing the object-to-camera transformation; nor does it involve implicit or explicit three-dimensional shape reconstruction. At recognition time, attributes computed from a single image are used to index the model database, and both qualitative and quantitative verification procedures eliminate potential false matches. The approach has been implemented and examples are presented.     

On recognizing and positioning curved 3-D objects from imagecontours

An approach for explicitly relating the shape of image contours to models of curved three-dimensional objects is presented. This relationship is used for object recognition and positioning. Object models consist of collections of parametric surface patches and their intersection curves; this includes nearly all representations used in computer-aided geometric design and computer vision. The image contours considered are the projections of surface discontinuities and occluding contours. Elimination theory provides a method for constructing the implicit equation of these contours for an object observed under orthographic or perspective projection. This equation is parameterized by the object's position and orientation with respect to the observer. Determining these parameters is reduced to a fitting problem between the theoretical contour and the observed data points. The proposed approach readily extends to parameterized models. It has been implemented for a simple world composed of various surfaces of revolution and tested on several real images     

Local shape approximation from shading

Shading can be used as an independent cue for exact shape recovery, or it can be used as a supplementary cue for shape interpolation between features whose depths are known from other cues. Exact shape cannot be inferred from a local analysis of shading. However, for shape interpolation a crude local approximation may be sufficient. This paper explores the limits of such local approximations that are easy to compute. In particular, the shape of shading is used to approximate the surface in areas of monotonic change of intensity. This analysis is accompanied by a method for computing the direction of a single-point light source from the shading on occluding contours. A qualitative classification of shape near shading singularities is also discussed.This work was performed at the Massachusetts Institute of Technology, Center for Biological Information Processing.     

Accurate recovery of three-dimensional shape from image focus

A new shape-from-focus method is described which is based on a new concept, named focused image surface (FIS). FIS of an object is defined as the surface formed by the set of points at which the object points are focused by a camera lens. According to paraxial-geometric optics, there is a one-to-one correspondence between the shape of an object and the shape of its FIS. Therefore, the problem of shape recovery can be posed as the problem of determining the shape of the FIS. From the shape of FIS the shape of the object is easily obtained. In this paper the shape of the FIS is determined by searching for a shape which maximizes a focus measure. In contrast to previous literature where the focus measure is computed over the planar image detector of the camera, here the focus measure is computed over the FIS. This results in more accurate shape recovery than the traditional methods. Also, using FIS, a more accurate focused image can be reconstructed from a sequence of images than is possible with traditional methods. The new method has been implemented on an actual camera system, and the results of shape recovery and focused image reconstruction are presented     

A Unified Theory of Uncalibrated Stereo for Both Perspective and Affine Cameras

This paper addresses the recovery of structure and motion from uncalibrated images of a scene under full perspective or under affine projection. Particular emphasis is placed on the configuration of two views, while the extension to $N$ views is given in Appendix. A unified expression of the fundamental matrix is derived which is valid for any projection model without lens distortion (including full perspective and affine camera). Affine reconstruction is considered as a special projective reconstruction. The theory is elaborated in a way such that everyone having knowledge of linear algebra can understand the discussion without difficulty. A new technique for affine reconstruction is developed, which consists in first estimating the affine epipolar geometry and then performing a triangulation for each point match with respect to an implicit common affine basis.     

Surfaces from stereo: integrating feature matching, disparityestimation, and contour detection

An approach is described that integrates the processes of feature matching, contour detection, and surface interpolation to determine the three-dimensional distance, or depth, of objects from a stereo pair of images. Integration is necessary to ensure that the detected surfaces are smooth. Surface interpolation takes into account detected occluding and ridge contours in the scene; interpolation is performed within regions enclosed by these contours. Planar and quadratic patches are used as local models of the surface. Occluded regions in the image are identified, and are not used for matching and interpolation. A coarse-to-fine algorithm is presented that generates a multiresolution hierarchy of surface maps, one at each level of resolution. Experimental results are given for a variety of stereo images     

A review on egomotion by means of differential epipolar geometry applied to the movement of a mobile robot

The estimation of camera egomotion is an old problem in computer vision. Since the 1980s, many approaches based on both the discrete and the differential epipolar constraint have been proposed. The discrete case is used mainly in self-calibrated stereoscopic systems, whereas the differential case deals with a single moving camera. This article surveys several methods for 3D motion estimation unifying the mathematics convention which are then adapted to the common case of a mobile robot moving on a plane. Experimental results are given on synthetic data covering more than 0.5 million estimations. These surveyed algorithms have been programmed and are available on the Internet.     

Single frame correction of motion artifacts in PMD-based time of flight cameras

One of the leading time of flight imaging technologies for depth sensing is based on Photonic Mixer Devices (PMD). In PMD sensors each pixel samples the correlation between emitted and received light signals. Current PMD cameras compute eight correlation samples per pixel in four sequential stages to obtain depth with invariance to signal amplitude and offset variations. With motion, PMD pixels capture different depths at each stage. As a result, correlation samples are not coherent with a single depth, producing artifacts. We propose to detect and remove motion artifacts from a single frame taken by a PMD camera. The algorithm we propose is very fast, simple and can be easily included in camera hardware. We recover depth of each pixel by exploiting consistency of the correlation samples and local neighbors of the pixel. In addition, our method obtains the motion flow of occluding contours in the image from a single frame. The system has been validated in real scenes using a commercial low-cost PMD camera and high speed dynamics. In all cases our method produces accurate results and it highly reduces motion artifacts.     

Epipolar geometry from profiles under circular motion

Addresses the problem of motion estimation from profiles (apparent contours) of an object rotating on a turntable in front of a single camera. A practical and accurate technique for solving this problem from profiles alone is developed. It is precise enough to reconstruct the shape of the object. No correspondences between points or lines are necessary. Symmetry of the surface of revolution swept out by the rotating object is exploited to obtain the image of the rotation axis and the homography relating epipolar lines in two views robustly and elegantly. These, together with geometric constraints for images of rotating objects, are used to obtain first the image of the horizon, which is the projection of the plane that contains the camera centers, and then the epipoles, thus fully determining the epipolar geometry of the image sequence. The estimation of this geometry by this sequential approach avoids many of the problems found in other algorithms. The search for the epipoles, by far the most critical step, is carried out as a simple 1D optimization. Parameter initialization is trivial and completely automatic at all stages. After the estimation of the epipolar geometry, the Euclidean motion is recovered using the fixed intrinsic parameters of the camera obtained either from a calibration grid or from self-calibration techniques. Finally, the spinning object is reconstructed from its profiles using the motion estimated in the previous stage. Results from real data are presented, demonstrating the efficiency and usefulness of the proposed methods     

Curve and Surface Duals and the Recognition of Curved 3D Objects from their Silhouettes

This article addresses the problem of recognizing a solid bounded by a smooth surface in a single image. The proposed approach is based on a new representation for two- and three-dimensional shapes, called their signature, that exploits the close relationship between the dual of a surface and the dual of its silhouette in weak-perspective images. Objects are modeled by rotating them in front of a camera without any knowledge of or constraints on their motion. The signatures of their silhouettes are concatenated into a single object signature. To recognize an object from novel viewpoint other than those used during modeling, the signature of the contours extracted from a test photograph is matched to the signatures of all modeled objects signatures. This approach has been implemented, and recognition examples are presented.     

平行断层轮廓线的RBF隐函数曲面造型

将基于径向基函数（Radial Basis Function,RBF)的隐函数插值技术应用于平行断层轮廓线的曲面造型，由于RBF造型方法以曲面能量最小化为目标，因此能够生成较为光滑的曲面，其缺点是计算量较大，文中提出以分段进行曲面重构的局部RBF技术来降低问题的规模和复杂度，并提出相应的快速隐函数多边形化的算法，实验结果表明，该算法是一个较实用的造型方法。