Direct surface reconstruction from a moving sensor

A technique is described for surface reconstruction from a sequence of images when the sensor motion is known. The reconstruction takes the form of an orthographic depth map in a viewer-independent frame. Following a mathematical justification, the technique is illustrated with examples using noisy synthetic images.     

Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth,Illumination and Albedo

Shape from shading (SfS) and stereo are two fundamentally different strategies for image-based 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on a matching process that establishes correspondences across images. This difference in approaching the reconstruction problem yields complementary advantages that are worthwhile being combined. So far, however, most “joint” approaches are based on an initial stereo mesh that is subsequently refined using shading information. In this paper we follow a completely different approach. We propose a joint variational method that combines both cues within a single minimisation framework. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detail-preserving anisotropic second-order smoothness term. Moreover, we extend the resulting model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes the approach applicable to objects with non-uniform albedo as well as to scenes with unknown illumination. Experiments for synthetic and real-world images demonstrate the benefits of our combined approach: They not only show that our method is capable of generating very detailed reconstructions, but also that joint approaches are feasible in practice.     

A Generic and Provably Convergent Shape-from-Shading Method for Orthographic and Pinhole Cameras

We describe a mathematical and algorithmic study of the Lambertian “Shape-From-Shading” problem for orthographic and pinhole cameras. Our approach is based upon the notion of viscosity solutions of Hamilton-Jacobi equations. This approach provides a mathematical framework in which we can show that the problem is well-posed (we prove the existence of a solution and we characterize all the solutions). Our contribution is threefold. First, we model the camera both as orthographic and as perspective (pinhole), whereas most authors assume an orthographic projection (see Horn and Brooks (1989) for a survey of the SFS problem up to 1989 and Zhang et al. (1999), Kozera (1998), Durou et al. (2004) for more recent ones); thus we extend the applicability of shape from shading methods to more realistic acquisition models. In particular it extends the work of Prados et al. (2002a) and Rouy and Tourin (1992). We provide some novel mathematical formulations of this problem yielding new partial differential equations. Results about the existence and uniqueness of their solutions are also obtained. Second, by introducing a “generic” Hamiltonian, we define a general framework allowing to deal with both models (orthographic and perspective), thereby simplifying the formalization of the problem. Thanks to this unification, each algorithm we propose can compute numerical solutions corresponding to all the modeling. Third, our work allows us to come up with two new generic algorithms for computing numerical approximations of the “continuous solution of the “Shape-From-Shading” problem as well as a proof of their convergence toward that solution. Moreover, our two generic algorithms are able to deal with discontinuous images as well as images containing black shadows. First online version published in October, 2005     

Analysis and Approximation of Some Shape-from-Shading Models for Non-Lambertian Surfaces

The reconstruction of a 3D object or a scene is a classical inverse problem in Computer Vision. In the case of a single image this is called the Shape-from-Shading (SfS) problem and it is known to be ill-posed even in a simplified version like the vertical light source case. A huge number of works deals with the orthographic SfS problem based on the Lambertian reflectance model, the most common and simplest model which leads to an eikonal-type equation when the light source is on the vertical axis. In this paper, we want to study non-Lambertian models since they are more realistic and suitable whenever one has to deal with different kind of surfaces, rough or specular. We will present a unified mathematical formulation of some popular orthographic non-Lambertian models, considering vertical and oblique light directions as well as different viewer positions. These models lead to more complex stationary non-linear partial differential equations of Hamilton–Jacobi type which can be regarded as the generalization of the classical eikonal equation corresponding to the Lambertian case. However, all the equations corresponding to the models considered here (Oren–Nayar and Phong) have a similar structure so we can look for weak solutions to this class in the viscosity solution framework. Via this unified approach, we are able to develop a semi-Lagrangian approximation scheme for the Oren–Nayar and the Phong model and to prove a general convergence result. Numerical simulations on synthetic and real images will illustrate the effectiveness of this approach and the main features of the scheme, also comparing the results with previous results in the literature.     

What can two images tell us about a third one?

This paper discusses the problem of predicting image features in an image from image features in two other images and the epipolar geometry between the three images. We adopt the most general camera model of perspective projection and show that a point can be predicted in the third image as a bilinear function of its images in the first two cameras, that the tangents to three corresponding curves are related by a trilinear function, and that the curvature of a curve in the third image is a linear function of the curvatures at the corresponding points in the other two images. Our analysis relies heavily on the use of the fundamental matrix which has been recently introduced (Faugeras et al, 1992) and on the properties of a special plane which we call the trifocal plane. Though the trinocular geometry of points and lines has been very recently addressed, our use of the differential properties of curves for prediction is unique.We thus completely solve the following problem: given two views of an object, predict what a third view would look like. The problem and its solution bear upon several areas of computer vision, stereo, motion analysis, and model-based object recognition. Our answer is quite general since it assumes the general perspective projection model for image formation and requires only the knowledge of the epipolar geometry for the triple of views. We show that in the special case of orthographic projection our results for points reduce to those of Ullman and Basri (Ullman and Basri, 1991). We demonstrate on synthetic as well as on real data the applicability of our theory.     

基于图像分割的三维重构

三维重构方法是医学图像可视化系统、治疗计划系统的重要技术。基于图像分割的三维重构方法结合了图像分割、等值面抽取、网格简化三种技术,是不同于传统Marching Cubes算法的一种三维重构方法。它首先将医学图像分割为二值图,然后利用Marching Cubes方法进行等值面抽取,最后对得到的网格模型进行简化。实验结果表明,基于图像分割的三维重构方法加快了Marching Cubes的运算速度,改善了重构的效果,有利于实现对基于三维重构的大型几何模型的实时绘制和交互。     

一种面向三维点集的快速表面重构算法

在对目前比较流行的空间三角化算法进行对比研究的基础上 ,对 Hugues Hoppe提出的算法进行了改进 ,即借鉴 Marching Cubes算法的基本思想 ,首先通过自动选取适当的参数 ,用包围盒方法将三维散乱点划分为数据区域 ;然后求取点的切平面及法向 ,同时采用广度优先算法遍历数据点来调整法向和快速地求取 Marching Cubes的等势函数 ;最后用基于查表法的 Marching Cubes来输出三角面片 ,即得到表面模型 .实验结果表明 ,改进后的算法效率有较大的提高 .新算法不仅适用于表面三维散乱点数据 ,也可以对体数据进行重构 ,具有一定的通用性 .     

Self-calibration of an affine camera from multiple views

A key limitation of all existing algorithms for shape and motion from image sequences under orthographic, weak perspective and para-perspective projection is that they require the calibration parameters of the camera. We present in this paper a new approach that allows the shape and motion to be computed from image sequences without having to know the calibration parameters. This approach is derived with the affine camera model, introduced by Mundy and Zisserman (1992), which is a more general class of projections including orthographic, weak perspective and para-perspective projection models. The concept of self-calibration, introduced by Maybank and Faugeras (1992) for the perspective camera and by Hartley (1994) for the rotating camera, is then applied for the affine camera.This paper introduces the 3 intrinsic parameters that the affine camera can have at most. The intrinsic parameters of the affine camera are closely related to the usual intrinsic parameters of the pin-hole perspective camera, but are different in the general case. Based on the invariance of the intrinsic parameters, methods of self-calibration of the affine camera are proposed. It is shown that with at least four views, an affine camera may be self-calibrated up to a scaling factor, leading to Euclidean (similarity) shape réconstruction up to a global scaling factor. Another consequence of the introduction of intrinsic and extrinsic parameters of the affine camera is that all existing algorithms using calibrated affine cameras can be assembled into the same framework and some of them can be easily extented to a batch solution.Experimental results are presented and compared with other methods using calibrated affine cameras.     

基于透视和正投影图像的3D运动与结构恢复

论文首次研究了由一幅正投影图像和一幅透视投影图像的特征点对应来进行刚体3D运动重建与结构恢复的问题,给出了有效的线性算法。以往的由运动恢复结构的工作主要集中于一组透视图像或一组正投影(通常为仿射)图像,文中采用了透视模型和正投影模型的组合。数据模拟实验结果显示该方法是比较有效和稳定的。     

A paraperspective factorization method for shape and motionrecovery

The factorization method, first developed by Tomasi and Kanade (1992), recovers both the shape of an object and its motion from a sequence of images, using many images and tracking many feature points to obtain highly redundant feature position information. The method robustly processes the feature trajectory information using singular value decomposition (SVD), taking advantage of the linear algebraic properties of orthographic projection. However, an orthographic formulation limits the range of motions the method can accommodate. Paraperspective projection, first introduced by Ohta et al. (1981), is a projection model that closely approximates perspective projection by modeling several effects not modeled under orthographic projection, while retaining linear algebraic properties. Our paraperspective factorization method can be applied to a much wider range of motion scenarios, including image sequences containing motion toward the camera and aerial image sequences of terrain taken from a low-altitude airplane     

A Fast Marching Method for Hamilton-Jacobi Equations Modeling Monotone Front Propagations

In this paper we present a generalization of the Fast Marching method introduced by J.A. Sethian in 1996 to solve numerically the eikonal equation. The new method, named Buffered Fast Marching (BFM), is based on a semi-Lagrangian discretization and is suitable for Hamilton-Jacobi equations modeling monotonically advancing fronts, including Hamilton-Jacobi-Bellman and Hamilton-Jacobi-Isaacs equations which arise in the framework of optimal control problems and differential games. We also show the convergence of the algorithm to the viscosity solution. Finally we present several numerical tests comparing the BFM method with other existing methods. This research was partially supported by the MIUR Project 2006 “Modellistica Numerica per il Calcolo Scientifico ed Applicazioni Avanzate” and by INRIA–Futurs and ENSTA, Paris, France.     

基于多尺度的Marching Cube改进算法

在医学图像三维重建中,经典的等值面重建算法Marching Cube是一种比较常用的算法。该算法具有可以在给定阈值的情况下提取任意三维数据场的等值面的优点,但因需计算大量的数据和三角面片而使得该算法速度较慢。提出了使用不同尺度的等值面重建理论,实现了一个多尺度的Marching Cube算法,经过实验对比,该算法比原Marching Cube算法具有效率高、速度快的优点。     

Multibody Grouping from Motion Images

We want to deduce, from a sequence of noisy two-dimensional images of a scene of several rigid bodies moving independently in three dimensions, the number of bodies and the grouping of given feature points in the images to the bodies. Prior processing is assumed to have identified features or points common to all frames and the images are assumed to be created by orthographic projection (i.e., perspective effects are minimal). We describe a computationally inexpensive algorithm that can determine which points or features belong to which rigid body using the fact that, with exact observations in orthographic projection, points on a single body lie in a three or less dimensional linear manifold of frame space. If there are enough observations and independent motions, these manifolds can be viewed as a set linearly independent, four or less dimensional subspaces. We show that the row echelon canonical form provides direct information on the grouping of points to these subspaces. Treatment of the noise is the most difficult part of the problem. This paper uses a statistical approach to estimate the grouping of points to subspaces in the presence of noise by computing which partition has the maximum likelihood. The input data is assumed to be contaminated with independent Gaussian noise. The algorithm can base its estimates on a user-supplied standard deviation of the noise, or it can estimate the noise from the data. The algorithm can also be used to estimate the probability of a user-specified partition so that the hypothesis can be combined with others using Bayesian statistics.     

CT冠脉造影中冠状动脉中心线的提取

提出了一种冠状动脉中心线的提取方法,该方法以FastMarching(快速行进)算法为基础,首先对图像进行下采样,接着使用血管增强滤波器对图像进行滤波,然后利用FastMarching算法提取下采样图像中初始点之间的最短路径,并使用最短路径在原始图像上计算冠状动脉中心线。在公共数据集上的验证实验表明,该方法具有较好的鲁棒性和准确性。     

Spherical surface parameterization for perspective shape from shading

We propose a new mathematical formulation for perspective shape from shading (PSFS) problems. Our approach is based on representing the unknown surface as a spherical surface, expressed by Euclidean distance to the optical centre, as opposed to the traditional representation by distance from the image plane. We show that our parameterization is better suited for perspective camera models and results in simpler models and equations for classical PSFS problems with a light source in the optical centre. The unknown distance field satisfies a simple isotropic Eikonal equation on the unit sphere in the case of a Lambertian surface reflection model. This is in contrast to previous methods with depth field parameterization, which result in anisotropic equations. Adding light attenuation to the model, we show that the distance field satisfies an Eikonal type of equation with a zero order term. We show how both Eikonal equations can be approximated by very efficient Fast Marching methods. A number of numerical tests and examples are provided to demonstrate our approach, and to compare with previous work. Our results indicate competitive accuracy and computational time that are several orders of magnitude faster than state-of-the-art iterative algorithms. A preliminary investigation indicates that our method could be used in more general PSFS problems.     

A Parallel Algorithm for Volume Projections on SIMD Mesh-Connected Computers

Projections are widely used in machine vision, volume rendering, and computer graphics. For applications with 3D volume data, we design a parallel projection algorithm on SIMD mesh-connected computers and implement the algorithm on the Parallel Algebraic Logic (PAL) computer. The algorithm is a parallel ray casting algorithm for both orthographic and perspective projections. It decomposes a volume projection into two transformations that can be implemented in the SIMD fashion to solve the data distribution and redistribution problem caused by non-regular data access patterns in volume projections.     

A solution to illumination direction estimation of a shaded image: Genetic algorithm

In oblique shape from shading (SfS), the illumination direction is essential for recovering the 3D surface of a shaded image. On the other hand, fast marching methods (FMM) are SfS algorithms that use the mechanism of wave propagation to reconstruct the surface. In this paper, the estimation of illumination direction is addressed and we model it as an optimization problem. The idea is to minimize the inconsistency of wave propagation of FMM during the reconstruction. As the consistency of wave propagation is a multi-modal function of illumination direction, genetic algorithm (GA) is utilized. The proposed algorithm is examined on four synthetic models and a real world object. The experimental results show that the proposed algorithm is superior to benchmark methods.     

Feature point tracking in time-varying images

This paper examines the problem of tracking feature points in dynamic imagery. Two types of imaging geometries are examined: orthographic projection and perspective projection. In each case, the goal is to perform the feature point tracking with a minimal number of images. The feature point tracking problem is shown to be intractable and a grddey heuristic is presented.     

基于仿真的无人机遥感图像拼接误差分析

针对多幅无人机遥感图像连续拼接时,累积误差逐渐增大导致拼接的图像严重扭曲变形的问题,提出了一种基于前方交会的投影误差修正算法。该算法根据前方交会原理,解算出同名点对所对应3D点的空间坐标;然后,将所有3D点正射投影到同一物平面上,并将正射点重投影到像平面上,获取校正后的同名点对;最后,利用M估计抽样一致性(MSAC)算法估计单应性矩阵,并进行图像拼接演示。仿真实验结果表明,所提算法能有效地消除投影误差,进而达到抑制无人机遥感图像拼接误差的目的。     

Lines in one orthographic and two perspective views

We introduce a linear algorithm to recover the Euclidean motion between an orthographic and two perspective cameras from straight line correspondences filling the gap in the analysis of motion estimation from line correspondences for various projection models. The general relationship between lines in three views is described by the trifocal tensor. Euclidean structure from motion for three perspective views is a special case in which the relationship is defined by a collection of three matrices. Here, we describe the case of two calibrated perspective views and an orthographic view. Similar to the other cases, our linear algorithm requires 13 or more line correspondences to recover 27 coefficients of the trifocal tensor.