Object pose from 2-D to 3-D point and line correspondences

In this paper we present a method for optimally estimating the rotation and translation between a camera and a 3-D object from point and/or line correspondences. First we devise an error function and second we show how to minimize this error function. The quadratic nature of this function is made possible by representing rotation and translation with a dual number quaternion. We provide a detailed account of the computational aspects of a trust-region optimization method. This method compares favourably with Newton's method which has extensively been used to solve the problem at hand, with Faugeras-Toscani's linear method (Faugeras and Toscani 1986) for calibrating a camera, and with the Levenberg-Marquardt non-linear optimization method. Finally we present some experimental results which demonstrate the robustness of our method with respect to image noise and matching errors.This work has been supported by the Esprit programme through the SECOND project (Esprit-BRA No. 6769).     

3-D to 2-D Pose Determination with Regions

This paper presents a novel approach to parts-based object recognition in the presence of occlusion. We focus on the problem of determining the pose of a 3-D object from a single 2-D image when convex parts of the object have been matched to corresponding regions in the image. We consider three types of occlusions: self-occlusion, occlusions whose locus is identified in the image, and completely arbitrary occlusions. We show that in the first two cases this is a convex optimization problem, derive efficient algorithms, and characterize their performance. For the last case, we prove that the problem of finding valid poses is computationally hard, but provide an efficient, approximate algorithm. This work generalizes our previous work on region-based object recognition, which focused on the case of planar models.     

On the location error of curved edges in low-pass filtered 2-D and3-D images

The authors study the location error of curved edges in two- and three-dimensional images after analog and digital low-pass filtering. The zero crossing of a second derivative filter is a well-known edge localization criterion. The second derivative in gradient direction (SDGD) produces a predictable bias in edge location towards the centers of curvature while the linear Laplace filter produces a shift in the opposite direction. Their sum called PLUS (PLUS=Laplace+SDGD) leads to an edge detector that finds curved edges one order more accurately than its constituents. This argument holds irrespective of the dimension. The influence of commonly used low-pass filters (such as the PSF originating from diffraction limited optics using incoherent light (2-D), the Gaussian filter with variable cutoff point (D-D), and the isotropic uniform filter (D-D)) is studied     

Perception of 3-D surfaces from 2-D contours

Inference of 3-D shape from 2-D contours in a single image is an important problem in machine vision. The authors survey classes of techniques proposed in the past and provide a critical analysis. They show that two kinds of symmetries in figures, which are known as parallel and skew symmetries, give significant information about surface shape for a variety of objects. They derive the constraints imposed by these symmetries and show how to use them to infer 3-D shape. They also discuss the zero Gaussian curvature (ZGC) surfaces in depth and show results on the recovery of surface orientation for various ZGC surfaces     

Matching 3-D Models to 2-D Images

We consider the problem of analytically characterizing the set of all 2-D images that a group of 3-D features may produce, and demonstrate that this is a useful thing to do. Our results apply for simple point features and point features with associated orientation vectors when we model projection as a 3-D to 2-D affine transformation. We show how to represent the set of images that a group of 3-D points produces with two lines (1-D subspaces), one in each of two orthogonal, high-dimensional spaces, where a single image group corresponds to one point in each space. The images of groups of oriented point features can be represented by a 2-D hyperbolic surface in a single high-dimensional space. The problem of matching an image to models is essentially reduced to the problem of matching a point to simple geometric structures. Moreover, we show that these are the simplest and lowest dimensional representations possible for these cases.We demonstrate the value of this way of approaching matching by applying our results to a variety of vision problems. In particular, we use this result to build a space-efficient indexing system that performs 3-D to 2-D matching by table lookup. This system is analytically built and accessed, accounts for the effects of sensing error, and is tested on real images. We also derive new results concerning the existence of invariants and non-accidental properties in this domain. Finally, we show that oriented points present unexpected difficulties: indexing requires fundamentally more space with oriented than with simple points, we must use more images in a motion sequence to determine the affine structure of oriented points, and the linear combinations result does not hold for oriented points.     

Character location in scene images from digital camera

In this paper, a robust, connected-component-based character locating method is presented. It is an important part of an optical character recognition (OCR) system. Color clustering is used to separate the color image into homogeneous color layers. Next, for each color layer, every connected component in color layers is analyzed using black adjacency graph (BAG), and the component-bounding box is computed. Then, for coarse detection of characters, an aligning-and-merging-analysis (AMA) scheme is proposed to locate all the potential characters using the information about the bounding boxes of connected components in all color layers. Finally, to eliminate false characters, a four-step identification of characters is used. The experimental results in this paper have proven that the method is effective.     

Recovery of the 3-D location and motion of a rigid object through camera image (An Extended Kalman Filter Approach)

This paper deals with the problem of locating a rigid object and estimating its motion in three dimensions. This involves determining the position and orientation of the object at each instant when an image is captured by a camera, and recovering the motion of the object between consecutive frames.In the implementation scheme used here, a sequence of camera images, digitized at the sample instants, is used as the initial input data. Measurements are made of the locations of certain features (e.g., maximum curvature points of an image contour, corners, edges, etc.) on the 2-D images. To measure the feature locations a matching algorithm is used, which produces correspondences between the features in the image and the object.Using the measured feature locations on the image, an algorithm is developed to solve the location and motion problem. The algorithm is an extended Kalman filter modeled for this application.Department of Electrical Engineering and Alberta Center for Machine Intelligence and Robotics, University of Alberta     

Matching 3-D line segments with applications to multiple-objectmotion estimation

A two-stage algorithm for matching line segments using three-dimensional data is presented. In the first stage, a tree-search based on the orientation of the line segments is applied to establish potential matches. the sign ambiguity of line segments is fixed by a simple congruency constraint. In the second stage, a Hough clustering technique based on the position of line segments is applied to verify potential matches. Any paired line segments of a match that cannot be brought to overlap by the translation determined by the clustering are removed from the match. Unlike previous methods, this algorithm combats noise more effectively, and ensures the global consistency of a match. While the original motivation for the algorithm is multiple-object motion estimation from stereo image sequences, the algorithm can also be applied to other domains, such as object recognition and object model construction from multiple views     

便携式三维扫描仪摄像机与电磁接收联合标定

摄像机和电磁接收模块的联合标定是便携式三维扫描仪应用中的重要问题。文中提出了一种对摄像机和电磁接收模块的联合标定方法,该方法依据摄像机内参数、激光平面参数和电磁接收模块相对基坐标系位姿矩阵之间的特定关系,进行一次标定实验即可求解出他们之间的变换矩阵。计算结果表明,此方法简单、实用、快捷,标定精度能满足便携式三维扫描仪的应用要求。     

Shape-based indexing scheme for camera view invariant 3-D object retrieval

Camera view invariant 3-D object retrieval is an important issue in many traditional and emerging applications such as security, surveillance, computer-aided design (CAD), virtual reality, and place recognition. One straightforward method for camera view invariant 3-D object retrieval is to consider all the possible camera views of 3-D objects. However, capturing and maintaining such views require an enormous amount of time and labor. In addition, all camera views should be indexed for reasonable retrieval performance, which requires extra storage space and maintenance overhead. In the case of shape-based 3-D object retrieval, such overhead could be relieved by considering the symmetric shape feature of most objects. In this paper, we propose a new shape-based indexing and matching scheme of real or rendered 3-D objects for camera view invariant object retrieval. In particular, in order to remove redundant camera views to be indexed, we propose a camera view skimming scheme, which includes: i) mirror shape pairing and ii) camera view pruning according to the symmetrical patterns of object shapes. Since our camera view skimming scheme considerably reduces the number of camera views to be indexed, it could relieve the storage requirement and improve the matching speed without sacrificing retrieval accuracy. Through various experiments, we show that our proposed scheme can achieve excellent performance.     

3-D reconstruction from tomographic data using 2-D active contours.

Reconstructing three-dimensional (3-D) shapes of structures like internal organs from tomographic data is an important problem in medical imaging. Various forms of the deformable surface model have been proposed to tackle it, but they are either computationally expensive or limited to tubular shapes. In this paper a 3-D reconstruction mechanism that requires only 2-D deformations is proposed. Advantages of the proposed model include that it is conformable to any 3-D shape, efficient, and highly parallelizable. Most importantly, it requires from the user an initial 2-D contour on only one of the tomograph slices to start with. Experimental results are shown to illustrate the performance of the model.     

A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation and Estimation

In this paper, we present an analytic solution to the problem of estimating an unknown number of 2-D and 3-D motion models from two-view point correspondences or optical flow. The key to our approach is to view the estimation of multiple motion models as the estimation of a single multibody motion model. This is possible thanks to two important algebraic facts. First, we show that all the image measurements, regardless of their associated motion model, can be fit with a single real or complex polynomial. Second, we show that the parameters of the individual motion model associated with an image measurement can be obtained from the derivatives of the polynomial at that measurement. This leads to an algebraic motion segmentation and estimation algorithm that applies to most of the two-view motion models that have been adopted in computer vision. Our experiments show that the proposed algorithm out-performs existing algebraic and factorization-based methods in terms of efficiency and robustness, and provides a good initialization for iterative techniques, such as Expectation Maximization, whose performance strongly depends on good initialization. This paper is an extended version of [34]. The authors thank Sampreet Niyogi for his help with the experimental section of the paper. This work was partially supported by Hopkins WSE startup funds, UIUC ECE startup funds, and by grants NSF CAREER IIS-0347456, NSF CAREER IIS-0447739, NSF CRS-EHS-0509151, NSF-EHS-0509101, NSF CCF-TF-0514955, ONR YIP N00014-05-1-0633 and ONR N00014-05-1-0836. René Vidal received his B.S. degree in Electrical Engineering (highest honors) from the Universidad Católica de Chile in 1997 and his M.S. and Ph.D. degrees in Electrical Engineering and Computer Sciences from the University of California at Berkeley in 2000 and 2003, respectively. In 2004, he joined The Johns Hopkins University as an Assistant Professor in the Department of Biomedical Engineering and the Center for Imaging Science. He has co-authored more than 70 articles in biomedical imaging, computer vision, machine learning, hybrid systems, robotics, and vision-based control. Dr. Vidal is recipient of the 2005 NFS CAREER Award, the 2004 Best Paper Award Honorable Mention at the European Conference on Computer Vision, the 2004 Sakrison Memorial Prize, the 2003 Eli Jury Award, and the 1997 Award of the School of Engineering of the Universidad Católica de Chile to the best graduating student of the school. Yi Ma received his two bachelors' degree in Automation and Applied Mathematics from Tsinghua University, Beijing, China in 1995. He received an M.S. degree in Electrical Engineering and Computer Science (EECS) in 1997, an M.A. degree in Mathematics in 2000, and a PhD degree in EECS in 2000 all from the University of California at Berkeley. Since August 2000, he has been on the faculty of the Electrical and Computer Engineering Department of the University of Illinois at Urbana-Champaign, where he is now an associate professor. In fall 2006, he is a visiting faculty at the Microsoft Research in Asia, Beijing, China. He has written more than 40 technical papers and is the first author of a book, entitled “An Invitation to 3-D Vision: From Images to Geometric Models,” published by Springer in 2003. Yi Ma was the recipient of the David Marr Best Paper Prize at the International Conference on Computer Vision in 1999 and Honorable Mention for the Longuet-Higgins Best Paper Award at the European Conference on Computer Vision in 2004. He received the CAREER Award from the National Science Foundation in 2004 and the Young Investigator Program Award from the Office of Naval Research in 2005.     

利用序列ISAR图像获取空间目标3-D信息的方法

如何从空间目标序列性二维(2-D,Two-Dimentional)逆合成孔径雷达(ISAR,Inverse Synthetic Aperture Radar)成像获取目标的三维(3-D)信息,是目标特征自动识别(ATR,Automatic Target Recognition)技术的重要研究课题。利用双向射线跟踪(BART,Bidirectional Analytic Ray Tracing)方法,计算连续多角度观测条件下空间目标的电磁散射数据,并由此获取空间目标的ISAR序列2-D图像。再利用KLT(Kanade-Lucas-Tomasi)特征跟踪算法,跟踪提取2\|D序列ISAR图像中的特征点(强散射点),获得其2-D坐标。然后,基于正交因式分解法(OFM,Orthographic Factorization Method),计算强散射点的3\|D坐标,获取空间目标的3-D信息。通过简单六棱柱模型,验证重构算法的精度;并以ENVISAT卫星模型为例,给出强散射点的3-D重构结果。结果表明,本文对空间目标3\|D信息获取方法能有效地从ISAR序列2-D图像中重构目标的三维信息。     

A Galerkin Method for the Simulation of the Transient 2-D/2-D and 3-D/3-D Linear Boltzmann Equation

Many production steps used in the manufacturing of integrated circuits involve the deposition of material from the gas phase onto wafers. Models for these processes should account for gaseous transport in a range of flow regimes, from continuum flow to free molecular or Knudsen flow, and for chemical reactions at the wafer surface. We develop a kinetic transport and reaction model whose mathematical representation is a system of transient linear Boltzmann equations. In addition to time, a deterministic numerical solution of this system of kinetic equations requires the discretization of both position and velocity spaces, each two-dimensional for 2-D/2-D or each three-dimensional for 3-D/3-D simulations. Discretizing the velocity space by a spectral Galerkin method approximates each Boltzmann equation by a system of transient linear hyperbolic conservation laws. The classical choice of basis functions based on Hermite polynomials leads to dense coefficient matrices in this system. We use a collocation basis instead that directly yields diagonal coefficient matrices, allowing for more convenient simulations in higher dimensions. The systems of conservation laws are solved using the discontinuous Galerkin finite element method. First, we simulate chemical vapor deposition in both two and three dimensions in typical micron scale features as application example. Second, stability and convergence of the numerical method are demonstrated numerically in two and three dimensions. Third, we present parallel performance results which indicate that the implementation of the method possesses very good scalability on a distributed-memory cluster with a high-performance Myrinet interconnect.     

A Constraint-Satisfaction Approach for 3-D Object Recognition by Integrating 2-D and 3-D Data

We present a framework for recognition of 3D objects by integrating 2D and 3D sensory data. The major thrust of this work is to efficiently utilize all relevant data, both 2D and 3D, in the early stages of recognition, in order to reduce the computational requirements of the recognition process. To achieve this goal, we formulate the problem as a constraint–satisfaction problem (CSP). Rather than directly solving the CSP, a problem of exponential complexity, we only enforce local consistency in low-order polynomial time. This step of local-consistency enforcement can significantly decrease the computational load on subsequent recognition modules by (1) significantly reducing the uncertainty in the correspondence between scene and model features and (2) eliminating many erroneous model objects and irrelevant scene features. A novel method is presented for efficiently constructing a CSP corresponding to a combination of 2D and 3D scene features. Performance of the proposed framework is demonstrated using simulated and real experiments involving visual (2D) and tactile (3D) data.     

Automatic mesh generation in 2-D and 3-D objects

The object of the paper is to propose a simple method of mesh generation for objects with complicated boundary configurations and present a program which can generate the element data required for Finite Element Analysis based on these methods. The area to be meshed is considered as a combination of different sub-areas which are further sub-divided into the required number of triangular elements. The main part of the program developed in FORTRAN is included in this paper.     

On learning to recognize 3-D objects from examples

Previous results on nonlearnability of visual concepts relied on the assumption that such concepts are represented as sets of pixels. The author uses an approach developed by Haussler (1989) to show that under an alternative, feature-based representation, recognition is probably approximately correct (PAC) learnable from a feasible number of examples in a distribution-free manner     

3-D Object Pose Determination Using Computer Vision

This paper is concerned with determining the position and orientation of a three-dimensional object using computer vision. An algorithm is presented which uses an enclosing object and the central moments for an initial estimate. A refined estimation is achieved by a nonlinear least squares technique.     

Self-assembly of 3-D structures using 2-D folding tiles

Natural Computing - Self-assembly is a process which is ubiquitous in natural, especially biological systems. It occurs when groups of relatively simple components spontaneously combine to form...