Toward object-based heuristics

Recovering the 3-D shape of an object from its 2-D image contour is an important problem in computer vision. In this correspondence, the author motivates and develops two object-based heuristics. The structured nature of objects is the motivation for the nonaccidental alignment criterion: parallel coordinate axes within the object's bounding contour correspond to object-centered coordinate axes. The regularity and symmetry inherent in many man-made objects is the motivation for the orthogonal basis constraint. An oblique set of coordinate axes in the image is presumed to be the projection of an orthogonal set of 3-D coordinate axes in the scene. These object-based heuristics are used to recover shape in both real and synthetic images     

Three-dimensional vision structure for robot applications

Focuses on the structure of robot sensing systems and the techniques for measuring and preprocessing 3-D data. To get the information required for controlling a given robot function, the sensing of 3-D objects is divided into four basic steps: transduction of relevant object properties (primarily geometric and photometric) into a signal; preprocessing the signal to improve it; extracting 3-D object features; and interpreting them. Each of these steps usually may be executed by several alternative techniques (tools). Tools for the transduction of 3-D data and data preprocessing are surveyed. The performance of each tool depends on the specific vision task and its environmental conditions, both of which are variable. Such a system includes so-called tool-boxes, one box for each sensing step, and a supervisor, which controls iterative sensing feedback loops and consists of a rule-based program generator and a program execution controller. Sensing step sequences and tools are illustrated for two 3-D vision applications at SRI International Company: visually guided robot arc welding and locating identical parts in a bin     

Active recognition through next view planning: a survey

3-D object recognition involves using image-computable features to identify 3-D object. A single view of a 3-D object may not contain sufficient features to recognize it unambiguously. One needs to plan different views around the given object in order to recognize it. Such a task involves an active sensor—one whose parameters (external and/or internal) can be changed in a purposive manner. In this paper, we review two important applications of an active sensor. We first survey important approaches to active 3-D object recognition. Next, we review existing approaches towards another important application of an active sensor namely, that of scene analysis and interpretation.     

Estimating the kinematics and structure of a rigid object from asequence of monocular images

The problem considered involves the use of a sequence of noisy monocular images of a three-dimensional moving object to estimate both its structure and kinematics. The object is assumed to be rigid, and its motion is assumed to be smooth. A set of object match points is assumed to be available, consisting of fixed features on the object, the image plane coordinates of which have been extracted from successive images in the sequence. Structure is defined as the 3-D positions of these object feature points, relative to each other. Rotational motion occurs about the origin of an object-centered coordinate system, while translational motion is that of the origin of this coordinate system. In this work, which is a continuation of the research done by the authors and reported previously (ibid., vol.PAMI-8, p.90-9, Jan. 1986), results of an experiment with real imagery are presented, involving estimation of 28 unknown translational, rotational, and structural parameters, based on 12 images with seven feature points     

Part I: Modeling image curves using invariant 3-D object curvemodels-a path to 3-D recognition and shape estimation from imagecontours

This paper and its companion are concerned with the problems of 3-D object recognition and shape estimation from image curves using a 3-D object curve model that is invariant to affine transformation onto the image space, and a binocular stereo imaging system. The objects of interest here are the ones that have markings (e.g., characters, letters, special drawings and symbols, etc.) on their surfaces. The 3-D curves on the object are modeled as B-splines, which are characterized by a set of parameters (the control points) from which the 3-D curve can be totally generated. The B-splines are invariant under affine transformations. That means that the affine projected object curve onto the image space is a B-spline whose control points are related to the object control points through the affine transformation. Part I deals with issues relating to the curve modeling process. In particular, the authors address the problems of estimating the control points from the data curve, and of deciding on the “best” order B-spline and the “best” number of control points to be used to model the image or object curve(s). A minimum mean-square error (mmse) estimation technique which is invariant to affine transformations is presented as a noniterative, simple, and fast approach for control point estimation. The “best” B-spline is decided upon using a Bayesian selection rule. Finally, we present a matching algorithm that allocates a sample curve to one of p prototype curves when the sample curve is an a priori unknown affine transformation of one of the prototype curves stored in the data base. The approach is tried on a variety of images of real objects     

Binocular motion tracking by gaze fixation control and three-dimensional shape reconstruction

It is an easy task for the human visual system to gaze continuously at an object moving in three-dimensional (3-D) space. While tracking the object, human vision seems able to comprehend its 3-D shape with binocular vision. We conjecture that, in the human visual system, the function of comprehending the 3-D shape is essential for robust tracking of a moving object. In order to examine this conjecture, we constructed an experimental system of binocular vision for motion tracking. The system is composed of a pair of active pan-tilt cameras and a robot arm. The cameras are for simulating the two eyes of a human while the robot arm is for simulating the motion of the human body below the neck. The two active cameras are controlled so as to fix their gaze at a particular point on an object surface. The shape of the object surface around the point is reconstructed in real-time from the two images taken by the cameras based on the differences in the image brightness. If the two cameras successfully gaze at a single point on the object surface, it is possible to reconstruct the local object shape in real-time. At the same time, the reconstructed shape is used for keeping a fixation point on the object surface for gazing, which enables robust tracking of the object. Thus these two processes, reconstruction of the 3-D shape and maintaining the fixation point, must be mutually connected and form one closed loop. We demonstrate the effectiveness of this framework for visual tracking through several experiments.     

3-D shape reconstruction in an active stereo vision system using genetic algorithms

The recovery of 3-D shape information (depth) using stereo vision analysis is one of the major areas in computer vision and has given rise to a great deal of literature in the recent past. The widely known stereo vision methods are the passive stereo vision approaches that use two cameras. Obtaining 3-D information involves the identification of the corresponding 2-D points between left and right images. Most existing methods tackle this matching task from singular points, i.e. finding points in both image planes with more or less the same neighborhood characteristics. One key problem we have to solve is that we are on the first instance unable to know a priori whether a point in the first image has a correspondence or not due to surface occlusion or simply because it has been projected out of the scope of the second camera. This makes the matching process very difficult and imposes a need of an a posteriori stage to remove false matching.In this paper we are concerned with the active stereo vision systems which offer an alternative to the passive stereo vision systems. In our system, a light projector that illuminates objects to be analyzed by a pyramid-shaped laser beam replaces one of the two cameras. The projections of laser rays on the objects are detected as spots in the image. In this particular case, only one image needs to be treated, and the stereo matching problem boils down to associating the laser rays and their corresponding real spots in the 2-D image. We have expressed this problem as a minimization of a global function that we propose to perform using Genetic Algorithms (GAs). We have implemented two different algorithms: in the first, GAs are performed after a deterministic search. In the second, data is partitioned into clusters and GAs are independently applied in each cluster. In our second contribution in this paper, we have described an efficient system calibration method. Experimental results are presented to illustrate the feasibility of our approach. The proposed method yields high accuracy 3-D reconstruction even for complex objects. We conclude that GAs can effectively be applied to this matching problem.     

Implicit and explicit camera calibration: theory and experiments

By implicit camera calibration, we mean the process of calibrating a camera without explicitly computing its physical parameters. Implicit calibration can be used for both three-dimensional (3-D) measurement and generation of image coordinates. In this paper, we present a new implicit model based on the generalized projective mappings between the image plane and two calibration planes. The back-projection and projection processes are modelled separately to ease the computation of distorted image coordinates from known world points. A set of constraints of perspectivity is derived to relate the transformation parameters of the two calibration planes. Under the assumption of the radial distortion model, we present a computationally efficient method for explicitly correcting the distortion of image coordinates in frame buffer without involving the computation of camera position and orientation. By combining with any linear calibration techniques, this method makes explicit the camera physical parameters. Extensive experimental comparison of our methods with the classic photogrammetric method and Tsai's (1986) method in the aspects of 3-D measurement (both absolute and relative errors), the prediction of image coordinates, and the effect of the number of calibration points, is made using real images from 15 different depth values     

3-D shape recovery using distributed aspect matching

An approach to the recovery of 3-D volumetric primitives from a single 2-D image is presented. The approach first takes a set of 3-D volumetric modeling primitives and generates a hierarchical aspect representation based on the projected surfaces of the primitives; conditional probabilities capture the ambiguity of mappings between levels of the hierarchy. From a region segmentation of the input image, the authors present a formulation of the recovery problem based on the grouping of the regions into aspects. No domain-independent heuristics are used; only the probabilities inherent in the aspect hierarchy are exploited. Once the aspects are recovered, the aspect hierarchy is used to infer a set of volumetric primitives and their connectivity. As a front end to an object recognition system, the approach provides the indexing power of complex 3-D object-centered primitives while exploiting the convenience of 2-D viewer-centered aspect matching; aspects are used to represent a finite vocabulary of 3-D parts from which objects can be constructed     

Formation preserving path finding in 3-D terrains

Navigation of a group of autonomous agents that are required to maintain a formation is a challenging task which has not been studied much especially in 3-D terrains. This paper presents a novel approach to collision free path finding of multiple agents preserving a predefined formation in 3-D terrains. The proposed method could be used in many areas like navigation of semi-automated forces (SAF) at unit level in military simulations and non-player characters (NPC) in computer games. The proposed path finding algorithm first computes an optimal path from an initial point to a target point after analyzing the 3-D terrain data from which it constructs a weighted graph. Then, it employs a real-time path finding algorithm specifically designed to realize the navigation of the group from one waypoint to the successive one on the optimal path generated at the previous stage, preserving the formation and avoiding collision. Software was developed to test the methods discussed here.     

Object recognition by combining paraperspective images

This paper provides a study on object recognition under paraperspective projection. Discussed is the problem of determining whether or not a given image was obtained from a 3-D object to be recognized. First it is clarified that paraperspective projection is the first-order approximation of perspective projection. Then it is shown that, if we represent an object as a set of its feature points and the object undergoes a rigid transformation or an affine transformation, any paraperspective image can be expressed as a linear combination of several appropriate paraperspective images: we need at least three images for rigid transformations; whereas we need at least two images for affine transformations. Particularly in the case of a rigid transformation, the coefficients of the combination have to satisfy two conditions: orthogonality and norm equality. A simple algorithm to solve the above problem based on these properties is presented: a linear, single-shot algorithm. Some experimental results with synthetic images and real images are also given.This work was done while the author was with ATR Auditory and Visual Perception Research Laboratories.Advanced Research Laboratory Hitachi, Ltd.     

基于曲线的纹理映射

本文提出一种纹理映射的新方法。二维平面中的一条曲线与三维曲面上的一条曲线相对应，两条曲线上的点再相对应。当对应曲线足够多时，就可获理完成纹理映射所需的足够信息。     

广义的以物体为中心的行程编码：一种新的三维物体模型

三维物体的表达和识别是图象理解和场景分析的核心问题,三维模型在三维物体的识别和场景分析中具有十分重要的作用.三维模型应该是以物体为中心的,能够提供该场景的所有有用信息.物体的大小,形状及朝向应均可从该模型中提取得到.本文提出了一种新的三维物体模型——广义的以物体为中心的行程编码.它包括物体的GORC物理数据结构,详细的形状描述和抽象描述.物体的高层次的表达可以通过以GORC编码的物理数据直接提取得到.三维的GORC是二维的以物体为中心的行程编码在三维上的推广,它兼有物体的体积表达和表面表达的优点.三维物体的GORC模型可以很容易地由其深度信息构造得出,基于GORC的投影运算,图象代数运算以及特征提取均可非常有效地实现.     

Paraperspective ≡ affine

It is shown that the set of all paraperspective images with arbitrary reference point and the set of all affine images of a 3-D object are identical. Consequently, all uncalibrated paraperspective images of an object can be constructed from a 3-D model of the object by applying an affine transformation to the model, and every affine image of the object represents some uncalibrated paraperspective image of the object. It follows that the paraperspective images of an object can be expressed as linear combinations of any two non-degenerate images of the object. When the image position of the reference point is given the parameters of the affine transformation (and, likewise, the coefficients of the linear combinations) satisfy two quadratic constraints. Conversely, when the values of parameters are given the image position of the reference point is determined by solving a bi-quadratic equation.     

Moment feature-based three-dimensional tracking using two high-speed vision systems

When considering real-world applications of robot control with visual servoing, both three-dimensional (3-D) information and a high feedback rate are required. We have developed a 3-D target-tracking system with a 1-ms feedback rate using two high-speed vision systems called Column Parallel Vision (CPV) systems. To obtain 3-D information, such as position, orientation and shape parameters of the target object, a feature-based algorithm has been introduced using moment feature values extracted from vision systems for a spheroidal object model. Also, we propose a new 3-D self-windowing method to extract the target in 3-D space using epipolar geometry, which is an extension of the conventional self-windowing method in 2-D images.