Reconstructing a 3-D depth map from one or more images

Several algorithms are suggested for recovering depth and orientation maps of a surface from its image intensities. They combine the advantages of stereo vision and shape-from-shading (SFS) methods. These algorithms generate accurate, unambiguous and dense surface depth and orientation maps. Most of the existing SFS algorithms cannot be directly extended to combine stereo images because the recovery of surface depth and that of orientation are separated in these formulations. We first present an SFS algorithm that couples the generation of depth and orientation maps. This formulation also ensures that the reconstructed surface depth and its orientation are consistent. The SFS algorithm for a single image is then extended to utilize stereo images. The correspondence over stereo images is established simultaneously with the generation of surface depth and orientation. An alternative approach is also suggested for combining stereo and SFS techniques. This approach can be used to combine needle maps which are directly available from other sources such as photometric stereo. Finally we present an algorithm to combine sparse depth measurements with an orientation map to reconstruct a surface. The same algorithm can be combined with the above algorithms for solving the SFS problem with sparse depth measurements. Thus various information sources can be used to accurately reconstruct a surface.     

Color-encoded structured light for rapid active ranging

In this paper, we discuss a novel strategy for rapid acquisition of the range map of a scene employing color-encoded structured light. This technique offers several potential advantages including increased speed and improved accuracy. In this approach we illuminate the scene with a single encoded grid of colored light stripes. The indexing problem, that of matching a detected image plane stripe with its position in the projection grid, is solved from a knowledge of the color grid encoding. In fact, the possibility exists for the first time to acquire high-resolution range data in real time for modest cost, since only a single projection and single color image are required. Grid to grid alignment problems associated with previous multistripe techniques are eliminated, as is the requirement for dark interstices between grid stripes. Scene illumination is more uniform, simplifying the stripe detection problem, and mechanical difficulties associated with the equipment design are significantly reduced.     

Automatic Reconstruction of Buildings from Stereoscopic Image Sequences

A vision–based 3-D scene analysis system is described that is capable to model complex real–world scenes like streets and buildings automatically from stereoscopic image pairs. Input to the system is a sequence of stereoscopic images taken with two standard CCD Cameras and TV lenses. The relative orientation of both cameras to each other is known by calibration. The camerapair is then moved throughout the scene and a long sequence of closely spaced views is recorded. Each of the stereoscopic image pairs is rectified and a dense map of 3-D suface points is obtained by area correlation, object segmentation, interpolation, and triangulation. 3-D camera motion relative to the scene coordinate system is tracked directly from the image sequence which allows to fuse 3-D surface measurements from different viewpoints into a consistent 3-D model scene. The surface geometry of each scene object is approximated by a triangular surface mesh which stores the suface texture in a texture map. From the textured 3-D models, realistic looking image sequences from arbitrary view points can be synthesized using computer graphics.     

具有相切面的曲面立体线画图标记研究

为了更好地理解三维景物，介绍了前人对线画图标记的研究成果，同时建立了一种标记具有相切面的曲面立体线画图的方法，并给出了具有相切面的曲面立体线画图的标记规则。由于线画图中的节点是由三面角构成的顶点的投影，因此对于画出隐藏线的具有相切面的曲面立体线画图，其合法的节点标记形式有38种，其中Y型节点有6种，W型节点有18种，V型节点有14种。新的标记方法适用于由平面立体和曲面立体组合而成的复杂立体线画图。     

Machine vision system for curved surface inspection

This application-oriented paper discusses a non-contact 3D range data measurement system to improve the performance of the existing 2D herring roe grading system. The existing system uses a single CCD camera with unstructured halogen lighting to acquire and analyze the shape of the 2D shape of the herring roe for size and deformity grading. Our system will act as an additional system module, which can be integrated into the existing 2D grading system, providing the additional third dimension to detect deformities in the herring roe, which were not detected in the 2D analysis. Furthermore, the additional surface depth data will increase the accuracy of the weight information used in the existing grading system. In the proposed system, multiple laser light stripes are projected into the herring roe and the single B/W CCD camera records the image of the scene. The distortion in the projected line pattern is due to the surface curvature and orientation. Utilizing the linear relation between the projected line distortion and surface depth, the range data was recovered from a single camera image. The measurement technique is described and the depth information is obtained through four steps: (1) image capture, (2) stripe extraction, (3) stripe coding, (4) triangulation, and system calibration. Then, this depth information can be converted into the curvature and orientation of the shape for deformity inspection, and also used for the weight estimation. Preliminary results are included to show the feasibility and performance of our measurement technique. The accuracy and reliability of the computerized herring roe grading system can be greatly improved by integrating this system into existing system in the future.     

Gaussian curvature using fundamental forms for binary voxel data

Local curvature characterizes every point of a surface and measures its deviation from a plane, locally. One application of local curvature measures within the field of image and geometry processing is object segmentation. Here, we present and evaluate a novel algorithm based on the fundamental forms to calculate the curvature on surfaces of objects discretized with respect to a regular three-dimensional grid. Thus, our new algorithm is applicable to voxel data, which are created e.g. from computed tomography (CT). Existing algorithms for binary data used the Gauss map, rather than fundamental forms. For the calculation of the fundamental forms, derivatives of a surface in tangent directions in every point of the surface have to be computed. Since the surfaces exist on grids with restricted resolution, these derivatives have to be discretized. In the presented method, this is realized by projecting the tangent plane onto the discrete object surface. The most important parameter of the proposed algorithm is the size of the chosen window for the calculation of the gradient. The size of this window has to be selected according to object size as well as with respect to distances between objects. In our experiments, an algorithm based on the Gauss map provided inconsistent values for simple test objects, whereas our method provides consistent values. We report quantitative results on various test geometries, compare our method to two algorithms working on gray value data and demonstrate the practical applicability of our novel algorithm to CT-reconstructions of Greenlandic firn.     

A state of the art in structured light patterns for surface profilometry

Shape reconstruction using coded structured light is considered one of the most reliable techniques to recover object surfaces. Having a calibrated projector-camera pair, a light pattern is projected onto the scene and imaged by the camera. Correspondences between projected and recovered patterns are found and used to extract 3D surface information. This paper presents an up-to-date review and a new classification of the existing techniques. Some of these techniques have been implemented and compared, obtaining both qualitative and quantitative results. The advantages and drawbacks of the different patterns and their potentials are discussed.     

Interactive machinability analysis of free-form surfaces using multiple-view image space techniques on the GPU

In this paper we present a set of graphics hardware accelerated algorithms to interactively evaluate the machinability of complex free-form surfaces. These algorithms work in image space and easily interface with all common formats available on CAD systems. The running time of these algorithms is independent of the complexity of the surface to be analyzed and depends only on the size of the projected image of the surface and the largest available tool head. Interactive speed is achieved through clever use of data-parallel techniques that map nicely onto the programming model of modern programmable graphics processing units. We demonstrate a method for pre-calculating and storing the machinability of a surface within a texture to further reduce rendering costs. The algorithms are implemented and tested on a complex set of parts and their performance has been analyzed.     

双目视觉的新颖实时局部2维地图构建方法

根据轮式机器人移动的特点,提出一种采用双目视觉的新颖实时局部2维栅格地图构建算法。首先,提出虚拟高度线投影成像原理,将场景均匀栅格化,在栅格中引入虚拟高度线,并将其投影到立体视觉系统的立体图对中产生投影线,将求解场景点高度值的问题转化为在投影线上和水平视差搜索范围内寻找具有最大相似测度的对应点问题;然后,提出一种新颖的局部2维地图构建方法,该方法以机器人所能越过的最大高度为阈值,对高于阈值部分的虚拟高度投影线上的场景点由其在图像对中的相似测度确定其是否为障碍物区域。实验结果表明,该方法满足机器人导航所要求的有效性和实时性,并可应用到构建3维地图。     

Multiflash stereopsis: depth-edge-preserving stereo with small baseline illumination

Traditional stereo matching algorithms are limited in their ability to produce accurate results near depth discontinuities, due to partial occlusions and violation of smoothness constraints. In this paper, we use small baseline multi-flash illumination to produce a rich set of feature maps that enable acquisition of discontinuity preserving point correspondences. First, from a single multi-flash camera, we formulate a qualitative depth map using a gradient domain method that encodes object relative distances. Then, in a multiview setup, we exploit shadows created by light sources to compute an occlusion map. Finally, we demonstrate the usefulness of these feature maps by incorporating them into two different dense stereo correspondence algorithms, the first based on local search and the second based on belief propagation. Experimental results show that our enhanced stereo algorithms are able to extract high quality, discontinuity preserving correspondence maps from scenes that are extremely challenging for conventional stereo methods. We also demonstrate that small baseline illumination can be useful to handle specular reflections in stereo imagery. Different from most existing active illumination techniques, our method is simple, inexpensive, compact, and requires no calibration of light sources.     

Learning the Semantic Landscape: embedding scene knowledge in object tracking

The accuracy of object tracking methodologies can be significantly improved by utilizing knowledge about the monitored scene. Such scene knowledge includes the homography between the camera and ground planes and the occlusion landscape identifying the depth map associated with the static occlusions in the scene. Using the ground plane, a simple method of relating the projected height and width of people objects to image location is used to constrain the dimensions of appearance models. Moreover, trajectory modeling can be greatly improved by performing tracking on the ground-plane tracking using global real-world noise models for the observation and dynamic processes. Finally, the occlusion landscape allows the tracker to predict the complete or partial occlusion of object observations. To facilitate plug and play functionality, this scene knowledge must be automatically learnt. The paper demonstrates how, over a sufficient length of time, observations from the monitored scene itself can be used to parameterize the semantic landscape.     

Geometric constraints and stereo disparity computation

Most stereo techniques compute disparity assuming that it varies slowly along surfaces. We quantify and justify this assumption, using weak assumptions about surface orientation distributions in the world to derive the density of disparity surface orientations. The small disparity change assumption is justified by the orientation density's heavy bias toward disparity surfaces that are nearly parallel to the image plane. In addition, the bias strengthens with smaller baselines, larger focal lengths, and as surfaces move farther from the cameras. To analyze current stereo techniques, we derive three densities from the first density, those of the disparity gradient magnitude, the directional derivative of disparity, and the difference in disparity between neighboring surface points. The latter may be used in Bayesian algorithms computing dense disparity fields. The directional derivative density and the disparity difference density both show that feature-based algorithms should strongly favor small disparity changes, contrary to several well-known algorithms. Finally, we use our original surface orientation density and the gradient magnitude density to derive two new surfaces-from-stereo techniques, techniques combining feature-based matching and surface reconstruction. The first uses the densities to severely restrict the search range for the optimum fit. The second incorporates the surface orientation density into the optimization criteria, producing a Bayesian formulation. Both algorithms are shown to be efficient and effective.The authors would like to thank James Miller for helpful discussions and comments on earlier versions of this paper and Wes Turner for making the camera system work. They would also like to acknowledge the financial support of the National Science Foundation under grant IRI-9217195.     

虚拟海洋场景的实时模拟研究与实现

为了提高虚拟海洋场景的实时性和真实感,提出了用多重Perlin噪声的叠加和波浪粒子相结合的方法建立实时海洋动画系统.该系统采用Perlin噪声和投影网格绘制技术模拟海面几何形态;并采用Perlin噪声和波浪粒子来跟踪水波的运动,从而快速、简便地实现了水上物体运动产生的水波特效;在光影效果上加入纹理技术建立光照模型,生成了复杂的、真实感强的海洋场景.仿真实验结果表明,该方法适用于大多数流体的模拟,渲染的海洋场景具有很好的真实感和时效性.     

The correspondence framework for 3D surface matching algorithms

Beyond the inherent technical challenges, current research into the three dimensional surface correspondence problem is hampered by a lack of uniform terminology, an abundance of application specific algorithms, and the absence of a consistent model for comparing existing approaches and developing new ones. This paper addresses these challenges by presenting a framework for analysing, comparing, developing, and implementing surface correspondence algorithms. The framework uses five distinct stages to establish correspondence between surfaces. It is general, encompassing a wide variety of existing techniques, and flexible, facilitating the synthesis of new correspondence algorithms. This paper presents a review of existing surface correspondence algorithms, and shows how they fit into the correspondence framework. It also shows how the framework can be used to analyse and compare existing algorithms and develop new algorithms using the framework’s modular structure. Six algorithms, four existing and two new, are implemented using the framework. Each implemented algorithm is used to match a number of surface pairs. Results demonstrate that the correspondence framework implementations are faithful implementations of existing algorithms, and that powerful new surface correspondence algorithms can be created.     

Nonstructured light-based sensing for 3D reconstruction

Structured light-based sensing (SLS) requires the illumination to be coded either spatially or temporally in the illuminated pattern. However, while the former demands the use of uniquely coded spatial windows whose size grows with the reconstruction resolution and thereby demanding increasing smoothness on the imaged scene, the latter demands the use of multiple image captures. This article presents how the illumination of a very simple pattern plus a single image capture can also achieve 3D reconstruction. The illumination and imaging setting has the configuration of a typical SLS system, comprising a projector and a camera. The difference is, the illumination is not much more than a checkerboard-like pattern - a non-structured pattern in the language of SLS - that does not provide direct correspondence between the camera’s image plane and the projector’s display panel. The system works from the image progressively, first constructing the orientation map of the target object from the observed grid-lines, then inferring the depth map by the use of a few tricks related to interpolation. The system trades off little accuracy of the traditional SLSs with simplicity of its operation. Compared to temporally coded SLSs, the system has the essence that it requires only one image capture to operate; compared with spatially coded SLSs, it requires no use of spatial windows, and in turn a less degree of smoothness on the object surface; compared with methods like shape from shading and photometric stereo, owing to the use of artificial illumination it is less affected by the surface reflectance property of the target surface and the ambient lighting condition.     

Multi-object detection and tracking by stereo vision

This paper presents a new stereo vision-based model for multi-object detection and tracking in surveillance systems. Unlike most existing monocular camera-based systems, a stereo vision system is constructed in our model to overcome the problems of illumination variation, shadow interference, and object occlusion. In each frame, a sparse set of feature points are identified in the camera coordinate system, and then projected to the 2D ground plane. A kernel-based clustering algorithm is proposed to group the projected points according to their height values and locations on the plane. By producing clusters, the number, position, and orientation of objects in the surveillance scene can be determined for online multi-object detection and tracking. Experiments on both indoor and outdoor applications with complex scenes show the advantages of the proposed system.     

Near-real-time stereo matching with slanted surface modeling and sub-pixel accuracy

This paper presents a new stereo matching algorithm which takes into consideration surface orientation at the per-pixel level. Two disparity calculation passes are used. The first pass assumes that surfaces in the scene are fronto-parallel and generates an initial disparity map, from which the disparity plane orientations of all pixels are estimated and refined. In the second pass, the matching costs for different pixels are aggregated along the estimated disparity plane orientations using adaptive support weights, where the support weights of neighboring pixels are calculated using a combination of four terms: a spatial proximity term, a color similarity term, a disparity similarity term, and an occlusion handling term. The disparity search space is quantized at sub-pixel level to improve the accuracy of the disparity results. The algorithm is designed for parallel execution on Graphics Processing Units (GPUs) for near-real-time processing speed. The evaluation using Middlebury benchmark shows that the presented approach outperforms existing real-time and near-real-time algorithms in terms of subpixel level accuracy.     

A surface based approach to recognition of geometric features for quality freeform surface machining

The paper presents a surface-based approach for geometric feature recognition for the purpose of automating the process planning of freeform surface machining. The proposed approach consists of the following four steps for recognition of the geometric features: conversion and preprocessing of the surface geometry data, subdivision of NURBS surface, reconstruction of surface orientation areas, and recognition of geometric features. The proposed scheme assumes that the input geometry data form is based on an IGES CAD model and the surface model can be represented in the form of trimmed NURBS surfaces. The connectivity relations of the product surfaces are analyzed and each surface is subdivided into orientation regions based on the surface normal vector over a certain point density grid, and then all the connected regions with the same orientation type are grouped into surface orientation areas. After that, the geometric feature will be recognized through the analysis of area connectivity and relationship. The paper describes the developed algorithms on surface orientation region subdivision, surface orientation area reconstruction, and geometric feature recognition. The verified feasibility study of the developed method is also presented.