A low cost 3D scanner based on structured light

Automatic 3D acquisition devices (often called 3D scanners) allow to build highly accurate models of real 3D objects in a cost- and time-effective manner. We have experimented this technology in a particular application context: the acquisition of Cultural Heritage artefacts. Specific needs of this domain are: medium-high accuracy, easy of use, affordable cost of the scanning device, self-registered acquisition of shape and color data, and finally operational safety for both the operator and the scanned artefacts. According to these requirements, we designed a low-cost 3D scanner based on structured light which adopts a new, versatile colored stripe pattern approach. We present the scanner architecture, the software technologies adopted, and the first results of its use in a project regarding the 3D acquisition of an archeological statue.     

Three-dimensional digitization of highly reflective and transparent objects using multi-wavelength range sensing

Digitizing specular and transparent objects pose significant problems using traditional 3-D scanning techniques due to the reflection and refraction that interfere with the optical scanning process used for triangulation. In this paper, we present how one can digitize those difficult objects by modifying a commercial 3-D acquisition system with an interchangeable ultraviolet and infrared light source. Experimental results show that the proposed technique can generate accurate 3-D models of these optically challenging objects without major modifications to the 3-D scanner. The results were obtained without preprocessing or multi-view manipulations. The precision of the 3-D measurements is evaluated relative to the visible spectrum acquisition obtained by painting the test objects with matte paint to suppress optical difficulties. Results shows that wavelength changes in the 3-D acquisition system do not change the scanner precision but solve many of the issues that specular and transparent objects poses.     

三维全身扫描仪的建模仿真

随着计算机全面进入3D时代，三维全身扫描系统对社会的价值越来越显著，获取人体表面的三维几何信息，是建立3D人体模型的先决条件。该文提出了一种基于结构光方法的三维全身扫描仪的系统仿真方案，使用3D Studio MAX软件仿真得到虚拟扫描头运动形成的带激光条纹的图像序列，最后用仿真数据进行三维重建。关键步骤包括用自适应阈值法提取激光带中心，深度图像的获取，以及多扫描头数据的融合等。实验取得了满意的效果，获得了人体的完整轮廓线，验证了系统仿真方案的合理性与高效性。     

Pattern codification strategies in structured light systems

Coded structured light is considered one of the most reliable techniques for recovering the surface of objects. This technique is based on projecting a light pattern and viewing the illuminated scene from one or more points of view. Since the pattern is coded, correspondences between image points and points of the projected pattern can be easily found. The decoded points can be triangulated and 3D information is obtained. We present an overview of the existing techniques, as well as a new and definitive classification of patterns for structured light sensors. We have implemented a set of representative techniques in this field and present some comparative results. The advantages and constraints of the different patterns are also discussed.     

Registration and integration of multiple object views for 3D modelconstruction

Automatic 3D object model construction is important in applications ranging from manufacturing to entertainment, since CAD models of existing objects may be either unavailable or unusable. We describe a prototype system for automatically registering and integrating multiple views of objects from range data. The results can then be used to construct geometric models of the objects. New techniques for handling key problems such as robust estimation of transformations relating multiple views and seamless integration of registered data to form an unbroken surface have been proposed and implemented in the system. Experimental results on real surface data acquired using a digital interferometric sensor as well as a laser range scanner demonstrate the good performance of our system     

A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus

Global or indirect illumination effects such as interreflections and subsurface scattering severely degrade the performance of structured light-based 3D scanning. In this paper, we analyze the errors in structured light, caused by both long-range (interreflections) and short-range (subsurface scattering) indirect illumination. The errors depend on the frequency of the projected patterns, and the nature of indirect illumination. In particular, we show that long-range effects cause decoding errors for low-frequency patterns, whereas short-range effects affect high-frequency patterns. Based on this analysis, we present a practical 3D scanning system which works in the presence of a broad range of indirect illumination. First, we design binary structured light patterns that are resilient to individual indirect illumination effects using simple logical operations and tools from combinatorial mathematics. Scenes exhibiting multiple phenomena are handled by combining results from a small ensemble of such patterns. This combination also allows detecting any residual errors that are corrected by acquiring a few additional images. Our methods can be readily incorporated into existing scanning systems without significant overhead in terms of capture time or hardware. We show results for several scenes with complex shape and material properties.     

化身1号:一个快速的三维彩色全身人像捕捉系统

提出一个快速的三维彩色全身人像捕捉系统（名为化身1号）：1.5秒采集数据,1分钟内全自动地建立三维全身人像.化身1号使用了8台自制的结构光扫描仪,采集数据的精度达到1毫米.这8个扫描仪预先配置好空间位置：4个抓拍头部,4个抓拍身体.从而,能够有效覆盖采集成人对象的全身区域.在离线标定完8个扫描仪后,化身1号就可以捕获真实对象,在一个坐标系内处理各扫描仪采集的局部数据,历经全局注册和无缝隙的纹理拼接,鲁棒地建立高逼真度的纹理人像模型.该模型的几何精度达到1毫米,并具有照片真实感的色彩信息.实验结果表明,在高逼真度建模需求的前提下,化身1号的采集和处理速度均处于当前最优水平.     

Using physically Based Rendering to Benchmark Structured Light Scanners

Structured light scanning is ubiquituous in 3D acquisition. It is capable of capturing high geometric detail at a low cost under a variety of challenging scene conditions. Recent methods have demonstrated robustness in the presence of artifacts due to global illumination, such as inter‐reflections and sub‐surface scattering, as well as imperfections caused by projector defocus. For comparing approaches, however, the quantitative evaluation of structured lighting schemes is hindered by the challenges in obtaining ground truth data, resulting in a poor understanding for these methods across a wide range of shapes, materials, and lighting configurations. In this paper, we present a benchmark to study the performance of structured lighting algorithms in the presence of errors caused due to the above properties of the scene. In order to do this, we construct a synthetic structured lighting scanner that uses advanced physically based rendering techniques to simulate the point cloud acquisition process. We show that, under conditions similar to that of a real scanner, our synthetic scanner replicates the same artifacts found in the output of a real scanner. Using this synthetic scanner, we perform a quantitative evaluation of four different structured lighting techniques – gray‐code patterns, micro‐phase shifting, ensemble codes, and unstructured light scanning. The evaluation, performed on a variety of scenes, demonstrate that no one method is capable of adequately handling all sources of error – each method is appropriate for addressing distinct sources of error.     

Fast Estimation and Rendering of Indirect Highlights

This paper proposes a method for efficiently rendering indirect highlights. Indirect highlights are caused by the primary light source reflecting off two or more glossy surfaces. Accurately simulating such highlights is important to convey the realistic appearance of materials such as chrome and shiny metal. Our method models the glossy BRDF at a surface point as a directional distribution, using a spherical von Mises‐Fisher (vMF) distribution. As our main contribution, we merge multiple vMFs into a combined multimodal distribution. This effectively creates a filtered radiance response function, allowing us to efficiently estimate indirect highlights. We demonstrate our method in a near‐interactive application for rendering scenes with highly glossy objects. Our results produce realistic reflections under both local and environment lighting.     

Multiview photometric stereo

This paper addresses the problem of obtaining complete, detailed reconstructions of textureless shiny objects. We present an algorithm which uses silhouettes of the object, as well as images obtained under changing illumination conditions. In contrast with previous photometric stereo techniques, ours is not limited to a single viewpoint but produces accurate reconstructions in full 3D. A number of images of the object are obtained from multiple viewpoints, under varying lighting conditions. Starting from the silhouettes, the algorithm recovers camera motion and constructs the object's visual hull. This is then used to recover the illumination and initialise a multi-view photometric stereo scheme to obtain a closed surface reconstruction. There are two main contributions in this paper: Firstly we describe a robust technique to estimate light directions and intensities and secondly, we introduce a novel formulation of photometric stereo which combines multiple viewpoints and hence allows closed surface reconstructions. The algorithm has been implemented as a practical model acquisition system. Here, a quantitative evaluation of the algorithm on synthetic data is presented together with complete reconstructions of challenging real objects. Finally, we show experimentally how even in the case of highly textured objects, this technique can greatly improve on correspondence-based multi-view stereo results.     

Hole filling of triangular mesh segments using systematic grey prediction

Recent advances in reverse engineering technology have made it possible to acquire accurate point information about model surfaces using 3D laser scanners. However, data capture can be reduced by limitations in the scanner structure or characteristics of the scanned objects, resulting in holes in the triangular mesh model and reducing the model’s quality. The current study seeks to fill these holes using software-based calculations and systematic grey prediction, thus preserving the original exterior surface of the reconstructed model. A 3D digital dental model was acquired using a 3D laser scanner, followed by hole detection and surface mending. Due to the simplicity and minimal dataset used in grey theory, a grey prediction system was built to perform prediction-adjustment to preserve the original dental exterior surface.     

A novel approach to documenting artifacts at the Gold Museum in Bogota

We present our work on novel digitizing techniques to create virtual exhibitions in the context of the Gold Museum in Bogotá, Colombia, a world renown space for precious pre-Colombian gold artifacts. In order to solve issues related to high specular reflection in gold artifacts, we developed a multi-spectral approach that solves some of the shortfalls of many commercial scanners. We also integrated commercial haptic devices into a new virtual installation that allows visitors to touch, hear, and see virtual approximations of the real objects. As an evaluation methodology, we compare the results of a scanner with normal light with our approach, and we also present results from user studies on our virtual installation.     

Extracting objects from range and radiance images

In this paper, we present a pipeline and several key techniques necessary for editing a real scene captured with both cameras and laser range scanners. We develop automatic algorithms to segment the geometry from range images into distinct surfaces, register texture from radiance images with the geometry, and synthesize compact high-quality texture maps. The result is an object-level representation of the scene which can be rendered with modifications to structure via traditional rendering methods. The segmentation algorithm for geometry operates directly on the point cloud from multiple registered 3D range images instead of a reconstructed mesh. It is a top-down algorithm which recursively partitions a point set into two subsets using a pairwise similarity measure. The result is a binary tree with individual surfaces as leaves. Our image registration technique performs a very efficient search to automatically find the camera poses for arbitrary position and orientation relative to the geometry. Thus, we can take photographs from any location without precalibration between the scanner and the camera. The algorithms have been applied to large-scale real data. We demonstrate our ability to edit a captured scene by moving, inserting, and deleting objects     

MusiDB: A personalized search engine for music

The increasing use of structured information on the web demands new ways of searching and integrating data from different sources. In this paper, we focus on the use of unique representations of data objects in terms of public repositories (in this case MusicBrainz) and the use of recommendation mechanisms as a basis for supporting information access. We have implemented a prototypical system with the corresponding functionality in the area of digital music. We discuss the challenges of providing integrated access to structured web resources and the solutions adopted in the MusiDB system.     

LexAGen: An interactive incremental scanner generator

This paper describes LexAGen, an interactive scanner generator which is the first component of an interactive compiler generation environment. LexAGen can generate fast scanners for languages whose tokens can be specified by regular grammars. However, LexAGen also supports several context-sensitive programming language constructs such as nested comments and the interaction between floating-point numbers and the range operator in Modula-2. In addition, LexAGen includes a fast new algorithm for keyword identification. However, the most important and novel aspects of LexAGen are that it constructs scanners incrementally and that specifications can be executed anytime for validation testing. LexAGen specifications are expressed and entered interactively in a restricted BNF format (no left recursion). All syntactic errors and token conflicts are detected and reported immediately as LexAGen incrementally constructs a deterministic finite automaton to represent the scanner. At any time, the user can test the scanner fragment which has been entered by supplying text to be scanned. Alternatively, the user can generate a C-code scanner from the automaton. The generated automaton uses a direct execution approach and is quite fast. LexAGen is implemented in Smalltalk-80. Its extensive use of interactive graphics makes it very easy to use. In addition, the object-oriented paradigm of Smalltalk-80 is the basis for the incremental analysis, the error detection scheme and an intermediate representation which can be easily modified to generate scanners in other target languages such as Pascal, Modula-2 and Ada.     

The 3D Model Acquisition Pipeline

Three-dimensional (3D) image acquisition systems are rapidly becoming more affordable, especially systems based on commodity electronic cameras. At the same time, personal computers with graphics hardware capable of displaying complex 3D models are also becoming inexpensive enough to be available to a large population. As a result, there is potentially an opportunity to consider new virtual reality applications as diverse as cultural heritage and retail sales that will allow people to view realistic 3D objects on home computers.
Although there are many physical techniques for acquiring 3D data—including laser scanners, structured light and time-of-flight—there is a basic pipeline of operations for taking the acquired data and producing a usable numerical model. We look at the fundamental problems of range image registration, line-of-sight errors, mesh integration, surface detail and color, and texture mapping. In the area of registration we consider both the problems of finding an initial global alignment using manual and automatic means, and refining this alignment with variations of the Iterative Closest Point methods. To account for scanner line-of-sight errors we compare several averaging approaches. In the area of mesh integration, that is finding a single mesh joining the data from all scans, we compare various methods for computing interpolating and approximating surfaces. We then look at various ways in which surface properties such as color (more properly, spectral reflectance) can be extracted from acquired imagery. Finally, we examine techniques for producing a final model representation that can be efficiently rendered using graphics hardware.     

Surface radiance correction for shape from shading

It is well known that many surfaces exhibit reflectance that is not well modelled by Lambert's law. This is the case not only for surfaces that are rough or shiny, but also those that are matte and composed of materials that are particle suspensions. As a result, standard Lambertian shape from shading methods cannot be applied directly to the analysis of rough and shiny surfaces. In order to overcome this difficulty, in this paper, we consider how to reconstruct the Lambertian component for rough and shiny surfaces when the object is illuminated in the viewing direction (retroreflection). To do this we make use of the diffuse reflectance models described by Oren and Nayar, and by Wolff. Our experiments with synthetic and real-world data reveal the effectiveness of the correction method, leading to improved surface normal and height recovery.     

Forestry crane posture estimation with a two‐dimensional laser scanner

Crane posture estimation is the stepping stone to forest machine automation. Here, we introduce a robust minimal perception solution, that is, one that uses minimal constraints for maximal benefits. Specifically, we introduce a robust particle‐filter‐based method to estimate and track the posture of a flexible hydraulic crane by using only low‐cost equipment, namely, a two‐dimensional (2D) laser scanner, two short magnetically attached metal tubes as targets, and an angle sensor. An important feature of our method is that it incorporates control signals for hydraulic actuators. In contrast to the previous works employing laser scanners, we do not use the full shape of the crane to estimate the crane posture, but, instead, we use only two small targets in the field of view of the laser scanner. Thus, a large share of the range data is useful for other purposes, for example, to map the surrounding environment. We test the proposed method in a challenging forest environment and show that the particle filter is able to estimate the posture of the hydraulic crane efficiently and reliably in the presence of occlusions and obstructions. During our comprehensive testing, the tip position was measured with average errors smaller than 4.3 cm whereas the absolute maximum error was 15 cm.     

Matting and Compositing for Fresnel Reflection on Wavy Surfaces

This paper introduces a framework that can extract an alpha matte from a single image with Fresnel reflection, and that can composite other objects with the image such that plausible reflections are included. Our method handles reflections in a plane with small undulations, for example, a water surface with waves or a glossy tabletop. During the matting stage, our method first estimates the transmission color, which is assumed to be uniform, and then calculates a reflection image and alpha matte based on user markups. However, accurate extraction of the matte becomes challenging when a plane has small undulations because these create perturbations in the matte. We therefore propose a filter that can refine the matte effectively. In the compositing stage, the reflection of a composited object is synthesized by ray tracing in real time. We demonstrate the effectiveness of our method through comparisons with ground‐truth data and results using natural images as inputs.