Pattern codification strategies in structured light systems

Coded structured light is considered one of the most reliable techniques for recovering the surface of objects. This technique is based on projecting a light pattern and viewing the illuminated scene from one or more points of view. Since the pattern is coded, correspondences between image points and points of the projected pattern can be easily found. The decoded points can be triangulated and 3D information is obtained. We present an overview of the existing techniques, as well as a new and definitive classification of patterns for structured light sensors. We have implemented a set of representative techniques in this field and present some comparative results. The advantages and constraints of the different patterns are also discussed.     

Computation of surface orientation and structure of objects using grid coding

In this correspondence, algorithms are introduced to infer surface orientation and structure of visible object surfaces using grid coding. We adopt the active lighting technique to spatially ``encode' the scene for analysis. The observed objects, which can have surfaces of arbitrary shape, are assumed to rest on a plane (base plane) in a scene which is ``encoded' with light cast through a grid plane. Two orthogonal grid patterns are used, where each pattern is obtained with a set of equally spaced stripes marked on a glass pane. The scene is observed through a camera and the object surface orientation is determined using the projected patterns on the object surface. If the surfaces under consideration obey certain smoothness constraints, a dense orientation map can be obtained through proper interpolation. The surface structure can then be recovered given this dense orientation map. Both planar and curved surfaces can be handled in a uniform manner. The algorithms we propose yield reasonably accurate results and are relatively tolerant to noise, especially when compared to shape-from-shading techniques. In contrast to other grid coding techniques reported which match the grid junctions for depth reconstruction under the stereopsis principle, our techniques use the direction of the projected stripes to infer local surface orientation and do not require any correspondence relationship between either the grid lines or the grid junctions to be specified. The algorithm has the ability to register images and can therefore be embedded in a system which integrates knowledge from multiple views.     

A modified fuzzy C-means image segmentation algorithm for use with uneven illumination patterns

A novel fuzzy C-mean (FCM) algorithm is proposed for use when active or structured light patterns are projected onto a scene. The underlying inhomogeneous illumination intensity due to the point source nature of the projection, surface orientation and curvature has been estimated and its effect on the object segmentation minimized. Firstly, we modified the recursive FCM algorithm to include biased illumination field estimation. New clustering center and fuzzy clustering functions resulted based on the intensity and average intensity of a pixel neighborhood based object function. Finally, a dilation operator was used on the initial segmented image for further refinement. Experimental results showed the proposed method was effective for segmenting images illuminated by patterns containing underlying biased intensity fields. A higher accuracy was obtained than for traditional FCM and thresholding techniques.     

Robust active stereo vision using Kullback-Leibler divergence

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.     

Stable and fast techniques for unambiguous compound phase coding

Phase shift methods have proven to be very robust and accurate for photometric 3D reconstruction. One problem of these approaches is the existence of ambiguities arising from the periodicity of the fringe patterns. While several techniques for disambiguation exist, all of them require the projection of a significant number of additional patterns. For instance, a global Gray coding sequence or supplemental sinusoidal patterns of different periods are commonly used to complement the basic phase shift technique. In this paper we propose four new coding strategies that encode the index of the projected column using several phases and that mix the resulting phases into a controllable number of projected patterns from which the position can be recovered with subpixel precision. One notable characteristic of the proposed approaches is that we can allocate the additional number of patterns specifically to improve precision or provide higher robustness to noise. The proposed approaches are analyzed and compared with the state of the art, showing their ability to be tuned towards high precision in low noise conditions or robustness with respect to noise.     

Chip making to dye for

A University of Washington researcher is using common food dyes to improve photolithography, the dominant semiconductor microfabrication technology. The microfluidic masks are made by filling transparent silicone channels with red dye, which blocks UV light, to create the desired patterns to be projected onto the photoresist.     

Application of grayscale expansion for accuracy improvement in phase-measuring profilometry

A method based on grayscale expansion for improving measurement accuracy in phase-measuring profilometry (PMP) is proposed. Digital light projector (DLP) has been widely used for projecting phase-shifting sinusoidal grating in PMP. However, due to its international standardization, DLP can only project at most 256 grayscales for all gray patterns. Moreover, the gray digitalization error may cause lower fidelity for the projected sinusoidal grating. In order to reduce the gray digitalization error, the sinusoidal grating can thus be designed as a 10-bit static pattern with 766 grayscales at most and then is split into three 8-bit static patterns to synthesize a repeated video. While this repeated video, rather than the static pattern, is projected onto the measured object by strictly controlling CCD camera integral time to be an integral multiple of three video refresh cycles, the captured deformed patterns are of 766 grayscales and can provide much more detailed information in phase calculation than those with 256 grayscales. Experimental results demonstrate the feasibility and validity of the proposed method. Both the measuring precision and the repeatability of PMP are improved efficiently.     

Subspace and projected clustering: experimental evaluation and analysis

Subspace and projected clustering have emerged as a possible solution to the challenges associated with clustering in high-dimensional data. Numerous subspace and projected clustering techniques have been proposed in the literature. A comprehensive evaluation of their advantages and disadvantages is urgently needed. In this paper, we evaluate systematically state-of-the-art subspace and projected clustering techniques under a wide range of experimental settings. We discuss the observed performance of the compared techniques, and we make recommendations regarding what type of techniques are suitable for what kind of problems.     

Optically induced electrohydrodynamic instability-based micro-patterning of fluidic thin films

Projected light patterns are used to induce electrohydrodynamic instabilities in a polymer thin film sandwiched between two electrodes. Using this optically induced electrohydrodynamic instability (OEHI) phenomenon, we have successfully demonstrated rapid, microscale patterning of polydimethylsiloxane (PDMS) pillar arrays on a thin-film hydrogenated amorphous silicon layer on top of an indium titanium oxide glass substrate. This glass substrate is the bottom electrode in a two-electrode, parallel-plate capacitor configuration with a micron-scale gap. Within this gap are a thin film of spin-coated PDMS and a thin layer of air. Primary pillar growth is first observed within 5–90 s in the dark regions of the projected patterns and pillar growth eventually spreads to the illuminated regions when the initial PDMS thickness is <2 μm. Experimental data characterizing the change in pillar diameters (between 15 and 30 μm in diameter) show that they can be decoupled from the inter-pillar spacing (maintaining a constant ~84 μm pitch between pillar centers) by controlling the applied DC voltage (between 110 and 210 V). Experimental results also show the importance of the optically induced lateral electric field on controlling pillar formation. This OEHI method of rapid pillar generation, with voltage control of the pillar diameter and control of pillar position via projected light patterns, presents new opportunities for low cost, efficient, and simple fabrication of micro, and perhaps nanoscale, polymer structures that could be used in many bioMEMS applications.     

Material light: exploring expressive materials

The control of material appearance has become richer than before, giving designers new expressive freedom. Designers need tools and techniques to handle this freedom when designing products. We present a simple but powerful technique to explore material expression in the conceptual phase of the design process. Colours and patterns are projected on foam and paper models to enable designers to quickly visualise and judge materials in context of the products shape. This paper is part of the 3AD design colloquium creative connections.     

A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus

Global or indirect illumination effects such as interreflections and subsurface scattering severely degrade the performance of structured light-based 3D scanning. In this paper, we analyze the errors in structured light, caused by both long-range (interreflections) and short-range (subsurface scattering) indirect illumination. The errors depend on the frequency of the projected patterns, and the nature of indirect illumination. In particular, we show that long-range effects cause decoding errors for low-frequency patterns, whereas short-range effects affect high-frequency patterns. Based on this analysis, we present a practical 3D scanning system which works in the presence of a broad range of indirect illumination. First, we design binary structured light patterns that are resilient to individual indirect illumination effects using simple logical operations and tools from combinatorial mathematics. Scenes exhibiting multiple phenomena are handled by combining results from a small ensemble of such patterns. This combination also allows detecting any residual errors that are corrected by acquiring a few additional images. Our methods can be readily incorporated into existing scanning systems without significant overhead in terms of capture time or hardware. We show results for several scenes with complex shape and material properties.     

A review and evaluation of methods estimating ego-motion

If a visual observer moves through an environment, the patterns of light that impinge its retina vary leading to changes in sensed brightness. Spatial shifts of brightness patterns in the 2D image over time are called optic flow. In contrast to optic flow visual motion fields denote the displacement of 3D scene points projected onto the camera’s sensor surface. For translational and rotational movement through a rigid scene parametric models of visual motion fields have been defined. Besides ego-motion these models provide access to relative depth, and both ego-motion and depth information is useful for visual navigation.In the past 30 years methods for ego-motion estimation based on models of visual motion fields have been developed. In this review we identify five core optimization constraints which are used by 13 methods together with different optimization techniques.¹ In the literature methods for ego-motion estimation typically have been evaluated by using an error measure which tests only a specific ego-motion. Furthermore, most simulation studies used only a Gaussian noise model. Unlike, we test multiple types and instances of ego-motion. One type is a fixating ego-motion, another type is a curve-linear ego-motion. Based on simulations we study properties like statistical bias, consistency, variability of depths, and the robustness of the methods with respect to a Gaussian or outlier noise model. In order to achieve an improvement of estimates for noisy visual motion fields, part of the 13 methods are combined with techniques for robust estimation like m-functions or RANSAC. Furthermore, a realistic scenario of a stereo image sequence has been generated and used to evaluate methods of ego-motion estimation provided by estimated optic flow and depth information.     

Two-Part Texture Mappings

Most published techniques for mapping two-dimensional texture patterns onto three-dimensional curved surfaces assume that either the texture pattern has been predistorted to compensate for the distortion of the mapping or the curved surfaces are represented parametrically. We address the problem of mapping undistorted planar textures onto arbitrarily represented surfaces. Our mapping technique is done in two parts. First the texture pattern is embedded in 3-space on an intermediate surface. Then the pattern is projected onto the target surface in a way that depends only on the geometry of the target object (not on its parameterization). Both steps have relatively low distortion, so the original texture need not be predistorted. We also discuss interactive techniques that make two-part mapping practical.     

Self‐calibration three‐dimensional light field display based on scalable multi‐LCDs

Approach to achieve self‐calibration three‐dimensional (3D) light field display is investigated in this paper. The proposed 3D light field display is constructed up on spliced multi‐LCDs, lens and diaphragm arrays, and directional diffuser. The light field imaging principle, hardware configuration, diffuser characteristic, and image reconstruction simulation are described and analyzed, respectively. Besides the light field imaging, a self‐calibration method is proposed to improve the imaging performance. An image sensor is deployed to capture calibration patterns projected onto and then reflected by the polymer dispersed liquid crystal film, which is attached to and shaped the diffuser. These calibration components are assembled with the display unit and can be switched between display mode and calibration mode. In the calibration mode, the imperfect imaging relations of optical components are captured and calibrated automatically. We demonstrate our design by implementing the prototype of proposed 3D light field display by using modified off‐the‐shelf products. The proposed approach successfully meets the requirement of real application on scalable configuration, fast calibration, large viewing angular range, and smooth motion parallax.     

Mining sequential patterns by pattern-growth: the PrefixSpan approach

Sequential pattern mining is an important data mining problem with broad applications. However, it is also a difficult problem since the mining may have to generate or examine a combinatorially explosive number of intermediate subsequences. Most of the previously developed sequential pattern mining methods, such as GSP, explore a candidate generation-and-test approach [R. Agrawal et al. (1994)] to reduce the number of candidates to be examined. However, this approach may not be efficient in mining large sequence databases having numerous patterns and/or long patterns. In this paper, we propose a projection-based, sequential pattern-growth approach for efficient mining of sequential patterns. In this approach, a sequence database is recursively projected into a set of smaller projected databases, and sequential patterns are grown in each projected database by exploring only locally frequent fragments. Based on an initial study of the pattern growth-based sequential pattern mining, FreeSpan [J. Han et al. (2000)], we propose a more efficient method, called PSP, which offers ordered growth and reduced projected databases. To further improve the performance, a pseudoprojection technique is developed in PrefixSpan. A comprehensive performance study shows that PrefixSpan, in most cases, outperforms the a priori-based algorithm GSP, FreeSpan, and SPADE [M. Zaki, (2001)] (a sequential pattern mining algorithm that adopts vertical data format), and PrefixSpan integrated with pseudoprojection is the fastest among all the tested algorithms. Furthermore, this mining methodology can be extended to mining sequential patterns with user-specified constraints. The high promise of the pattern-growth approach may lead to its further extension toward efficient mining of other kinds of frequent patterns, such as frequent substructures.     

Manipulation and patterning of carbon nanotubes utilizing optically induced dielectrophoretic forces

This study reports an optically driven platform upon which the manipulation and patterning of carbon nanotubes (CNTs) can be accomplished. A photoconductive layer made of amorphous silicon generates a nonuniform electric field within the developed platform at specific optically illuminated sites, which are usually referred to as “virtual electrodes,” that induces dielectrophoretic forces for manipulating the CNTs. The software-controlled light patterns enable a variety of flexible manipulation modes since it is now possible to dynamically reconfigure the optically projected electrode patterns. This approach allows for real-time manipulation and miscellaneous patterning of CNTs. The sorting and separation of bundled and dispersed CNTs is also demonstrated. This developed platform may be promising for rapid fabrication of CNT-based nanosensors together with nanoelectronics, purification as well as classification of synthesized CNTs and other applications requiring nanoscale manipulation.     

Iterative projected clustering by subspace mining

Irrelevant attributes add noise to high-dimensional clusters and render traditional clustering techniques inappropriate. Recently, several algorithms that discover projected clusters and their associated subspaces have been proposed. We realize the analogy between mining frequent itemsets and discovering dense projected clusters around random points. Based on this, we propose a technique that improves the efficiency of a projected clustering algorithm (DOC). Our method is an optimized adaptation of the frequent pattern tree growth method used for mining frequent itemsets. We propose several techniques that employ the branch and bound paradigm to efficiently discover the projected clusters. An experimental study with synthetic and real data demonstrates that our technique significantly improves on the accuracy and speed of previous techniques.     

Unstructured Light Scanning Robust to Indirect Illumination and Depth Discontinuities

Reconstruction from structured light can be greatly affected by indirect illumination such as interreflections between surfaces in the scene and sub-surface scattering. This paper introduces band-pass white noise patterns designed specifically to reduce the effects of indirect illumination, and still be robust to standard challenges in scanning systems such as scene depth discontinuities, defocus and low camera-projector pixel ratio. While this approach uses unstructured light patterns that increase the number of required projected images, it is up to our knowledge the first method that is able to recover scene disparities in the presence of both indirect illumination and scene discontinuities. Furthermore, the method does not require calibration (geometric nor photometric) or post-processing such as phase unwrapping or interpolation from sparse correspondences. We show results for a few challenging scenes and compare them to correspondences obtained with the Phase-shift method and the recently introduced method by Gupta et al., designed specifically to handle indirect illumination.     

三维视觉测量技术及应用进展

三维视觉测量是计算机视觉与精密测量原理交叉融合的前沿高新技术,是工业4.0的基础支撑,是以网络化、智能化制造为变革特征的先进制造业的核心关键技术。经过几十年的发展,三维视觉测量技术在基础研究和应用研究上均获得了快速深入发展,形成了理论方法、技术工艺、系统研发和产品应用四位一体较为完备的方向体系,呈现出理论系统化、方法多维化、精度精准化和速度快捷化的发展趋势,成为智能制造过程控制、产品质量检验保证和装备整机服役测试的不可或缺的优选技术。本文主要围绕单相机、双相机和结构光等典型三维视觉测量技术展开论述,概要介绍其关键技术内涵,综述其发展现状、前沿动态、热点问题和发展趋势。重点论述条纹投影三维测量技术和相位测量偏折术。最后给出了三维视觉测量的发展趋势与未来展望。