Camera–projector matching using unstructured video

This paper presents a novel approach for matching 2-D points between a video projector and a digital camera. Our method is motivated by camera–projector applications for which the projected image needs to be warped to prevent geometric distortion. Since the warping process often needs geometric information on the 3-D scene obtained from a triangulation, we propose a technique for matching points in the projector to points in the camera based on arbitrary video sequences. The novelty of our method lies in the fact that it does not require the use of pre-designed structured light patterns as is usually the case. The backbone of our application lies in a function that matches activity patterns instead of colors. This makes our method robust to pose, severe photometric and geometric distortions. It also does not require calibration of the color response curve of the camera–projector system. We present quantitative and qualitative results with synthetic and real-life examples, and compare the proposed method with the scale invariant feature transform (SIFT) method and with a state-of-the-art structured light technique. We show that our method performs almost as well as structured light methods and significantly outperforms SIFT when the contrast of the video captured by the camera is degraded.     

A state of the art in structured light patterns for surface profilometry

Shape reconstruction using coded structured light is considered one of the most reliable techniques to recover object surfaces. Having a calibrated projector-camera pair, a light pattern is projected onto the scene and imaged by the camera. Correspondences between projected and recovered patterns are found and used to extract 3D surface information. This paper presents an up-to-date review and a new classification of the existing techniques. Some of these techniques have been implemented and compared, obtaining both qualitative and quantitative results. The advantages and drawbacks of the different patterns and their potentials are discussed.     

Point Pattern Synthesis via Irregular Convolution

Point pattern synthesis is a fundamental tool with various applications in computer graphics. To synthesize a point pattern, some techniques have taken an example‐based approach, where the user provides a small exemplar of the target pattern. However, it remains challenging to synthesize patterns that faithfully capture the structures in the given exemplar. In this paper, we present a new example‐based point pattern synthesis method that preserves both local and non‐local structures present in the exemplar. Our method leverages recent neural texture synthesis techniques that have proven effective in synthesizing structured textures. The network that we present is end‐to‐end. It utilizes an irregular convolution layer, which converts a point pattern into a gridded feature map, to directly optimize point coordinates. The synthesis is then performed by matching inter‐ and intra‐correlations of the responses produced by subsequent convolution layers. We demonstrate that our point pattern synthesis qualitatively outperforms state‐of‐the‐art methods on challenging structured patterns, and enables various graphical applications, such as object placement in natural scenes, creative element patterns or realistic urban layouts in a 3D virtual environment.     

基于最右扩展枚举的半结构化数据最大模式挖掘方法研究

本文以标记有序树作为半结构化数据的数据模型 ,研究了半结构化数据的树状最大频繁模式挖掘问题 .已有挖掘算法通常挖掘所有频繁模式 ,其中很多模式为其它模式的子模式 ,针对该问题 ,设计实现了一种最大模式挖掘算法 .该算法采用最右扩展枚举方法无重复枚举所有候选模式 ,利用频繁模式扩展森林实现高效剪枝扩展和挖掘频繁叶模式 ,通过计算频繁叶模式间的包含关系挖掘树状最大频繁模式 .试验结果表明该算法具有良好性能     

基于格雷码和多步相移法的双目立体视觉三维测量技术研究

传统的被动式双目立体视觉三维测量技术,具有操作简单,使用灵活方便,相机标定技术成熟的优点,但是对于特征点稀疏图像,寻找匹配点困难,匹配精度低。编码结构光测量方式通过向待测物体投射特定的编码图案,获取编码图像进行解码求解物体的三维信息,具有着测量精度高,速度快的优点,但是存在着投影仪标定精度低,实现难度大的缺点。提出了将双目立体视觉和编码结构光相结合的三维测量方法,在完成双目校正的基础上,向待测物体投射格雷码图案和多步相移图案,给予被测物体容易识别和可控制的特征信息,最后求取物体的三维信息。而且通过实验论证了投射多步相移图案比起4步相移图案,测量精度更高,能够更好的体现物体细节。     

A log-linear approach to mining significant graph-relational patterns

Objects in many application domains can be characterized as link-based data, having both network (graph) information as well as structured information describing the nodes. Discovery of frequent patterns in this setting is vulnerable to problems that cannot occur in pattern mining on conventional data without network information. While patterns may appear to reflect novel characteristics of a combination of graph and node information, they may be expected based on patterns that could be found using conventional data mining techniques. We introduce a significance measure that identifies patterns that are unexpected based on node attributes in isolation and neighbor correlations. A statistical log-linear model is extended for this purpose and the structural symmetry of the link-based data is accounted for. Eliminating insignificant results reduces the output quantity by orders of magnitude. Efficiency is achieved by designing the pattern mining algorithm as a hybrid of conventional pattern mining and graph data mining. We demonstrate effectiveness and efficiency of the approach for yeast and for movie data.     

Auto‐calibration of a projector–camera stereo system for projection mapping

Standard camera and projector calibration techniques use a checkerboard that is manually shown at different poses to determine the calibration parameters. Furthermore, when image geometric correction must be performed on a three‐dimensional (3D) surface, such as projection mapping, the surface geometry must be determined. Camera calibration and 3D surface estimation can be costly, error prone, and time‐consuming when performed manually. To address this issue, we use an auto‐calibration technique that projects a series of Gray code structured light patterns. These patterns are captured by the camera to build a dense pixel correspondence between the projector and camera, which are used to calibrate the stereo system using an objective function, which embeds the calibration parameters together with the undistorted points. Minimization is carried out by a greedy algorithm that minimizes the cost at each iteration with respect to both calibration parameters and noisy image points. We test the auto‐calibration on different scenes and show that the results closely match a manual calibration of the system. We show that this technique can be used to build a 3D model of the scene, which in turn with the dense pixel correspondence can be used for geometric screen correction on any arbitrary surface.     

Prototype learning for structured pattern representation applied to on-line recognition of handwritten Japanese characters

This paper describes prototype learning for structured pattern representation with common subpatterns shared among multiple character prototypes for on-line recognition of handwritten Japanese characters. Prototype learning algorithms have not yet been shown to be useful for structured or hierarchical pattern representation. In this paper, we incorporate cost-free parallel translation to negate the location distributions of subpatterns when they are embedded in character patterns. Moreover, we introduce normalization into a prototype learning algorithm to extract true feature distributions in raw patterns to aggregate distributions of feature points to subpattern prototypes. We show that our proposed method significantly improves structured pattern representation for Japanese on-line character patterns.     

Range imaging with adaptive color structured light

In range sensing with time-multiplexed structured light, there is a trade-off between accuracy, robustness and the acquisition period. In this paper a novel structured light method is described. Adaptation of the number and form of the projection patterns to the characteristics of the scene takes place as part of the acquisition process. Noise margins are matched to the actual noise level, thus reducing the number of projection patterns to the necessary minimum. Color is used for light plane labeling. The dimension of the pattern space are thus increased without raising the number of projection patterns. It is shown that the color of an impinging light plane can be identified from the image of the illuminated scene, even with colorful scenes. Identification is local and does not rely on spatial color sequences. The suggested approach has been implemented and the theoretical results are supported by experiments     

The use of a uniquely encoded light pattern for range data acquisition

Building a three-dimensional vision system requires three-dimensional scene information in the form of range data. Using a structured light system accompanied by the triangulation method is a typical approach to obtain such data. In this paper, we present a ranging method using an encoded grid light pattern. Our method is different from the previous approaches in such a way that the codes are unique for any grid intersection points and maximal for a given set of code primitives. An efficient data acquisition procedure is obtained by applying the uniqueness of a code. The encoding scheme for the light pattern and the decoding scheme for the imaged illuminated pattern are presented. We also discuss a fault-tolerant approach which makes the ranging system robust.     

Using physically Based Rendering to Benchmark Structured Light Scanners

Structured light scanning is ubiquituous in 3D acquisition. It is capable of capturing high geometric detail at a low cost under a variety of challenging scene conditions. Recent methods have demonstrated robustness in the presence of artifacts due to global illumination, such as inter‐reflections and sub‐surface scattering, as well as imperfections caused by projector defocus. For comparing approaches, however, the quantitative evaluation of structured lighting schemes is hindered by the challenges in obtaining ground truth data, resulting in a poor understanding for these methods across a wide range of shapes, materials, and lighting configurations. In this paper, we present a benchmark to study the performance of structured lighting algorithms in the presence of errors caused due to the above properties of the scene. In order to do this, we construct a synthetic structured lighting scanner that uses advanced physically based rendering techniques to simulate the point cloud acquisition process. We show that, under conditions similar to that of a real scanner, our synthetic scanner replicates the same artifacts found in the output of a real scanner. Using this synthetic scanner, we perform a quantitative evaluation of four different structured lighting techniques – gray‐code patterns, micro‐phase shifting, ensemble codes, and unstructured light scanning. The evaluation, performed on a variety of scenes, demonstrate that no one method is capable of adequately handling all sources of error – each method is appropriate for addressing distinct sources of error.     

面向交叉点网络的符号解码方法的研究

计算机视觉是近年来研究的热点,结构光视觉是双目计算机视觉的一个分支,该技术采用一台投射器代替双目立体视觉的一台摄像机,由于其投射的结构光具有编码唯一性的特点,从而解决了双目视觉中像素匹配的难题。结构光技术中解码成为了一项重要的工作,根据符号的特点提出一种基于交叉点网络的符号解码方法。该方法有效地实现了交叉点的提取,并形成全局的交叉点网络。针对细化的十字交叉变形,研究了交叉点合并方法,提出了基于识圈法的内部交叉点定位方法,以及基于向量角度的符号分类方法,有效地实现了符号码字的还原。最终实现物体的三维重建目标。     

Calibrating a Structured Light Stripe System: A Novel Approach

The problem associated with calibrating a structured light stripe system is that known world points on the calibration target do not normally fall onto every light stripe plane illuminated from the projector. We present in this paper a novel calibration method that employs the invariance of the cross ratio to overcome this problem. Using 4 known non-coplanar sets of 3 collinear world points and with no prior knowledge of the perspective projection matrix of the camera, we show that world points lying on each light stripe plane can be computed. Furthermore, by incorporating the homography between the light stripe and image planes, the 4 × 3 image-to-world transformation matrix for each stripe plane can also be recovered. The experiments conducted suggest that this novel calibration method is robust, economical, and is applicable to many dense shape reconstruction tasks.     

A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus

Global or indirect illumination effects such as interreflections and subsurface scattering severely degrade the performance of structured light-based 3D scanning. In this paper, we analyze the errors in structured light, caused by both long-range (interreflections) and short-range (subsurface scattering) indirect illumination. The errors depend on the frequency of the projected patterns, and the nature of indirect illumination. In particular, we show that long-range effects cause decoding errors for low-frequency patterns, whereas short-range effects affect high-frequency patterns. Based on this analysis, we present a practical 3D scanning system which works in the presence of a broad range of indirect illumination. First, we design binary structured light patterns that are resilient to individual indirect illumination effects using simple logical operations and tools from combinatorial mathematics. Scenes exhibiting multiple phenomena are handled by combining results from a small ensemble of such patterns. This combination also allows detecting any residual errors that are corrected by acquiring a few additional images. Our methods can be readily incorporated into existing scanning systems without significant overhead in terms of capture time or hardware. We show results for several scenes with complex shape and material properties.     

Real-time structured light coding for adaptive patterns

Coded structured light is a technique that allows the 3D reconstruction of poorly or non-textured scene areas. With the codes uniquely associated with visual primitives of the projected pattern, the correspondence problem is quickly solved by means of local information only, with robustness against disturbances like high surface curvatures, partial occlusions, out-of-field of view or out-of-focus. Real-time 3D reconstruction with one shot is possible with pseudo-random arrays, where the encoding is done in a single pattern using spatial neighbourhood. To correct more mismatched visual primitives and to get patterns globally more robust, a higher Hamming distance between all the used codewords should be suited. Recent works in the structured light field have shown a growing interest for adaptive patterns. These can account for geometrical or spectral specificities of the scene to provide better features matching and reconstructions. Up till today, such patterns cannot benefit from the robustness offered by spatial neighbourhood coding with a minimal Hamming distance constraint, because the existing algorithms for such a class of coding are designed with an offline coding only. In this article, we show that due to two new contributions, a mixed exploration/exploitation search behaviour and a O(n ²) to ～O(n) complexity reduction using the epipolar constraint, the real-time coding of patterns having similar properties than those coded offline can be achieved. This allows to design a complete closed-loop processing pipeline for adaptive patterns.     

基于二值几何编码图案的高精度结构光系统参数标定方法研究

该文提出了一种基于空间编码图案的结构光系统参数标定方法。与传统的基于棋盘格图案标 定策略不同的是,该文采用结构光编码图案实现了系统的高精度标定,具体实施步骤包括：(1)根据编码图案的几何分布特性提出了一种编码特征点检测算子,基于检测出的编码特征点构建拓扑结构,利 用仿射变换原理及双线性插值算法提取出编码几何元素图像;(2)将几何元素识别转化为监督分类问题,通过采集大量训练样本训练卷积神经网络,实现编码元素的准确识别和解码过程;(3)利用射影变 换原理建立相机像平面与投影机像平面之间的对应关系,利用此对应关系将标定板上棋盘格角点在相机像平面上的坐标转换至投影机像平面,最终实现了对相机和投影仪内外部参数的同时标定。标定结果显示,该方法对投影仪的标定重投影误差不超过 0.3 像素;三维重建实验结果显示,与传统标定方法相比,该文方法能够显著提升系统的标定和三维重建精度。     

Looking into the seeds of time: Discovering temporal patterns in large transaction sets

This paper studies the problem of mining frequent itemsets along with their temporal patterns from large transaction sets. A model is proposed in which users define a large set of temporal patterns that are interesting or meaningful to them. A temporal pattern defines the set of time points where the user expects a discovered itemset to be frequent. The model is general in that (i) no constraints are placed on the interesting patterns given by the users, and (ii) two measures—inclusiveness and exclusiveness—are used to capture how well the temporal patterns match the time points given by the discovered itemsets. Intuitively, these measures indicate to what extent a discovered itemset is frequent at time points included in a temporal pattern p, but not at time points not in p. Using these two measures, one is able to model many temporal data mining problems appeared in the literature, as well as those that have not been studied. By exploiting the relationship within and between itemset space and pattern space simultaneously, a series of pruning techniques are developed to speed up the mining process. Experiments show that these pruning techniques allow one to obtain performance benefits up to 100 times over a direct extension of non-temporal data mining algorithms.     

Surround structured lighting: 3-D scanning with orthographic illumination

This paper presents a new system for rapidly acquiring complete 3-D surface models using a single orthographic structured light projector, a pair of planar mirrors, and one or more synchronized cameras. Using the mirrors, we project structured light patterns that illuminate the object from all sides (not just the side of the projector) and are able to observe the object from several vantage points simultaneously. This system requires that projected planes of light to be parallel, so we construct an orthographic projector using a Fresnel lens and a commercial DLP projector. A single Gray code sequence is used to encode a set of vertically-spaced light planes within the scanning volume, and five views of the illuminated object are obtained from a single image of the planar mirrors located behind it. From each real and virtual camera we recover a dense 3-D point cloud spanning the entire object surface using traditional structured light algorithms. A key benefit of this design is to ensure that each point on the object surface can be assigned an unambiguous Gray code sequence, despite the possibility of being illuminated from multiple directions. In addition to presenting a prototype implementation, we also develop a complete set of mechanical alignment and calibration procedures for utilizing orthographic projectors in computer vision applications. As we demonstrate, the proposed system overcomes a major hurdle to achieving full 360° reconstructions using a single structured light sequence by eliminating the need for merging multiple scans or multiplexing several projectors.     

Finding maximal homogeneous clique sets

Many datasets can be encoded as graphs with sets of labels associated with the vertices. We consider this kind of graphs and we propose to look for patterns called maximal homogeneous clique sets, where such a pattern is a subgraph that is structured in several large cliques and where all vertices share enough labels. We present an algorithm based on graph enumeration to compute all patterns satisfying user-defined constraints on the number of separated cliques, on the size of these cliques, and on the number of labels shared by all the vertices. Our approach is tested on real datasets based on a social network of scientific collaborations and on a biological network of protein–protein interactions. The experiments show that the patterns are useful to exhibit subgraphs organized in several core modules of interactions. Performances are reported on real data and also on synthetic ones, showing that the approach can be applied on different kinds of large datasets.