Using geometric constraints through parallelepipeds for calibration and 3D modeling

This paper concerns the incorporation of geometric information in camera calibration and 3D modeling. Using geometric constraints enables more stable results and allows us to perform tasks with fewer images. Our approach is motivated and developed within a framework of semi-automatic 3D modeling, where the user defines geometric primitives and constraints between them. In this paper, first a duality that exists between the shape parameters of a parallelepiped and the intrinsic parameters of a camera is described. Then, a factorization-based algorithm exploiting this relation is developed. Using images of parallelepipeds, it allows us to simultaneously calibrate cameras, recover shapes of parallelepipeds, and estimate the relative pose of all entities. Besides geometric constraints expressed via parallelepipeds, our approach simultaneously takes into account the usual self-calibration constraints on cameras. The proposed algorithm is completed by a study of the singular cases of the calibration method. A complete method for the reconstruction of scene primitives that are not modeled by parallelepipeds is also briefly described. The proposed methods are validated by various experiments with real and simulated data, for single-view as well as multiview cases.     

Multiple motion scene reconstruction with uncalibrated cameras

In this paper, we describe a reconstruction method for multiple motion scenes, which are scenes containing multiple moving objects, from uncalibrated views. Assuming that the objects are moving with constant velocities, the method recovers the scene structure, the trajectories of the moving objects, the camera motion, and the camera intrinsic parameters (except skews) simultaneously. We focus on the case where the cameras have unknown and varying focal lengths while the other intrinsic parameters are known. The number of the moving objects is automatically detected without prior motion segmentation. The method is based on a unified geometrical representation of the static scene and the moving objects. It first performs a projective reconstruction using a bilinear factorization algorithm and, then, converts the projective solution to a Euclidean one by enforcing metric constraints. Experimental results on synthetic and real images are presented.     

Interactive optimization of 3D shape and 2D correspondence usingmultiple geometric constraints via POCS

The traditional approach of handling motion tracking and structure from motion (SFM) independently in successive steps exhibits inherent limitations in terms of achievable precision and incorporation of prior geometric constraints about the scene. This paper proposes a projections onto convex sets (POCS) framework for iterative refinement of the measurement matrix in the well-known factorization method to incorporate multiple geometric constraints about the scene, thereby improving the accuracy of both 2D feature point tracking and 3D structure estimates. Regularities in the scene, such as points on line and plane and parallel lines and planes, that can be interactively identified and marked at each POCS iteration, enforce rank and parallelism constraints on appropriately defined local measurement matrices, one for each constraint. The POCS framework allows for the integration of the information in each of these local measurement matrices into a single measurement matrix that is "closest" to the initial observed measurement matrix in Frobenius norm, which is then factored in the usual manner. Experimental results demonstrate that the proposed interactive POCS framework consistently improves both 2D correspondences and 3D shape/motion estimates and similar results cannot be achieved by enforcing these constraints as either post or preprocessing     

Tracking 3-D motion from straight lines with trifocal tensors

We present a novel approach to track the position and orientation of a stereo camera using line features in the images. The method combines the strengths of trifocal tensors and Bayesian filtering. The trifocal tensor provides a geometric constraint to lock line features among every three frames. It eliminates the explicit reconstruction of the scene even if the 3-D scene structure is not known. Such a trifocal constraint thus makes the algorithm fast and robust. The twist motion model is applied to further improve its computation efficiency. Another major contribution is that our approach can obtain the 3-D camera motion using as little as 2 line correspondences instead of 13 in the traditional approaches. This makes the approach attractive for realistic applications. The performance of the proposed method has been evaluated using both synthetic and real data with encouraging results. Our algorithm is able to estimate 3-D camera motion in real scenarios accurately having little drifting from an image sequence longer than a 1,000 frames.     

Singularity theoretical modeling and animation of garment wrinkle formation processes

To understand the nature of garments as worn, it is essential to model and animate the formation process of garment wrinkles. Because the number of components making up a garment is extremely high, simulating its behavior under dynamic constraints requires a very large amount of computation, and the result is difficult to analyze and understand. We show that exploiting geometric features of wrinkles can greatly increase the understandability of the computed result, while not much increasing the amount of computation needed. We present the modeling primitives of garment wrinkles, which can suitably represent geometric features of wrinkles under the dynamic constraints. Extracting geometric features based on singularity theory enables us to model the qualitative shape change of wrinkles. The formation process of wrinkles is animated by using these primitives.     

Automatic geometric reasoning in structure and motion estimation

We present a system for doing automatic surveying or structure and motion analysis given 1D images of a 2D surrounding. Nothing is known about the structure of the scene features or of the motion of the camera. The system automatically identifies and tracks the image of new points and solves the structure and motion problem. One key feature of the system is the ability to hypothesize, test and incorporate simple constraints on the scene, e.g. that two object points are the same, that several points are coplanar. In this paper we develop and test the theory for automatic geometric reasoning. Ideas on hypothesis generation and testing are presented. It is also shown how to update the uncertainty representation of the database.     

3-D Scene Data Recovery Using Omnidirectional Multibaseline Stereo

A traditional approach to extracting geometric information from a large scene is to compute multiple 3-D depth maps from stereo pairs or direct range finders, and then to merge the 3-D data. However, the resulting merged depth maps may be subject to merging errors if the relative poses between depth maps are not known exactly. In addition, the 3-D data may also have to be resampled before merging, which adds additional complexity and potential sources of errors.This paper provides a means of directly extracting 3-D data covering a very wide field of view, thus by-passing the need for numerous depth map merging. In our work, cylindrical images are first composited from sequences of images taken while the camera is rotated 360° about a vertical axis. By taking such image panoramas at different camera locations, we can recover 3-D data of the scene using a set of simple techniques: feature tracking, an 8-point structure from motion algorithm, and multibaseline stereo. We also investigate the effect of median filtering on the recovered 3-D point distributions, and show the results of our approach applied to both synthetic and real scenes.     

Incremental model-based estimation using geometric constraints

We present a model-based framework for incremental, adaptive object shape estimation and tracking in monocular image sequences. Parametric structure and motion estimation methods usually assume a fixed class of shape representation (splines, deformable superquadrics, etc.) that is initialized prior to tracking. Since the model shape coverage is fixed a priori, the incremental recovery of structure is decoupled from tracking, thereby limiting both processes in their scope and robustness. In this work, we describe a model-based framework that supports the automatic detection and integration of low-level geometric primitives (lines) incrementally. Such primitives are not explicitly captured in the initial model, but are moving consistently with its image motion. The consistency tests used to identify new structure are based on trinocular constraints between geometric primitives. The method allows not only an increase in the model scope, but also improves tracking accuracy by including the newly recovered features in its state estimation. The formulation is a step toward automatic model building, since it allows both weaker assumptions on the availability of a prior shape representation and on the number of features that would otherwise be necessary for entirely bottom-up reconstruction. We demonstrate the proposed approach on two separate image-based tracking domains, each involving complex 3D object structure and motion.     

Approximate triangulation and region growing for efficient segmentation and smoothing of range images

Decomposing sensory measurements into coherent parts is a fundamental prerequisite for scene understanding that is required for solving complex tasks, e.g., in the field of mobile manipulation. In this article, we describe methods for efficient segmentation of range images and organized point clouds. In order to achieve real-time performance in complex environments, we focus our approach on simple but robust solutions. We present a fast approach to surface reconstruction in range images and organized point clouds by means of approximate polygonal meshing. The obtained local surface information and neighborhoods are then used to (1) smooth the underlying measurements, and (2) segment the image into planar regions and other geometric primitives. A comparative evaluation using publicly available data sets shows that our approach achieves state-of-the-art performance while being significantly faster than other methods.     

Detecting image forgeries using metrology

Image forgery technology has become popular for tampering with digital photography. This paper presents a framework for detecting fake regions using single view metrology and enforcing geometric constraints from shadows. In particular, we describe how to (1) estimate the region of interest’s 3D measurements from a single perspective view of a scene given only minimal geometric information determined from the image, (2) determine the fake region by exploring the imaged shadow relations that are modeled by the planar homology. We also show that image forgery on the vertical plane or arbitrary plane can be detected through the measurement on such plane. Our approach efficiently extracts geometric constraints from a single image and makes use of them for the digital forgery detection. Experimental results on both the synthetic data against noise and visually plausible images demonstrate the performance of the proposed method.     

Rank Conditions on the Multiple-View Matrix

Geometric relationships governing multiple images of points and lines and associated algorithms have been studied to a large extent separately in multiple-view geometry. The previous studies led to a characterization based on multilinear constraints, which have been extensively used for structure and motion recovery, feature matching and image transfer. In this paper we present a universal rank condition on the so-called multiple-view matrix M for arbitrarily combined point and line features across multiple views. The condition gives rise to a complete set of constraints among multiple images. All previously known multilinear constraints become simple instantiations of the new condition. In particular, the relationship between bilinear, trilinear and quadrilinear constraints can be clearly revealed from this new approach. The theory enables us to carry out global geometric analysis for multiple images, as well as systematically characterize all degenerate configurations, without breaking image sequence into pairwise or triple-wise sets of views. This global treatment allows us to utilize all incidence conditions governing all features in all images simultaneously for a consistent recovery of motion and structure from multiple views. In particular, a rank-based multiple-view factorization algorithm for motion and structure recovery is derived from the rank condition. Simulation results are presented to validate the multiple-view matrix based approach.     

高度场八叉树的体特征表达算法

体特征表达对用户理解和认知虚拟环境有着至关重要的作用。当前的体特征表达算法由于存储量大且不易于在GPU中加速等问题,渲染效率低下,难以满足场景可视化的实时性需求。针对这一问题,提出了一种高效的高度场八叉树体特征表达算法,不仅解决了传统高度场仅能表达2.5维模型,无法表达真三维模型的问题,而且为体特征表达提供了一种新的可行途径。算法使用八叉树结构生成三维模型的高度场表示,将传统的z向高度场扩展到x,y,z三个方向的高度场。首先,提出了三角面片预处理方法,保证模型精度和数据的完整性;其次,提出了基于投影变换的高度场表示判断及栅格化方法,将几何图元转换成二维空间的高度场数据;最后,提出了基于高度场八叉树的光线投射算法。实验结果表明,算法能极大地减少存储量,具有较高的光线投射效率,表达三维模型时取得较好效果。     

Garuda: a scalable tiled display wall using commodity PCs

Cluster-based tiled display walls can provide cost-effective and scalable displays with high resolution and a large display area. The software to drive them needs to scale too if arbitrarily large displays are to be built. Chromium is a popular software API used to construct such displays. Chromium transparently renders any OpenGL application to a tiled display by partitioning and sending individual OpenGL primitives to each client per frame. Visualization applications often deal with massive geometric data with millions of primitives. Transmitting them every frame results in huge network requirements that adversely affect the scalability of the system. In this paper, we present Garuda, a client-server-based display wall framework that uses off-the-shelf hardware and a standard network. Garuda is scalable to large tile configurations and massive environments. It can transparently render any application built using the Open Scene Graph (OSG) API to a tiled display without any modification by the user. The Garuda server uses an object-based scene structure represented using a scene graph. The server determines the objects visible to each display tile using a novel adaptive algorithm that culls the scene graph to a hierarchy of frustums. Required parts of the scene graph are transmitted to the clients, which cache them to exploit the interframe redundancy. A multicast-based protocol is used to transmit the geometry to exploit the spatial redundancy present in tiled display systems. A geometry push philosophy from the server helps keep the clients in sync with one another. Neither the server nor a client needs to render the entire scene, making the system suitable for interactive rendering of massive models. Transparent rendering is achieved by intercepting the cull, draw, and swap functions of OSG and replacing them with our own. We demonstrate the performance and scalability of the Garuda system for different configurations of display wall. We also show that the server and network loads grow sublinearly with the increase in the number of tiles, which makes our scheme suitable to construct very large displays.     

Animation toolkit based on a database approach for reusing motions and models

As animations become more readily available, simultaneously the complexity of creating animations has also increased. In this paper, we address the issue by describing an animation toolkit based on a database approach for reusing geometric animation models and their motion sequences. The aim of our approach is to create a framework aimed for novice animators. Here, we use an alternative notion of a VRML scene graph to describe a geometric model, specifically intended for reuse. We represent this scene graph model as a relational database. A set of spatial, temporal, and motion operations are then used to manipulate the models and motions in an animation database. Spatial operations help in inserting/deleting geometric models in a new animation scene. Temporal and motion operations help in generating animation sequences in a variety of ways. For instance, motion information of one geometric model can be applied to another model or a motion sequence can be retargeted to meet additional constraints (e.g., wiping action on a table can be retargeted with constraints that reduce the size of the table). We present the design and implementation of this toolkit along with several interesting examples of animation sequences that can be generated using this toolkit.     

Perceptual primitives from an extended 4D Hough transform

Directional features extracted from Gabor wavelets responses were used to train a structure of self-organising maps, thus classifying each pixel in the image within a neuron-map. Resulting directional primitives were grouped into perceptual primitives introducing an extended 4D Hough transform to group pixels with similar directional features. These can then be used as perceptual primitives to detect salient structures. The proposed method has independently fixed parameters that do not need to be tuned for different kind or quality of images. We present results in application to noisy FLIR images and show that line primitives for complex structures, such as bridges, or simple structures, such as runways, can be found by this approach. We compare and demonstrate the quality of our results with those obtained through a parameter-dependent traditional Canny edge detector and Hough line finding process.     

Putting the User in the Loop for Image-Based Modeling

We refer to the task of recovering the 3D structure of an object or a scene using 2D images as image-based modeling. In this paper, we formulate the task of recovering the 3D structure as a discrete optimization problem solved via energy minimization. In this standard framework of a Markov random field (MRF) defined over the image we present algorithms that allow the user to intuitively interact with the algorithm. We introduce an algorithm where the user guides the process of image-based modeling to find and model the object of interest by manually interacting with the nodes of the graph. We develop end user applications using this algorithm that allow object of interest 3D modeling on a mobile device and 3D printing of the object of interest. We also propose an alternate active learning algorithm that guides the user input. An initial attempt is made at reconstructing the scene without supervision. Given the reconstruction, an active learning algorithm uses intuitive cues to quantify the uncertainty of the algorithm and suggest regions, querying the user to provide support for the uncertain regions via simple scribbles. These constraints are used to update the unary and the pairwise energies that, when solved, lead to better reconstructions. We show through machine experiments and a user study that the proposed approach intelligently queries the users for constraints, and users achieve better reconstructions of the scene faster, especially for scenes with textureless surfaces lacking strong textural or structural cues that algorithms typically require.