Effective energy-based multi-view piecewise planar stereo

For piecewise planar scene modeling, many challenging issues still persist, in particular, how to generate sufficient candidate planes and how to assign an optimal plane for each spatial patch. To address these issues, we present a novel multi-view piecewise planar stereo method for the complete reconstruction. In our method, reconstruction is formulated as an energy-based plane labeling problem, where photo-consistency and geometric constraints are incorporated to a unified superpixel-level MRF (Markov Random Field) framework. To enhance the efficacy of the plane inference and optimization, an effective multi-direction plane sweeping with much restricted search space is carried out to generate sufficient and reliable candidate planes. Experiments show that our method can effectively handle many challenging factors (e.g., slant surfaces, textureless regions) and achieve satisfactory results.     

Reconstructing piecewise planar scenes with multi-view regularization

Computational Visual Media - Reconstruction of man-made scenes from multi-view images is an important problem in computer vision and computer graphics. Observing that man-made scenes are usually...     

基于能量的图像立体匹配算法

通过引入能量函数,给出了一种快速高精度求取匹配点算法——基于能量的算法。首先快速求取初始匹配点,然后对初始匹配点的相邻区域的点进行能量估计,计算其判断函数值,再根据判断函数值对原始数据进行矫正,得到精确的匹配点数据,从而实现匹配点的快速高精度估计。     

Video-based, real-time multi-view stereo

We investigate the problem of obtaining a dense reconstruction in real-time, from a live video stream. In recent years, multi-view stereo (MVS) has received considerable attention and a number of methods have been proposed. However, most methods operate under the assumption of a relatively sparse set of still images as input and unlimited computation time. Video based MVS has received less attention despite the fact that video sequences offer significant benefits in terms of usability of MVS systems. In this paper we propose a novel video based MVS algorithm that is suitable for real-time, interactive 3d modeling with a hand-held camera. The key idea is a per-pixel, probabilistic depth estimation scheme that updates posterior depth distributions with every new frame. The current implementation is capable of updating 15 million distributions/s. We evaluate the proposed method against the state-of-the-art real-time MVS method and show improvement in terms of accuracy.     

任意视角的多视图立体匹配系统

为了得到高精度深度图,通过特征点提取与匹配、计算基础矩阵F、引导互匹配、捆集调整等一系列技术,求出每幅视图在同一空间坐标系下的高精度摄像机矩阵P.为任意角度的多个视图建立可扩展的主框架,不需要图像矫正.基于图像坐标和自动计算出的视差范围建立三维网络,为每条边分配适当的权值,把多视图立体匹配问题转化为网络最大流问题.算法保证了高准确率和平滑性,实验结果表明此方法适用于三维模型重建.     

基于立体视觉的三维重建算法

针对PMVS(patch multi-view stereo)算法在三维重建过程中容易出现重建表面不够光滑和连续,某些区域与真实物体的形状有偏差,以及在片面扩展时算法的空间和时间复杂度非常高,在处理高分辨率图像时需要付出巨大的时间和空间代价等问题,提出了一种加入法向调整的PMVS改进算法,使重建出的表面在某些细节部位更加符合真实物体形状,同时通过采用多分辨率分级重建的策略在一定程度上缩短了算法的运行时间,提高了重建效率。通过实验表明了改进后的算法的准确性和高效性。     

Textured mesh surface reconstruction of large buildings with multi-view stereo

There are three main approaches for reconstructing 3D models of buildings. Laser scanning is accurate but expensive and limited by the laser’s range. Structure-from-motion (SfM) and multi-view stereo (MVS) recover 3D point clouds from multiple views of a building. MVS methods, especially patch-based MVS, can achieve higher density than do SfM methods. Sophisticated algorithms need to be applied to the point clouds to construct mesh surfaces. The recovered point clouds can be sparse in areas that lack features for accurate reconstruction, making recovery of complete surfaces difficult. Moreover, segmentation of the building’s surfaces from surrounding surfaces almost always requires some form of manual inputs, diminishing the ease of practical application of automatic 3D reconstruction algorithms. This paper presents an alternative approach for reconstructing textured mesh surfaces from point cloud recovered by patch-based MVS method. To a good first approximation, a building’s surfaces can be modeled by planes or curve surfaces which are fitted to the point cloud. 3D points are resampled on the fitted surfaces in an orderly pattern, whose colors are obtained from the input images. This approach is simple, inexpensive, and effective for reconstructing textured mesh surfaces of large buildings. Test results show that the reconstructed 3D models are sufficiently accurate and realistic for 3D visualization in various applications.     

Practical BRDF reconstruction using reliable geometric regions from multi-view stereo

Computational Visual Media - In this paper, we present a practical method for reconstructing the bidirectional reflectance distribution function (BRDF) from multiple images of a real object...     

基于改进PMS算法的多视点立体视图优化算法

针对多视点视图匹配存在的不足,提出一种多视点法平面与视差空间及聚合成本最优化的改进PMS算法.设计多视点视图优化框架,推导出视点聚合成本函数;结合多视点聚合成本函数最优化方法,设计法平面与视差及聚合成本最优化的改进PMS算法,给出算法步骤;对提出的算法在不同噪声下的角度误差、相同噪声不同视点数的角度误差和相同噪声不同视...     

Accurate,dense and shading-aware multi-view stereo reconstruction using metaheuritic optimization

Among the modern means of 3D geometry creation that exist in the literature, there are the Multi-View Stereo (MVS) reconstruction methods that received much attention from the research community and the multimedia industry. In fact, several methods showed that it is possible to recover geometry only from images with reconstruction accuracies paralleling that of excessively expensive laser scanners. The majority of these methods perform on images such as online community photo collection and estimate the surface position with its orientation by minimizing a matching cost function defined over a small local region. However, these datasets not only they are large but also contain more challenging scenes setups with different photometric effects; therefore fine-grained details of an object’s surface cannot be captured. This paper presents a robust multi-view stereo method based on metaheuristic optimization namely the Particle Swarm Optimization (PSO) in order to find the optimal depth, orientation, and surface roughness. To deal with the various shading and stereo mismatch problems caused by rough surfaces, shadows, and interreflections, we propose to use a robust matching/energy function which is a combination of two similarity measurements. Finally, our method computes individual depth maps that can be merged into compelling scene reconstructions. The proposed method is evaluated quantitatively using well-known Middlebury datasets and the obtained results show a high completeness score and comparable accuracy to those of the current top performing algorithms.

     

Correspondenceless stereo and motion: planar surfaces

It is shown that a binocular observer can recover the depth and three-dimensional motion of a rigid planar patch without using any correspondences between the left and right image frames (static) or between the successive dynamic frames (dynamic). Uniqueness and robustness issues are studied with respect to this problem and experimental results are given from the application of the theory to real images     

Optimum uniform piecewise linear approximation of planar curves

Two-dimensional digital curves are often uniformly approximated by polygons or piecewise linear curves. Several algorithms have been proposed in the literature to find such curves. We present an algorithm that finds a piecewise linear curve with the minimal number of segments required to approximate a curve within a uniform error with fixed initial and final points. We compare our optimal algorithm to several suboptimal algorithms with respect to the number of linear segments required in the approximation and the execution time of the algorithm.     

Null controllability of planar bimodal piecewise linear systems

This article investigates the null controllability of planar bimodal piecewise linear systems, which consist of two second order LTI systems separated by a line crossing through the origin. It is interesting to note that even when both subsystems are controllable in the classical sense, the whole piecewise linear system may be not null controllable. On the other hand, a piecewise linear system could be null controllable even when it has uncontrollable subsystems. First, the evolution directions from any non-origin state are studied from the geometric point of view, and it turns out that the directions usually span an open half space. Then, we derive an explicit and easily verifiable necessary and sufficient condition for a planar bimodal piecewise linear system to be null controllable. Finally, the article concludes with several numerical examples and discussions on the results and future work.     

Embedded planar surface segmentation system for stereo images

An embedded system is developed to segment stereo images using disparity. The recent developments in the embedded system architecture have allowed real time implementation of low-level vision tasks such as stereo disparity computation. At the same time, an intermediate level task such as segmentation is rarely attempted in an embedded system. To solve the planar surface segmentation problem, which is iterative in nature, our system implements a Segmentation–Estimation framework. In the segmentation phase, segmentation labels are assigned based on the underlying plane parameters. Connected component analysis is carried out on the segmentation result to select the largest spatially connected area for each plane. From the largest areas, the parameters for each plane are reestimated. This iterative process was implemented on TMS320DM642 based embedded system that operates at 3–5 frames per second on images of size 320 × 240.     

Reconstruction of piecewise planar objects from point clouds

This article discusses the reverse engineering problem of reconstructing objects with planar faces. We will present the main geometric features of a modeling system which are the detection of planar faces and the generation of a cad model. The algorithms are applied to the problem of reconstruction of buildings from airborne laser scanner data.     

A random sampling strategy for piecewise planar scene segmentation

We investigate the problem of automatically creating 3D models of man-made environments that we represent as collections of textured planes. A typical approach is to automatically reconstruct a sparse 3D model made of points, and to manually indicate their plane membership, as well as the delineation of the planes: this is the piecewise planar segmentation phase. Texture images are then extracted by merging perspectively corrected input images. We propose an automatic approach to the piecewise planar segmentation phase, that detects the number of planes to approximate the scene surface to some extent, and the parameters of these planes, from a sparse 3D model made of points. Our segmentation method is inspired from the robust estimator ransac. It generates and scores plane hypotheses by random sampling of the 3D points. Our plane scoring function and our plane comparison function, required to prevent detecting the same plane twice, are designed to detect planes with large or small support. The plane scoring function recovers the plane delineation and quantifies the saliency of the plane hypothesis based on approximate photoconsistency. We finally refine all the 3D model parameters, i.e., the planes and the points on these planes, as well as camera pose, by minimizing the reprojection error with respect to the measured image points, using bundle adjustment. The approach is validated on simulated and real data.     

Interactive object modelling based on piecewise planar surface patches

Detecting elements such as planes in 3D is essential to describe objects for applications such as robotics and augmented reality. While plane estimation is well studied, table-top scenes exhibit a large number of planes and methods often lock onto a dominant plane or do not estimate 3D object structure but only homographies of individual planes. In this paper we introduce MDL to the problem of incrementally detecting multiple planar patches in a scene using tracked interest points in image sequences. Planar patches are reconstructed and stored in a keyframe-based graph structure. In case different motions occur, separate object hypotheses are modelled from currently visible patches and patches seen in previous frames. We evaluate our approach on a standard data set published by the Visual Geometry Group at the University of Oxford [24] and on our own data set containing table-top scenes. Results indicate that our approach significantly improves over the state-of-the-art algorithms.     

Sharp error bounds for piecewise linear interpolation of planar curves

Curves are commonly drawn by piecewise linear interpolation, but to worry about the error is rather seldom. In the present paper we give a strong mathematical error analysis for curve segments with bounded curvature and length. Though the result seems very clear, the proof turned out to be unexpectedly hard, comparable to that of the famous four vertex theorem.     

Registration and interactive planar segmentation for stereo images of polyhedral scenes

We introduce a two-step iterative segmentation and registration method to find coplanar surfaces among stereo images of a polyhedral environment. The novelties of this paper are: (i) to propose a user-defined initialization easing the image matching and segmentation, (ii) to incorporate color appearance and planar projection information into a Bayesian segmentation scheme, and (iii) to add consistency to the projective transformations related to the polyhedral structure of the scenes. The method utilizes an assisted Bayesian color segmentation scheme. The initial user-assisted segmentation is used to define search regions for planar homography image registration. The two reliable methods cooperate to obtain probabilities for coplanar regions with similar color information that are used to get a new segmentation by means of quadratic Markov measure fields (QMMF). We search for the best regions by iterating both steps: registration and segmentation.