一种基于多视立体视觉的多视图直线匹配方法

提出了一种基于多视立体视觉（Multiple view stereo，MVS）进行多视图直线匹配的方法. 本文方法首先利用MVS所得到的三维点云及其可见性信息，建立三维点与图像直线的对应关系. 根据此对应关系，为每条图像直线建立由一个三维点集和一个三维单位向量构成的描述子，用以衡量图像直线之间的相似性及一致性. 之后，本文方法以所有图像直线为顶点建立一个图，并引入了图谱分析来获取统一的顶点距离度量. 最后，本方法对DBSCAN聚类算法进行了修改，并用修改后的算法从图谱分析结果中获取可靠的直线匹配. 实验显示，本方法比已有方法更加鲁棒，并且有更高的准确率.     

The Geometry and Matching of Lines and Curves Over Multiple Views

This paper describes the geometry of imaged curves in two and three views. Multi-view relationships are developed for lines, conics and non-algebraic curves. The new relationships focus on determining the plane of the curve in a projective reconstruction, and in particular using the homography induced by this plane for transfer from one image to another. It is shown that given the fundamental matrix between two views, and images of the curve in each view, then the plane of a conic may be determined up to a two fold ambiguity, but local curvature of a curve uniquely determines the plane. It is then shown that given the trifocal tensor between three views, this plane defines a homography map which may be used to transfer a conic or the curvature from two views to a third. Simple expressions are developed for the plane and homography in each case.A set of algorithms are then described for automatically matching individual line segments and curves between images. The algorithms use both photometric information and the multiple view geometric relationships. For image pairs the homography facilitates the computation of a neighbourhood cross-correlation based matching score for putative line/curve correspondences. For image triplets cross-correlation matching scores are used in conjunction with line/curve transfer based on the trifocal geometry to disambiguate matches. Algorithms are developed for both short and wide baselines. The algorithms are robust to deficiencies in the segment extraction and partial occlusion.Experimental results are given for image pairs and triplets, for varying motions between views, and for different scene types. The methods are applicable to line/curve matching in stereo and trinocular rigs, and as a starting point for line/curve matching through monocular image sequences.     

Novel view synthesis by cascading trilinear tensors

Presents a new method for synthesizing novel views of a 3D scene from two or three reference images in full correspondence. The core of this work is the use and manipulation of an algebraic entity, termed the “trilinear tensor”, that links point correspondences across three images. For a given virtual camera position and orientation, a new trilinear tensor can be computed based on the original tensor of the reference images. The desired view can then be created using this new trilinear tensor and point correspondences across two of the reference images     

Towards reliable matching of images containing repetitive patterns

This paper aims to solve the problem of matching images containing repetitive patterns. Although repetitive patterns widely exist in real world images, these images are difficult to be matched due to local ambiguities even if the viewpoint changes are not very large. It is still an open and challenging problem. To solve the problem, this paper proposes to match pairs of interest points and then obtain point correspondences from the matched pairs of interest points based on the low distortion constraint, which is meant that the distortions of point groups should be small across images. By matching pairs of interest points, local ambiguities induced by repetitive patterns can be reduced to some extent since information in a much larger region is used. Meanwhile, owing to our newly defined compatibility measure between one correspondence and a set of point correspondences, the obtained point correspondences are very reliable. Experiments have demonstrated the effectiveness of our method and its superiority to the existing methods.     

基于匹配测度加权求解基础矩阵的三维重建算法

双视图三维重建中引入了同名特征点的匹配测度,为提高基础矩阵F的精度提供了数据处理指导,将特征点匹配和F解算及三维重建环节连接成一个整体。在分析特征点提取、匹配方法的基础上定义了同名点匹配测度函数。用测度函数作为匹配点的权值对归一化8点求解F的算法进行加权,并使用随机采样一致性(RANSAC)鲁棒算法解决匹配野值问题。根据已知的像机内参数，从基础矩阵中分解相对运动，并用模型的内点进行运动优化，最后三角交会得到三维重建结果。实验结果表明,此算法达到了线性求解F矩阵和三维重建的鲁棒性高精度实现。     

The 3D Line Motion Matrix and Alignment of Line Reconstructions

We study the problem of aligning two 3D line reconstructions in projective, affine, metric or Euclidean space.We introduce the 6 × 6 3D line motion matrix that acts on Plücker coordinates. We characterize its algebraic properties and its relation to the usual 4 × 4 point motion matrix, and propose various methods for estimating 3D motion from line correspondences, based on cost functions defined in images or 3D space. We assess the quality of the different estimation methods using simulated data and real images.     

基于SIFT算法的目标特征检测与提取技术研究

验证了一种能够在不同图像之间进行同一个物体相匹配的方法,具有很强的可靠性,称之为SIFT算法(尺度不变特征变换).SIFT算法能够处理图像间发生的尺度变换、旋转、很大范围内的仿射形变、视角变换、噪声以及光照变换.它的功能十分强大,甚至可以仅仅根据一个简单的物体特征,在一个大型数据库中的许多高品质图像中进行相应目标的寻找...     

Rigidity checking of 3D point correspondences under perspectiveprojection

An algorithm is described which rapidly verifies the potential rigidity of three-dimensional point correspondences from a pair of two-dimensional views under perspective projection. The output of the algorithm is a simple yes or no answer to the question “Could these corresponding points from two views be the projection of a rigid configuration?” Potential applications include 3D object recognition from a single previous view and correspondence matching for stereo or motion over widely separated views. The rigidity checking problem is different from the structure-from-motion problem because it is often the case that two views cannot provide an accurate structure-from-motion estimate due to ambiguity and ill conditioning, whereas it is still possible to give an accurate yes/no answer to the rigidity question. Rigidity checking verifies point correspondences using 3D recovery equations as a matching condition. The proposed algorithm improves upon other methods that fall under this approach because it works with as few as six corresponding points under full perspective projection, handles correspondences from widely separated views, makes full use of the disparity of the correspondences, and is integrated with a linear algorithm for 3D recovery due to Kontsevich (1993). Results are given for experiments with synthetic and real image data. A complete implementation of this algorithm is being made publicly available     

Polyhedral Object Localization in an Image by Referencing to a Single Model View

Identifying a three-dimensional (3D) object in an image is traditionally dealt with by referencing to a 3D model of the object. In the last few years there has been a growing interest of using not a 3D shape but multiple views of the object as the reference. This paper attempts a further step in the direction, using not multiple views but a single clean view as the reference model. The key issue is how to establish correspondences from the model view where the boundary of the object is explicitly available, to the scene view where the object can be surrounded by various distracting entities and its boundary disturbed by noise. We propose a solution to the problem, which is based upon a mechanism of predicting correspondences from just four particular initial point correspondences. The object is required to be polyhedral or near-polyhedral. The correspondence mechanism has a computational complexity linear with respect to the total number of visible corners of the object in the model view. The limitation of the mechanism is also analyzed thoroughly in this paper. Experimental results over real images are presented to illustrate the performance of the proposed solution.     

Conic reconstruction and correspondence from two views

Conics are widely accepted as one of the most fundamental image features together with points and line segments. The problem of space reconstruction and correspondence of two conics from two views is addressed in this paper. It is shown that there are two independent polynomial conditions on the corresponding pair of conics across two views, given the relative orientation of the two views. These two correspondence conditions are derived algebraically and one of them is shown to be fundamental in establishing the correspondences of conics. A unified closed-form solution is also developed for both projective reconstruction of conics in space from two uncalibrated camera views and metric reconstruction from two calibrated camera views. Experiments are conducted to demonstrate the discriminality of the correspondence conditions and the accuracy and stability of the reconstruction both for simulated and real images     

Estimating motion and structure from correspondences of linesegments between two perspective images

Presents an algorithm for determining 3D motion and structure from correspondences of line segments between two perspective images. To the author's knowledge, this paper is the first investigation of use of line segments in motion and structure from motion. Classical methods use their geometric abstraction, namely straight lines, but then three images are necessary for the motion and structure determination process. In this paper the author shows that it is possible to recover motion from two views when using line segments. The assumption used is that two matched line segments contain the projection of a common part of the corresponding line segment in space, i.e., they overlap. Indeed, this is what the author uses to match line segments between different views. This assumption constrains the possible motion between two views to an open set in motion parameter space. A heuristic, consisting of maximizing the overlap, leads to a unique solution. Both synthetic and real data have been used to test the proposed algorithm, and excellent results have been obtained with real data containing a relatively large set of line segments     

Three-dimensional data acquisition by trinocular vision

For recognition of three-dimensional (3D) shapes and measurement of 3D positions of objects it is important for a vision system to be able to measure the 3D data of dense points in the environment. One approach is to measure the distance on the basis of the triangulation principle from the disparity of two images. However, this binocular vision method has difficulty in finding a correspondence of features between two images. This correspondence problem can be solved geometrically by adding another camera, i.e. by trinocular vision. This paper presents the principles and implementation details of trinocular vision. On the basis of the proposed method, we carried out several experiments, from which we found that many correct correspondences could be established, even for images of a complex scene, by only the geometrical constraint of trinocular vision. However, when there are dense points in the image, multiple candidate points are found and a unique correspondence cannot be established. Two approaches to solve this problem are discussed in this paper.     

Matching point features with ordered geometric, rigidity, anddisparity constraints

This correspondence presents a matching algorithm for obtaining feature point correspondences across images containing rigid objects undergoing different motions. First point features are detected using newly developed feature detectors. Then a variety of constraints are applied starting with simplest and following with more informed ones. First, an intensity-based matching algorithm is applied to the feature points to obtain unique point correspondences. This is followed by the application of a sequence of newly developed heuristic tests involving geometry, rigidity, and disparity. The geometric tests match two-dimensional geometrical relationships among the feature points, the rigidity test enforces the three dimensional rigidity of the object, and the disparity test ensures that no matched feature point in an image could be rematched with another feature, if reassigned another disparity value associated with another matched pair or an assumed match on the epipolar line. The computational complexity is proportional to the numbers of detected feature points in the two images. Experimental results with indoor and outdoor images are presented, which show that the algorithm yields only correct matches for scenes containing rigid objects     

Reconstructing camera projection matrices from multiple pairwise overlapping views

In this work we address the problem of projective reconstruction from multiple views with missing data. Factorization based algorithms require point correspondences across all the views. In many applications this is an unrealistic assumption. Current methods that solve the problem of projective reconstruction with missing data require correspondence information across triplets of images. We propose a projective reconstruction method that yields a consistent camera set given the fundamental matrices between pairs of views without directly using the image correspondences. The algorithm is based on breaking the reconstruction problem into small steps. In each step, we eliminate as much uncertainty as possible.     

图像三维重建中的特征点匹配方法研究

针对传统三维重建中点特征匹配算法的局限性,提出了基于相关系数的图像点特征匹配方法,实现了对图像点特征匹配与三维重建同步进行的目的。基于同一物体的两幅图像,给出了其特征匹配和三维重建的算法,并通过实验数据分析了此算法的有效性。     

Stereo Image Analysis of Non-Lambertian Surfaces

Stereo image analysis is based on establishing correspondences between a pair of images by determining similarity measures for potentially corresponding image parts. Such similarity criteria are only strictly valid for surfaces with Lambertian (diffuse) reflectance characteristics. Specular reflections are viewpoint dependent and may thus cause large intensity differences at corresponding image points. In the presence of specular reflections, traditional stereo approaches are often unable to establish correspondences at all, or the inferred disparity values tend to be inaccurate, or the established correspondences do not belong to the same physical surface point. The stereo image analysis framework for non-Lambertian surfaces presented in this contribution combines geometric cues with photometric and polarimetric information into an iterative scheme that allows to establish stereo correspondences in accordance with the specular reflectance behaviour and at the same time to determine the surface gradient field based on the known photometric and polarimetric reflectance properties. The described approach yields a dense 3D reconstruction of the surface which is consistent with all observed geometric and photopolarimetric data. Initially, a sparse 3D point cloud of the surface is computed by traditional blockmatching stereo. Subsequently, a dense 3D profile of the surface is determined in the coordinate system of camera 1 based on the shape from photopolarimetric reflectance and depth technique. A synthetic image of the surface is rendered in the coordinate system of camera 2 using the illumination direction and reflectance properties of the surface material. Point correspondences between the rendered image and the observed image of camera 2 are established with the blockmatching technique. This procedure yields an increased number of 3D points of higher accuracy, compared to the initial 3D point cloud. The improved 3D point cloud is used to compute a refined dense 3D surface profile. These steps are iterated until convergence of the 3D reconstruction. An experimental evaluation of our method is provided for areas of several square centimetres of forged and cast iron objects with rough surfaces displaying both diffuse and significant specular reflectance components, where traditional stereo image analysis largely fails. A comparison to independently measured ground truth data reveals that the root-mean-square error of the 3D reconstruction results is typically of the order 30–100 μm at a lateral pixel resolution of 86 μm. For two example surfaces, the number of stereo correspondences established by the specular stereo algorithm is several orders of magnitude higher than the initial number of 3D points. For one example surface, the number of stereo correspondences decreases by a factor of about two, but the 3D point cloud obtained with the specular stereo method is less noisy, contains a negligible number of outliers, and shows significantly more surface detail than the initial 3D point cloud. For poorly known reflectance parameters we observe a graceful degradation of the accuracy of 3D reconstruction.     

Matching Widely Separated Views Based on Affine Invariant Regions

Invariant regions are self-adaptive image patches that automatically deform with changing viewpoint as to keep on covering identical physical parts of a scene. Such regions can be extracted directly from a single image. They are then described by a set of invariant features, which makes it relatively easy to match them between views, even under wide baseline conditions. In this contribution, two methods to extract invariant regions are presented. The first one starts from corners and uses the nearby edges, while the second one is purely intensity-based. As a matter of fact, the goal is to build an opportunistic system that exploits several types of invariant regions as it sees fit. This yields more correspondences and a system that can deal with a wider range of images. To increase the robustness of the system, two semi-local constraints on combinations of region correspondences are derived (one geometric, the other photometric). They allow to test the consistency of correspondences and hence to reject falsely matched regions. Experiments on images of real-world scenes taken from substantially different viewpoints demonstrate the feasibility of the approach.     

Uncertainty propagation and the matching of junctions as featuregroupings

The interpretation of the 3D world from image sequences requires the identification and correspondences of key features in the scene. We describe a robust algorithm for matching groupings of features related to the objects in the scene. We consider the propagation of uncertainty from the feature detection stage through the grouping stage to provide a measure of uncertainty at the matching stage. We focus upon indoor scenes and match junctions, which are groupings of line segments that meet at a single point. A model of the uncertainty in junction detection is described, and the junction uncertainty under the epipolar constraint is determined. Junction correspondence is achieved through matching of each line segment associated with the junction. A match likelihood is then derived based upon the detection uncertainties and then combined with information on junction topology to create a similarity measure. A robust matching algorithm is proposed and used to match junctions between pairs of images. The presented experimental results on real images show that the matching algorithm produces sufficiently reliable results for applications such as structure from motion