图像漫游：双灭点的松弛蜘蛛网格算法

为了解决蜘蛛网格算法在图像漫游中局限性，提出了双灭点的松弛蜘蛛网格算法。该算法在用于图像实时漫游时，用双灭点进行计算，其与前人提出的方法相比，更具通用性，不仅可以解决单灭点、双灭点、多灭点和灭线等不同类型的照片漫游问题，而且减轻了用户交互的工作量。此外，为了进一步提高图像的绘制质量，以获得高质量的漫游效果，还采用了一种分片绘制的策略。多种类型的图片实验证明了该算法的有效性。     

Geo-registration of wide-baseline panoramic image sequences using a digital map reference

This paper describes a method of geo-registering a sequence of panoramic images to a digital map by matching pixel information from the images with information on the building footprint contained in a digital map. Recently, images captured at the ground level using a Mobile Mapping System (MMS), such as the panoramic images displayed by Google Street View, have been considered as a valuable resource for three-dimensional (3D) building modeling. However, the wide intervals between these panoramic images, as well as locational and directional error from the related sensors, make it difficult to analyze the image data. This paper demonstrates a formulation method for connecting pixels in panoramic images with information on footprint vertices and building lines contained in a digital map. To allow both pixel and footprint information consistent in 3D space, each panoramic image is tilt-corrected in pre-processing to upright the image using the estimated pitch and roll of a vehicle and removing the pitch and roll effects from the panoramic image pixels. Through the proposed formulation, a single panoramic image can be easily geo-registered with simple user-provided constraints, and adjacent sequential images can then be automatically geo-registered using point feature matching. Experimental results showed a significant reduction in the locational and directional error of sequential panoramic images, and the proposed vanishing point (VP) based validation process was found to successfully detect failure cases.     

一种快速的人工场景图像消失点检测方法

人工场景中包含了大量的空间平行线以及垂直边,这些空间平行线映射到图像中相交产生的交点即消失点。消失点检测对摄像机标定、三维场景重建等都有着重要的意义。传统的消失点检测算法往往基于二维霍夫参数空间,复杂度高、效率低。因此,提出一种新的方法,先检测图像中较长的边界线,并将检测到的线段进行筛选、分组;然后利用消失点与焦距之间的制约关系,确定三向消失点的位置以及焦距的大小。该方法将传统的二维霍夫参数空间转换为二级一维霍夫参数空间。实验表明,这种方法运算复杂度低、运行时间短。在室外场景图像中,鲁棒性好,且保持较高的准确率。     

混合漫游系统

提出一种新颖的场景绘制方法,首先对图像进行处理,用交互的方法区分场景的背景和前景;然后分别建立背景和前景的3D几何模型,采用一种分片绘制的策略,进一步提高了绘制质量．通过将动态网格叠加到几何模型中,在静态图像中产生动态纹理,丰富场景内容,从而建立了一种混合场景表示,实现逼真的场景实时漫游．实验证明,该算法具有较好的普适性,可以处理包含灭点或灭线的不同类型照片、绘画和全景图．     

Spatial layout recovery from a single omnidirectional image and its matching-free sequential propagation

The goal of this work is to recover the spatial layout of indoor environments from omnidirectional images assuming a Manhattan world structure. We propose a new method for scene structure recovery from a single image. This method is based on the line extraction for omnidirectional images, line classification, and vanishing points estimation combined with a new hierarchical expansion procedure for detecting floor and wall boundaries. Each single omnidirectional image independently provides a useful hypothesis of the 3D scene structure. In order to enhance the robustness and accuracy of this single image-based hypothesis, we extend this estimation with a new homography-based procedure applied to the various hypotheses obtained along the sequence of consecutive images. A key point in this contribution is the use of geometrical constraints for computing the homographies from a single line of the floor. The homography parametrization proposed allows the design of a matching-free method for spatial layout propagation along a sequence of images. Experimental results show single image layout recovery performance and the improvement obtained with the propagation of the hypothesis through the image sequence.     

基于图像序列的虚拟场景重建和漫游

基于图像的建模和绘制方法与基于传统的几何绘制方法相比有很多优点,但是在场景几何参数未知的情况下,要实现单幅图片和图像序列的漫游,难度很大.TIP（tour into a picture）技术使得在一幅图像中漫游成为可能.为了实现场景几何参数未知情况下的场景漫游,在对TIP技术进行扩展的基础上,提出了一种能够在未知视点路径的图像序列中实现场景漫游的方法.这种算法在相机未定标的情况下,不仅解决了图像序列建模和场景漫游过程中前后图像场景不能平滑过渡的问题,并且扩大了TIP中视线方向变化的动态范围,从而在保证图像质量的情况下,实现了由图像序列到场景的无限制漫游.实际图像序列的实验结果表明,该算法是有效的,具有实际应用价值.     

3D human modeling from a single depth image dealing with self-occlusion

This paper presents a 2D to 3D conversion scheme to generate a 3D human model using a single depth image with several color images. In building a complete 3D model, no prior knowledge such as a pre-computed scene structure and photometric and geometric calibrations is required since the depth camera can directly acquire the calibrated geometric and color information in real time. The proposed method deals with a self-occlusion problem which often occurs in images captured by a monocular camera. When an image is obtained from a fixed view, it may not have data for a certain part of an object due to occlusion. The proposed method consists of following steps to resolve this problem. First, the noise in a depth image is reduced by using a series of image processing techniques. Second, a 3D mesh surface is constructed using the proposed depth image-based modeling method. Third, the occlusion problem is resolved by removing the unwanted triangles in the occlusion region and filling the corresponding hole. Finally, textures are extracted and mapped to the 3D surface of the model to provide photo-realistic appearance. Comparison results with the related work demonstrate the efficiency of our method in terms of visual quality and computation time. It can be utilized in creating 3D human models in many 3D applications.     

Using vanishing points for camera calibration and coarse 3D reconstruction from a single image

In this paper, we show how to calibrate a camera and to recover the geometry and the photometry (textures) of objects from a single image. The aim of this work is to make it possible walkthrough and augment reality in a 3D model reconstructed from a single image. The calibration step does not need any calibration target and makes only four assumptions: (1) the single image contains at least two vanishing points, (2) the length (in 3D space) of one line segment (for determining the translation vector) in the image is known, (3) the principle point is the center of the image, and (4) the aspect ratio is fixed by the user. Each vanishing point is determined from a set of parallel lines. These vanishing points help determine a 3D world coordinate system R _o. After having computed the focal length, the rotation matrix and the translation vector are evaluated in turn for describing the rigid motion between R _o and the camera coordinate system R _c. Next, the reconstruction step consists in placing, rotating, scaling, and translating a rectangular 3D box that must fit at best with the potential objects within the scene as seen through the single image. With each face of a rectangular box, a texture that may contain holes due to invisible parts of certain objects is assigned. We show how the textures are extracted and how these holes are located and filled. Our method has been applied to various real images (pictures scanned from books, photographs) and synthetic images.     

一种优化的消失点估计方法及误差分析

空间一组平行直线在图像平面上所成的像的交点称为消失点. 消失点可以提供大量的场景三维结构信息. 本文提出一种新的优化的消失点估计方法. 该方法基于随机采样一致算法(Random sample consensus, RANSAC)对图像空间中的线段进行聚类, 通过最小化Sampson误差获得消失点的极大似然估计(Maximum likelihood estimation, MLE). 该方法不需要预知摄像机参数及直线的三维位置信息. 为了对该算法进行定量评估, 构造了基于反向传播的消失点误差传递模型. 实验结果验证了本文提出算法的有效性.     

应用图像蛛网分割的虚拟现实构建

分析了虚拟现实技术中传统变换拼接方法的不足,介绍了蜘蛛网格分割重生成3D虚拟图像的原理与方法,结合构建校园虚拟漫游系统的设计实例,着重介绍了以渲染漫游感受为目的,所提出的若干新概念和采取的若干新措施。     

Single View Metrology

We describe how 3D affine measurements may be computed from a single perspective view of a scene given only minimal geometric information determined from the image. This minimal information is typically the vanishing line of a reference plane, and a vanishing point for a direction not parallel to the plane. It is shown that affine scene structure may then be determined from the image, without knowledge of the camera's internal calibration (e.g. focal length), nor of the explicit relation between camera and world (pose).In particular, we show how to (i) compute the distance between planes parallel to the reference plane (up to a common scale factor); (ii) compute area and length ratios on any plane parallel to the reference plane; (iii) determine the camera's location. Simple geometric derivations are given for these results. We also develop an algebraic representation which unifies the three types of measurement and, amongst other advantages, permits a first order error propagation analysis to be performed, associating an uncertainty with each measurement.We demonstrate the technique for a variety of applications, including height measurements in forensic images and 3D graphical modelling from single images.     

一种基于TIP技术的场景重建和漫游方法

基于图像的建模和绘制方法与基于传统的几何绘制方法相比有很多优点。但是，在场景几何未知的情况下，要实现单幅图片和图像序列的漫游，难度很大。TIP技术使得在一幅图像中漫游成为可能，但它采用的平面建模方法有很大的局限性。本文在对TIP技术扩展的基础上提出了一种曲面建模的方法。该方法减少了TIP技术的限制，扩展了其适应性。实验结果表明，本文提出的算法是有效的，具有实际应用价值。     

高速公路云台相机的自动标定

目的 云台相机因监控视野广、灵活度高,在高速公路监控系统中发挥出重要的作用,但因云台相机焦距与角度不定时地随监控需求变化,对利用云台相机的图像信息获取真实世界准确的物理信息造成一定困难,因此进行云台相机非现场自动标定方法的研究对高速公路监控系统的应用具有重要价值。方法 本文提出了一种基于消失点约束与车道线模型约束的云台相机自动标定方法,以建立高速公路监控系统的图像信息与真实世界物理信息之间准确描述关系。首先,利用车辆目标运动轨迹的级联霍夫变换投票实现纵向消失点的准确估计,其次以车道线模型物理度量为约束,并采用枚举策略获取横向消失点的准确估计,最终在已知相机高度的条件下实现高速公路云台相机标定参数的准确计算。结果 将本文方法在不同的场景下进行实验,得到在不同的距离下的平均误差分别为4.63%、4.74%、4.81%、4.65%,均小于5%。结论 对多组高速公路监控场景的测试实验结果表明,本文提出的云台相机自动标定方法对高速公路监控场景的物理测量误差能够满足应用需求,与参考方法相比较而言具有较大的优势和一定的应用价值,得到的相机内外参数可用于计算车辆速度与空间位置等。     

三维模型的空间匹配与拼接

为获得敦煌莫高窟第172窟的全景三维模型,实现文化遗产数字化保护,提出一种三维模型的空间匹配与拼接方法。首先通过基于四目立体视觉系统的复杂场景三维重建方法,获取单一场景的三维模型;然后以场景真实三维点云数据的对应图像坐标为约束条件,进行高精度特征点匹配;最后以两对同名点在相邻两场景中的空间距离应该一致为条件,保留匹配精度高的点,用于计算旋转矩阵和平移矩阵,进行三维模型间的空间拼接。实验结果表明:与传统拼接方法相比,本文的模型拼接处无明显错位现象,视觉效果更好,传统方法与本方法的匹配误差分别为0.008 2和0.001 6,拼接精度得到显著改善。     

基于多感知因素的产品三维重构关键技术研究

从一幅简单的图形出发,利用图像中隐藏的透视信息进行三维重构,计算产品特征点的坐标及其相对位置.进行了透视元素、基本体素到组合体素的反求研究,并且给出了其相应的原理和算法.对于几何残缺信息的视觉补偿与推理等关键算法进行了研究,并利用非几何信息的处理与特征建模,搭建基于产品多感知因素的三维重构系统框架,实现了设计重用辅助产品创新设计.     

基于多感知因素的产品三维重构关键技术研究*

从一幅简单的图形出发,利用图像中隐藏的透视信息进行三维重构,计算产品特征点的坐标及其相对位置。进行了透视元素、基本体素到组合体素的反求研究,并且给出了其相应的原理和算法。对于几何残缺信息的视觉补偿与推理等关键算法进行了研究,并利用非几何信息的处理与特征建模,搭建基于产品多感知因素的三维重构系统框架,实现了设计重用辅助产品创新设计。     

Perspective Geometry Based Single Image Camera Calibration

This paper presents a novel method for 3D camera calibration. Calculation of the focal length and the optical center of the camera are the main objectives of this research work. The proposed technique requires a single image having two vanishing points. A rectangular prism is employed as the calibration target to generate vanishing points. The special arrangement of the calibration object adds more accuracy in finding the intrinsic parameters. Based on the geometry of the perspective distortion of the edges of the prisms from the image, vanishing points are found. There on, fixing up the picture plane followed by fixing up of the station point is carried out based on the relations that are formulated. Experimental results of our method are likened with Zhang’s method. Results are tabulated to show the accuracy of the proposed approach.

Pose normalization of 3D models via reflective symmetry on panoramic views

A novel pose normalization method, based on reflective symmetry computed on panoramic views, is presented. Qualitative and experimental investigation in 3D data-sets has led us to the observation that most objects possess a single plane of symmetry. Our approach is thus guided by this observation. Initially, through an iterative procedure, the symmetry plane of a 3D model is estimated, thus computing the first axis of the model. This is achieved by rotating the 3D model and computing reflective symmetry scores on panoramic view images. The other principal axes of the 3D model are estimated by computing the variance of the 3D model’s panoramic views. The proposed method is incorporated in a hybrid scheme, that serves as the pose normalization method in a state-of-the-art 3D object retrieval system. The effectiveness of this system, using the hybrid pose normalization scheme, is evaluated in terms of retrieval accuracy and the results clearly show improved performance against current approaches.     

Hypercatadioptric line images for 3D orientation and image rectification

In central catadioptric systems 3D lines are projected into conics. In this paper we present a new approach to extract conics in the raw catadioptric image, which correspond to projected straight lines in the scene. Using the internal calibration and two image points we are able to compute analytically these conics which we name hypercatadioptric line images. We obtain the error propagation from the image points to the 3D line projection in function of the calibration parameters. We also perform an exhaustive analysis on the elements that can affect the conic extraction accuracy. Besides that, we exploit the presence of parallel lines in man-made environments to compute the dominant vanishing points (VPs) in the omnidirectional image. In order to obtain the intersection of two of these conics we analyze the self-polar triangle common to this pair. With the information contained in the vanishing points we are able to obtain the 3D orientation of the catadioptric system. This method can be used either in a vertical stabilization system required by autonomous navigation or to rectify images required in applications where the vertical orientation of the catadioptric system is assumed. We use synthetic and real images to test the proposed method. We evaluate the 3D orientation accuracy with a ground truth given by a goniometer and with an inertial measurement unit (IMU). We also test our approach performing vertical and full rectifications in sequences of real images.     

Parametric Surface Representation with Bump Image for Dense 3D Modeling Using an RBG-D Camera

When constructing a dense 3D model of an indoor static scene from a sequence of RGB-D images, the choice of the 3D representation (e.g. 3D mesh, cloud of points or implicit function) is of crucial importance. In the last few years, the volumetric truncated signed distance function (TSDF) and its extensions have become popular in the community and largely used for the task of dense 3D modelling using RGB-D sensors. However, as this representation is voxel based, it offers few possibilities for manipulating and/or editing the constructed 3D model, which limits its applicability. In particular, the amount of data required to maintain the volumetric TSDF rapidly becomes huge which limits possibilities for portability. Moreover, simplifications (such as mesh extraction and surface simplification) significantly reduce the accuracy of the 3D model (especially in the color space), and editing the 3D model is difficult. We propose a novel compact, flexible and accurate 3D surface representation based on parametric surface patches augmented by geometric and color texture images. Simple parametric shapes such as planes are roughly fitted to the input depth images, and the deviations of the 3D measurements to the fitted parametric surfaces are fused into a geometric texture image (called the Bump image). A confidence and color texture image are also built. Our 3D scene representation is accurate yet memory efficient. Moreover, updating or editing the 3D model becomes trivial since it is reduced to manipulating 2D images. Our experimental results demonstrate the advantages of our proposed 3D representation through a concrete indoor scene reconstruction application.