Circular camera array system for 3‐D displays based on the reconstruction of parallax rays

Abstract— A circular camera system employing an image‐based rendering technique that captures light‐ray data needed for reconstructing three‐dimensional (3‐D) images by using reconstruction of parallax rays from multiple images captured from multiple viewpoints around a real object in order to display a 3‐D image of a real object that can be observed from multiple surrounding viewing points on a 3‐D display is proposed. An interpolation algorithm that is effective in reducing the number of component cameras in the system is also proposed. The interpolation and experimental results which were performed on our previously proposed 3‐D display system based on the reconstruction of parallax rays will be described. When the radius of the proposed circular camera array was 1100 mm, the central angle of the camera array was 40°, and the radius of a real 3‐D object was between 60 and 100 mm, the proposed camera system, consisting of 14 cameras, could obtain sufficient 3‐D light‐ray data to reconstruct 3‐D images on the 3‐D display.     

3D depth information extraction with omni-directional camera

This paper presents a novel 3D depth information extraction method without calibration. Firstly, this paper develops an omni-directional 3D camera system, which consists of a CCD camera, hyperbolic mirror, infrared laser diodes and diffractive of element (DOE). Secondly, a depth measurement model is proposed to obtain the 3D depth information. Finally, in order to calculate the speckle shift accurately between the reference image and the object image, a dot matrix pattern and sequence coding algorithm are designed to find the corresponding speckles in the two images. Experimental results show that the reconstructed depth data have a good correlation with the actual distance. The accuracy of the data is also found to be influenced by the distance between the object and the camera.     

基于视点相关透视纹理的矢量数据在三维地形上的叠加绘制

为了精确、高效地将传统二维矢量数据叠加到三维多分辨率地形上,提出一种基于视点相关透视纹理的叠加绘制算法.首先通过场景视点得到能充分逼近当前地形可视范围的透视投影,然后用该透视投影实时地将二维矢量数据绘制在与地形绑定的纹理上,同时在GPU中并行计算出纹理坐标.与传统的基于纹理的绘制方法相比,视点相关的透视投影能够提高纹理的像素利用率,从而减轻传统方法所产生的走样问题,提高矢量数据的绘制质量;且该算法不受地形几何数据的约束,能够与目前大多数多分辨率地形模型以及影像金字塔很好地结合在一起使用.     

Flow-based local optimization for image-to-geometry projection

The projection of a photographic data set on a 3D model is a robust and widely applicable way to acquire appearance information of an object. The first step of this procedure is the alignment of the images on the 3D model. While any reconstruction pipeline aims at avoiding misregistration by improving camera calibrations and geometry, in practice a perfect alignment cannot always be reached. Depending on the way multiple camera images are fused on the object surface, remaining misregistrations show up either as ghosting or as discontinuities at transitions from one camera view to another. In this paper we propose a method, based on the computation of Optical Flow between overlapping images, to correct the local misalignment by determining the necessary displacement. The goal is to correct the symptoms of misregistration, instead of searching for a globally consistent mapping, which might not exist. The method scales up well with the size of the data set (both photographic and geometric) and is quite independent of the characteristics of the 3D model (topology cleanliness, parametrization, density). The method is robust and can handle real world cases that have different characteristics: low level geometric details and images that lack enough features for global optimization or manual methods. It can be applied to different mapping strategies, such as texture or per-vertex attribute encoding.     

Texturing 3D Models of Real World Objects from Multiple Unregistered Photographic Views

As the efficiency of computer graphic rendering methods is increasing, generating realistic models is now becoming a limiting factor. In this paper we present a new technique to enhance already existing geometry models of real world objects with textures reconstructed from a sparse set of unregistered still photographs. The aim of the proposed technique is the generation of nearly photo-realistic models of arbitrarily shaped objects with minimal effort. In our approach, we require neither a prior calibration of the camera nor a high precision of the user's interaction. Two main problems have to be addressed of which the first is the recovery of the unknown positions and parameters of the camera. An initial estimate of the orientation is calculated from interactively selected point correspondences. Subsequently, the unknown parameters are accurately calculated by minimising a blend of objective functions in a 3D-2D projective registration approach. The key point of the proposed method of registration is a novel filtering approach which utilises the spatial information provided by the geometry model. Second, the individual images have to be combined yielding a set of consistent texture maps. We present a robust method to recover the texture from the photographs thereby preserving high spatial frequencies and eliminating artifacts, particularly specular highlights. Parts of the object not seen in any of the photographs are interpolated in the textured model. Results are shown for three complex example objects with different materials and numerous self-occlusions.     

基于摄影原理的三维纹理重建技术研究

讨论的是基于摄影原理的三维物体表面纹理的重建技术.该技术从国内外研究现状和主要不足出发,对三维物体表面纹理的采集和映射方法进行研究和改进,提出了较为合理的算法.纹理采集过程中分析了回转工作台的结构、纹理采集的原则并建立相关的坐标系;根据拍照和投影变换的原理建立三个坐标系对应点间的关系、推导出坐标变换关系式和转换矩阵;对模型表面微小面片法向量进行均匀化处理;通过计算面片的法向量与纹理采集方向向量的位置关系确定其最佳纹理,最后通过多角度映射达到三维物体表面纹理重建的目的.     

基于ToF相机的三维重建技术

针对利用ToF相机实现物体的重建提出一种新的三维重建算法,通过分析物体重建过程的特点,对KinectFusion的重建算法进行改进。点云匹配过程包括粗匹配和精匹配两个过程,最后进行全局优化,提高相机位姿估计的精度,从而得到更精确的三维重建结果。利用泊松重建算法重建物体表面,相比于TSDF算法,能够实现完整表面的重建。提出结合强度图对深度图进行多边滤波的算法以及一种新的补空洞法则,增强深度图像。多边滤波算法在PSNR和SSIM的评估中都优于双边滤波结果,提出的三维重建算法与KinectFusion三维重建结果对比,表面更完整,重建结果更优。     

基于硬件的实时地平映射

地平映射是用于计算凹凸映射表面阴影的一项技术.在可编程图形硬件上运用并扩展地平映射技术,力求更准确地表达凹凸表面的层次细节.采取在表面切空间对光照向量进行插值的方法,在多遍渲染算法的基础上,使用三维体纹理代替二维地平贴图,一遍渲染即得到较好的实时自阴影效果.     

一种真实物体表面反射属性采集与建模方法

为了快速获取物体表面反射属性数据和三维几何数据,研制了一套包括相机、光源、三维扫描仪、控制系统等的物体表面反射属性数据采集系统,在此基础上,提出一种单材质物体表面反射属性建模方法.该方法根据物体反射特性对漫反射与镜面反射分别建模,能够逼真地构造出物体表面反射属性模型,并且保证了参数求解的稳定性.最后设计并实现了反射属性模型编辑工具.实验结果表明,利用该编辑工具可以实时、可视地调整模型参数,从而满足用户对反射属性模型编辑的需求.     

Real-time visuomotor update of an active binocular head

In order for a binocular head to perform optimal 3D tracking, it should be able to verge its cameras actively, while maintaining geometric calibration. In this work we introduce a calibration update procedure, which allows a robotic head to simultaneously fixate, track, and reconstruct a moving object in real-time. The update method is based on a mapping from motor-based to image-based estimates of the camera orientations, estimated in an offline stage. Following this, a fast online procedure is presented to update the calibration of an active binocular camera pair. The proposed approach is ideal for active vision applications because no image-processing is needed at runtime for the scope of calibrating the system or for maintaining the calibration parameters during camera vergence. We show that this homography-based technique allows an active binocular robot to fixate and track an object, whilst performing 3D reconstruction concurrently in real-time.     

An uncalibrated lightfield acquisition system

Acquisition of image data for lightfield usually requires an expensive, complex and bulky setup. In this paper, we describe a simple method of acquiring the image data set. The method requires a normal handheld video camera, which is taken around the object to be rendered. We employ homography from the viewing/camera plane to the lightfield plane for obtaining the ray intersections with the lightfield planes. The computations involved are simple and make the method suitable for online lightfield acquisition.     

基于几何与图像混合绘制中的快速WARP变换算法研究

基于几何与图像的混合绘制中，3D Warp算法以严格的数学变换为基础，从而能够保证准确的投影关系，但该算法在实时绘制阶段需进行大量的数学运算，故其时间复杂度较高，该文提出了一种新的快速Warp变换算法，算法以3D Warp算法为基础，采用了崭新的投影过程，从而使时间复杂度较3D Warp算法有较大幅度的下降（降低约7．51倍）。同时，该算法是一种流水结构，能够有效利用现有的加速硬件，而无需改变图形硬件的体系结构。     

RGBD融合明暗恢复形状的全视角三维重建技术研究

为了高效、高精度、低成本地实现对物体的全视角三维重建, 提出一种使用深度相机融合光照约束实现全视角三维重建的方法。该重建方法中,在进行单帧重建时采用RGBD深度图像融合明暗恢复形状(Shape from shading,SFS)的重建方法, 即在原有的深度数据上加上额外的光照约束来优化深度值; 在相邻两帧配准时, 采用快速点特征直方图（Fast point feature histograms, FPFH）特征进行匹配并通过随机采样一致性（Random sample consensus, RANSAC）滤除错误的匹配点对求解粗配准矩阵, 然后通过迭代最近点（Iterative closest point, ICP）算法进行精配准得出两帧间的配准矩阵; 在进行全视角的三维重建时, 采用光束平差法优化相机位姿, 从而消除累积误差使首尾帧完全重合, 最后融合生成一个完整的模型。该方法融入了物体表面的光照信息,因此生成的三维模型更为光顺,也包含了更多物体表面的细节信息,提高了重建精度;同时该方法仅通过单张照片就能在自然光环境下完成对多反射率三维物体的重建,适用范围更广。本文方法的整个实验过程通过手持深度相机就能完成,不需要借助转台,操作更加方便。     

基于体平滑的体绘制纹理伪边界消除算法

直接体绘制能够清楚的显示三维数据场的内部信息,是科学可视化中非常重要的一类方法。其中,基于二维纹理映射的三维数据直接体绘制方法具有绘制速度快、可交互性强的优点。其基本思路是将三维数据体在时间或深度方向形成一组水平纹理切片,通过这些切片的纹理贴图,实现三维数据体的体绘制,在交互性和资源消耗之间取得了较好的平衡。本文针对基于二维纹理映射的直接体绘制方法中在透明与不透明边界产生阶梯状条纹的伪边界问题,提出了一种基于体平滑的算法。该算法通过在透明数据与不透明数据的边界进行体平滑,使采样过程中缺失的数据表现到抽样的切片上,从而在最终图像生成阶段淡化甚至消除阶梯状条纹伪边界。实验结果表明,相对于传统二维纹理映射方法,本算法实现体绘制效果平滑,提高了绘制效果。     

3D object model reconstruction from image sequence based on photometric consistency in volume space

This paper presents a system that can reconstruct a photorealistic 3D object model from an image sequence captured at arbitrary viewpoints. The whole system consists of four steps: camera calibration, volumetric modeling, polygonal model formation and texture mapping. We adopt the shape-from-silhouette approach for volumetric modeling. There are two common types of object surface that are difficult to reconstruct—textureless surface and concave surface. To tackle the problems, we propose to perform the volumetric modeling based on the constraints of viewpoint proximity and photometric consistency in the volume space. The volumetric model is converted to the mesh model for efficient manipulation. Finally, the texture map is generated from the image sequence to give the 3D model a photorealistic appearance. Some reconstructed object models are presented to demonstrate the superior performance of our system as compared with the conventional modeling technique based on the photo-consistency in the image space.     

基于Kinect的人体三维重建方法

通过Kinect体感仪,实现人体三维重建.使用Kinect体感仪,扫描获取人体三维数据,利用深度数据转换算法实现二维顶点的三维化,再通过红外相机姿态跟踪算法进行顶点集配准,求解出相机每次的相对位移与转动角度,实现相机姿态跟踪,并将每次拍摄到的点集转换到同一全局坐标系下,使用晶格化显示集成算法将点云集成到提前划分好精度及尺寸的体素晶格中,最后利用投影映射算法获得可视化的人体三维立体模型.使用Kinect体感仪及三脚架等辅助设备方便快捷地获取人体三维重建结果,并通过3D打印技术对模型进行输出.该研究实现了人体三维重建中人体扫描、处理、重建、输出全流程.     

基于特征轮廓的灰度图像定位三维物体方法

讨论了一种基于特征轮廓的从三维灰度图像确定三维物体位置和姿态的方法，该方法首先建立物体的三维网页模型，检测模型上的特征点，并建立该物体的特征轮廓模型，然后检测输入图像中物体上的特征点，形成特征轮廓，并与特征轮廓模型相匹配，就可得到该物体在三维空间中的姿态；最后使用最小二乘法对物体进行精确定位，实验证明，该方法在物体遮挡情况下不是很严重时，可以快速精确地从灰度图像定位三维物体。     

基于数字近景摄影测量的若干关键技术研究

研究了基于数字近景摄影的空间点坐标的测量方法;首先在物体表面粘贴两类标记点,然后通过对标记点进行识别和检测来确定标志点的空间位置,同时,对相关的编码与识别、标记点的亚像素中心定位、照相机标定与空间定向、非编码点的自动匹配等关键技术进行了研究,最后,给出了检测实例并进行了精度验证;实验表明,该方法操作简单,检测范围大,可满足三维物体特别是大面积物体空间检测的需求。     

单像机有源形状恢复方法研究

物体表面三维形状恢复是计算机视觉的一个重要研究内容.给出了一种利用网格结 构光,采用单像机恢复物体表面三维形状的方法.提出了一种新的定标方法和投影模板检测 算法,并构造完成了有源三维重建实验系统.结果表明该方法能快速、准确地恢复物体形状.     

Computing multiscale curve and surface skeletons of genus 0 shapes using a global importance measure

We present a practical algorithm for computing robust, multiscale curve and surface skeletons of 3D objects. Based on a model which follows an advection principle, we assign to each point on the skeleton a part of the object surface, called the collapse. The size of the collapse is used as a uniform importance measure for the curve and surface skeleton, so that both can be simplified by imposing a single threshold on this intuitive measure. The simplified skeletons are connected by default, without special precautions, due to the monotonicity of the importance measure. The skeletons possess additional desirable properties: They are centered, robust to noise, hierarchical, and provide a natural skeleton-to-boundary mapping. We present a voxel-based algorithm that is straightforward to implement and simple to use. We illustrate our method on several realistic 3D objects.