基于多幅图像的三维重建

针对当前三维重建系统中大量的交互和频繁的失真,根据运动进行物体三维重建的原理,设计了一个简便易操作的三维重建系统.首先经过相机标定,立体匹配点,进而运用双视点三维重建原理即三角测量得到三维坐标:然后运用运动的连续性,综合多视点下的三维信息结合图像特征点的二维局部属性进行优化,最后经过三角剖分和纹理渲染得到物体真实的形状.与传统的三维重建系统相比,该系统能够在用户交互很少的情况下进行三维重建,且能重建出完整的低失真的三维模型.     

基于立体视觉的三维视频轨迹跟踪

介绍了一种基于立体视觉原理的三维视频轨迹跟踪方法，该方法在实验空间安置了四个邻近的特征点，利用立体视觉原理对空间中特征点的运动轨迹进行分析，跟踪和预测，并根据四个特征点的位置关系计算出特征点所在平面的空间姿态参数。本文利用这种方法，将特征点安置到机器人手臂本体及操作对象上，实现了对机器人手臂动作轨迹的跟踪，预测以及碰撞检测和碰撞预警的功能。实验表明，该方法具有原理简单，计算是小，实时性强的特点。     

ST-Rec3D：基于结构和目标感知的三维重建

基于视图的三维重建旨在从二维图像恢复出其对应的三维形状。现有方法主要通过编码器-解码器结构,结合二元交叉熵函数及其变形,完成三维重建,取得较好的重建结果。然而,编码器在编码过程中缺乏对输入视图的结构感知能力,造成重建的三维模型几何细节不准确;以二元交叉熵函数为主的损失函数在体素分布不均衡的情况下,目标感知能力较差,导致其重建结果存在断裂、缺失等不完整性问题。针对此类问题,提出了一种具有结构和目标感知能力的三维重建网络(ST-Rec3D),以单视图或多视图为输入,由粗到细地重建出三维模型;结合注意力机制提出了一种具有空间结构感知能力的编码器,即结构编码器,以充分捕捉输入视图中的空间结构信息,有效感知重建物体的几何细节;将IoU损失引入到三维体素模型重建中,在体素分布不均衡的情况下,精准感知目标物体,确保重建物体的完整性和准确性。在ShapeNet和Pix3D数据集上的对比结果表明,ST-Rec3D在单视图和多视图上重建的三维模型的完整性和准确性均优于当前方法。     

Learning stratified 3D reconstruction

Stratified 3D reconstruction, or a layer-by-layer 3D reconstruction upgraded from projective to affine, then to the final metric reconstruction, is a well-known 3D reconstruction method in computer vision. It is also a key supporting technology for various well-known applications, such as streetview, smart3D, oblique photogrammetry. Generally speaking, the existing computer vision methods in the literature can be roughly classified into either the geometry-based approaches for spatial vision or the learning-based approaches for object vision. Although deep learning has demonstrated tremendous success in object vision in recent years, learning 3D scene reconstruction from multiple images is still rare, even not existent, except for those on depth learning from single images. This study is to explore the feasibility of learning the stratified 3D reconstruction from putative point correspondences across images, and to assess whether it could also be as robust to matching outliers as the traditional geometry-based methods do. In this study, a special parsimonious neural network is designed for the learning. Our results show that it is indeed possible to learn a stratified 3D reconstruction from noisy image point correspondences, and the learnt reconstruction results appear satisfactory although they are still not on a par with the state-of-the-arts in the structure-from-motion community due to largely its lack of an explicit robust outlier detector such as random sample consensus (RANSAC). To the best of our knowledge, our study is the first attempt in the literature to learn 3D scene reconstruction from multiple images. Our results also show that how to implicitly or explicitly integrate an outlier detector in learning methods is a key problem to solve in order to learn comparable 3D scene structures to those by the current geometry-based state-of-the-arts. Otherwise any significant advancement of learning 3D structures from multiple images seems difficult, if not impossible. Besides, we even speculate that deep learning might be, in nature, not suitable for learning 3D structure from multiple images, or more generally, for solving spatial vision problems.     

Symmetry-seeking models and 3D object reconstruction

We propose models of 3D shape which may be viewed as deformable bodies composed of simulated elastic material. In contrast to traditional, purely geometric models of shape, deformable models are active—their shapes change in response to externally applied forces. We develop a deformable model for 3D shape which has a preference for axial symmetry. Symmetry is represented even though the model does not belong to a parametric shape family such as (generalized) cylinders. Rather, a symmetry-seeking property is designed into internal forces that constrain the deformations of the model. We develop a framework for 3D object reconstruction based on symmetry-seeking models. Instances of these models are formed from monocular image data through the action of external forces derived from the data. The forces proposed in this paper deform the model in space so that the shape of its projection into the image plane is consistent with the 2D silhouette of an object of interest. The effectiveness of our approach is demonstrated using natural images.     

高效鲁棒三维结构化重建

目的结构化重建,即从离散点云或者原始三角网格中提取几何平面并将其拼接成紧凑的参数化3维模型,一直是计算机图形学领域中极具挑战性的问题。现有方法通常面临着两个挑战。一是传统的形状检测方法通常只考虑物体的局部特征,无法保证整体结果的准确性。二是现有的形状拼接算法往往受限于计算复杂度,从而只能处理由一百多个几何平面组成的物体,极大地限制了算法的应用场景。针对这些问题,提出了一种快速、鲁棒的结构化重建算法以自动地生成轻量的多边形网格。方法提出了一种多源区域增长算法,全局地从原始3维数据中提取特征平面。该策略保证了原始数据可以被正确地聚类到所属的平面区域。为了减轻几何平面分割3维空间带来的计算负担,采用了一种基于二叉空间分割树的结构将3维空间切分为凸多面体。提出了一种基于光线射击的马尔可夫能量方程以提取水密、无自相交的多边形网格。结果实验结果表明,本文方法可以在没有并行化方案的标准计算机上处理由上万个几何平面组成的物体。与传统的全相交分割相比,本文方法得到的多面体数目和运行时间都降低了至少两个数量级,总耗时可控制在5 s/万点以内。此外,模型化简前后的均方根误差平均控制在1%以内,面片化简比例控...     

Consistency constraints and 3D building reconstruction

Virtual architectural (indoor) scenes are often modeled in 3D for various types of simulation systems. For instance, some authors propose methods dedicated to lighting, heat transfer, acoustic or radio-wave propagation simulations. These methods rely in most cases on a volumetric representation of the environment, with adjacency and incidence relationships. Unfortunately, many buildings data are only given by 2D plans and the 3D needs varies from one application to another. To face these problems, we propose a formal representation of consistency constraints dedicated to building interiors and associated with a topological model. We show that such a representation can be used for: (i) reconstructing 3D models from 2D architectural plans (ii) detecting automatically geometrical, topological and semantical inconsistencies (iii) designing automatic and semi-automatic operations to correct and enrich a 2D plan. All our constraints are homogeneously defined in 2D and 3D, implemented with generalized maps and used in modeling operations. We explain how this model can be successfully used for lighting and radio-wave propagation simulations.     

Visual creation of inhabited 3D environments

The creation of virtual reality applications and 3D environments is a complex task that requires good programming skills and expertise in computer graphics and many other disciplines. The complexity increases when we want to include complex entities such as virtual characters and animate them. In this paper we present a system that assists in the tasks of setting up a 3D scene and configuring several parameters affecting the behavior of virtual entities like objects and autonomous virtual humans. Our application is based on a visual programming paradigm, supported by a semantic representation, an ontology for virtual environments. The ontology allows us to store and organize the components of a 3D scene, together with the knowledge associated with them. It is also used to expose functionalities in the given 3D engine. Based on a formal representation of its components, the proposed architecture provides a scalable VR system. Using this system, non-experts can set up interactive scenarios with minimum effort; no programming skills or advanced knowledge is required.     

多轮廓线的三维形体重构技术研究与实现

实际应用中,三维重构经常面对的不是直接的体数据信息,而是一序列的二维轮廓线数据,因此基于轮廓线的三维重构研究有着极其重要的实用价值。在多轮廓线的三维形体重构中,轮廓对应、轮廓拼接、分叉处理和末端轮廓线的封闭处理等是其关键技术。提出了三维重构中每一个实现步骤具体的解决方案。针对轮廓线绕向问题提出了夹角和检测法,有效避免了轮廓多边形的绕向误判;对轮廓线一对多分叉问题提出了按周长比率解决问题的思路;在末端轮廓线的三角剖分算法中提出了最大张角三角形方法,减少了三角剖分的计算量,达到了在各种形态轮廓线条件下能够实现正确的拼接。实现结果表明,轮廓线拼接过程中每个步骤的解决方法是正确有效的,相较于其他实现方法通用性更强。     

Numerical solution of 2D and 3D elliptic-type interface models with regular interfaces

Engineering with Computers - In the current paper, two numerical methods are proposed for the numerical solution of two- and three-dimensional elliptic partial differential equations (PDEs) with...     

基于视觉的三维重建技术综述*

基于视觉的三维重建仍然存在较大局限性。通过介绍基于视觉的三维重建技术的主要方法及其研究现状,对各种方法优缺点进行了比较分析,期望能够对该领域有较全面的把握,进一步明确未来的研究方向。     

FPGA加速三维CT图像重建

针对三维图像重建的经典算法(FDK算法)在FPGA上的加速,提出了并行无等待流水线的实现方法。实验结果表明,该方法获得了较高的加速比。     

Visual discomfort of stereoscopic 3D videos: Influence of 3D motion

Visual discomfort is one of the most frequent complaints of the viewers while watching 3D images and videos. Large disparity and large amount of motion are two main causes of visual discomfort. To quantify this influence, three objectives are set in this paper. The first one is the comparative analysis on the influence of different types of motion, i.e., static stereoscopic image, planar motion and in-depth motion, on visual discomfort. The second one is the investigation on the influence factors for each motion type, for example, the disparity offset, the disparity amplitude and velocity. The third one is to propose an objective model for visual discomfort. Thirty-six synthetic stereoscopic video stimuli with different types of motion are used in this study. In the subjective test, an efficient paired comparison method called Adaptive Square Design (ASD) was used to reduce the number of comparisons for each observer and keep the results reliable. The experimental results showed that motion does not always induce more visual discomfort than static conditions. The in-depth motion generally induces more visual discomfort than the planar motion. The relative disparity between the foreground and the background, and the motion velocity are identified as main factors for visual discomfort. According to the subjective results, an objective model for comparing visual discomfort induced by different types of motion is proposed which shows high correlation with the subjective perception.     

Robust3D: a robust 3D face reconstruction application

In the process of reconstructing a historical event such as a rock concert only from video, the reconstruction of faces and expressions of the musicians is obviously important. However, in the process of rebuilding appearance, because of the low quality of the video of the recorded concert, the result of the reconstruction may be far from the real appearance. In this paper, a robust 3D face reconstruction application is described that can be applied to a video recording. The application first uses DeblurGAN program to run anti-ambiguity calculation and removes the ambiguity in the concert video. Then, the super-resolution program is used to enlarge every frame of the concert video by four times, thus making every frame of the video clearer. Finally, the 3D faces are obtained after 3D reconstruction of the processed video frames via the 3DMM_CNN program.

     

基于三维规则数据场的快速光线投射法

光线投射法是体绘制技术中的经典算法。该算法原理简单、明了,并能产生高质量的显示图像,但由于所有体素参与了图像绘制,运行速度慢,从而极大制约了其在交互可视化中的运用。论文利用光线穿过三维规则数据场时与某一方向平面簇交点的快速确定及光线投射方向确定后两相邻平面间所截取线段为定长的特点,简化了光线亮度积分公式,从而在图像质量不减低的情况下,快速进行图像绘制。     

3D Visual Odometry for Road Vehicles

This paper describes a method for estimating the vehicle global position in a network of roads by means of visual odometry. To do so, the ego-motion of the vehicle relative to the road is computed using a stereo-vision system mounted next to the rear view mirror of the car. Feature points are matched between pairs of frames and linked into 3D trajectories. Vehicle motion is estimated using the non-linear, photogrametric approach based on RANSAC. This iterative technique enables the formulation of a robust method that can ignore large numbers of outliers as encountered in real traffic scenes. The resulting method is defined as visual odometry and can be used in conjunction with other sensors, such as GPS, to produce accurate estimates of the vehicle global position. The obvious application of the method is to provide on-board driver assistance in navigation tasks, or to provide a means for autonomously navigating a vehicle. The method has been tested in real traffic conditions without using prior knowledge about the scene nor the vehicle motion. We provide examples of estimated vehicle trajectories using the proposed method and discuss the key issues for further improvement.     

视景仿真中三维地球的建模

给出了一种基于Blue Marble纹理和GTOPO30高程数据的三维地球建模方法，可消除地表各分块间的缝隙。地表形状模拟精度高，结合多分辨率纹理和高程，实现了对地球的变视角和缩放操作，在飞行模拟、导弹发射、卫星绕地球飞行等涉及到地球大场景或全球视景的仿真中被广泛采用。该建模方法对类似的建模提供了一种新思路，有较高的通用性和实用性。     

切片级管线表面三维重建

在比较表面重建中的面绘制法和体绘制法以及面绘制中的切片级和体素级两种表面重建方法、分析表面重建流程的基础上,提出了一种计算型值点集的方法,它在计算弯道部分的管线时较为简便。该方法通过二次曲面求管线表面轮廓方程进而求得管线的型值点集。之后用三角网格对型值点集进行拼接来获得管线的表面模型,最后利用图形学的方法对管线表面模型进行绘制,达到管线表面三维重建的目的。文末利用Java3D给出了一个具体的实现。     

Visual Attention for Rendered 3D Shapes

Understanding the attentional behavior of the human visual system when visualizing a rendered 3D shape is of great importance for many computer graphics applications. Eye tracking remains the only solution to explore this complex cognitive mechanism. Unfortunately, despite the large number of studies dedicated to images and videos, only a few eye tracking experiments have been conducted using 3D shapes. Thus, potential factors that may influence the human gaze in the specific setting of 3D rendering, are still to be understood. In this work, we conduct two eye‐tracking experiments involving 3D shapes, with both static and time‐varying camera positions. We propose a method for mapping eye fixations (i.e., where humans gaze) onto the 3D shapes with the aim to produce a benchmark of 3D meshes with fixation density maps, which is publicly available. First, the collected data is used to study the influence of shape, camera position, material and illumination on visual attention. We find that material and lighting have a significant influence on attention, as well as the camera path in the case of dynamic scenes. Then, we compare the performance of four representative state‐of‐the‐art mesh saliency models in predicting ground‐truth fixations using two different metrics. We show that, even combined with a center‐bias model, the performance of 3D saliency algorithms remains poor at predicting human fixations. To explain their weaknesses, we provide a qualitative analysis of the main factors that attract human attention. We finally provide a comparison of human‐eye fixations and Schelling points and show that their correlation is weak.