Integrating Automated Range Registration with Multiview Geometry for the Photorealistic Modeling of Large-Scale Scenes

The photorealistic modeling of large-scale scenes, such as urban structures, requires a fusion of range sensing technology and traditional digital photography. This paper presents a system that integrates automated 3D-to-3D and 2D-to-3D registration techniques, with multiview geometry for the photorealistic modeling of urban scenes. The 3D range scans are registered using our automated 3D-to-3D registration method that matches 3D features (linear or circular) in the range images. A subset of the 2D photographs are then aligned with the 3D model using our automated 2D-to-3D registration algorithm that matches linear features between the range scans and the photographs. Finally, the 2D photographs are used to generate a second 3D model of the scene that consists of a sparse 3D point cloud, produced by applying a multiview geometry (structure-from-motion) algorithm directly on a sequence of 2D photographs. The last part of this paper introduces a novel algorithm for automatically recovering the rotation, scale, and translation that best aligns the dense and sparse models. This alignment is necessary to enable the photographs to be optimally texture mapped onto the dense model. The contribution of this work is that it merges the benefits of multiview geometry with automated registration of 3D range scans to produce photorealistic models with minimal human interaction. We present results from experiments in large-scale urban scenes.     

融合对象性和视觉显著度的单目图像2D转3D

受对象性测度和视觉显著度的启发,提出一种适用于单目图像2D转3D的对象窗深度中心环绕分布假设,给出融合对象性测度和视觉显著度的单目图像深度估计算法。首先计算图像的视觉显著度并将其映射成深度;其次在图像上随机采样若干个窗,并计算这些窗的对象性测度;再次,定义一个能量函数用于度量深度和对象性测度对彼此的影响程度,并通过迭代优化的方法改进深度和对象性测度的估计结果;最后,根据深度信息进行3D视频合成。实验结果表明,融入对象性测度信息后,显著改进了基于视觉显著度2D转3D的深度估计质量,保证了估计深度在对象边界处的不连续过渡和其他区域的平滑过渡。     

基于弹簧质点模型的二维/三维映射算法*

提出了一种用于服装设计的二维/三维映射算法.该算法基于弹簧质点变形模型,服装裁剪片二维到三维映射及三维到二维映射可以在该模型中得到统一的实现.在服装裁剪片二维到三维的映射过程中,二维裁剪片被放置在人体模型附近的初始位置,在缝合力的作用下,裁剪片自动变形并缝合到人体模型上.在服装裁剪片三维到二维映射的过程中,三维裁剪片被初始映射到指定的平面内,在弹性变形力的作用下,逐步变形并得到最终的二维裁剪片.在进行服装裁剪片二维到三维映射的过程中,考虑了干涉检验的问题.     

3D video components generation using object tracking technique

2D-to-3D conversion that would be a solution of the lack of 3D contents has been a worthy and challenging research field. In this paper, we propose a computer interactive conversion method to capture components which is used to generate 3D sequences. First, we divide the key frame into foreground and background, and then label the objects by convenient computer interactive operation. Depth information of objects is labeled after segmentation. Second, we use object tracking technique which synthesizes the advantages of kernel-based mean shift tracker and contour tracker to accomplish object depth capture for non-key frame. Finally, all the 3D information is prepared to render 3D sequences. After all, we propose our future work direction: a 2D-to-3D system which can generate 3D sequence interactively.     

基于雷达复杂1维距离数据的3维重建算法

基于刚体目标3维运动过程中的几何不变性,可以利用目标上多个散射点在单天线雷达1维距离像序列中的1维距离数据,重建出目标未知的3维结构和运动路径。针对此1维到3维的几何重构问题,提出了一种雷达刚体目标结构和运动的3维重建算法,该算法可利用散射点复杂的1维距离数据进行重建,并且采用非线性优化技术实现了对目标重建参数的捆绑调整(bundle adjustment)。另外,该算法中引入了目标的平移模型,使目标的平移参数能够与旋转参数一并求解,从而避免了距离对准操作的误差对重建精度的影响。仿真实验结果表明,由于重建数据中散射点的数量以及目标3维运动的丰富性得到了显著提高,尤其是最优化技术在算法中的成功应用,重建算法的鲁棒性得到了有效增强。     

基于邻域扩展的半自动2D转3D方法*

半自动2D转3D是解决当前3D影视内容匮乏的重要途径,其核心是将用户分配的稀疏深度转换成稠密深度。现有方法大多借助局部邻域进行深度插值,忽略了图像的全局约束关系,因而难以准确恢复深度图的对象边界。针对该问题,提出邻域扩展的最优化深度插值方法。首先引入邻域的邻域,建立邻域扩展的最优化深度插值能量模型;其次在相似的像素点与其邻域加权深度平均值的差异近似相等的假设条件下,将深度插值能量模型的最优化问题转换成一个稀疏线性方程组的求解问题。实验结果表明,与当前流行的半自动2D转3D方法相比,本文方法估计的深度图PSNR更高,同时增强了深度图的对象边界质量。     

Continuous Generalized Procrustes analysis

Two-dimensional shape models have been successfully applied to solve many problems in computer vision, such as object tracking, recognition, and segmentation. Typically, 2D shape models are learned from a discrete set of image landmarks (corresponding to projection of 3D points of an object), after applying Generalized Procustes Analysis (GPA) to remove 2D rigid transformations. However, the standard GPA process suffers from three main limitations. Firstly, the 2D training samples do not necessarily cover a uniform sampling of all the 3D transformations of an object. This can bias the estimate of the shape model. Secondly, it can be computationally expensive to learn the shape model by sampling 3D transformations. Thirdly, standard GPA methods use only one reference shape, which can might be insufficient to capture large structural variability of some objects.To address these drawbacks, this paper proposes continuous generalized Procrustes analysis (CGPA). CGPA uses a continuous formulation that avoids the need to generate 2D projections from all the rigid 3D transformations. It builds an efficient (in space and time) non-biased 2D shape model from a set of 3D model of objects. A major challenge in CGPA is the need to integrate over the space of 3D rotations, especially when the rotations are parameterized with Euler angles. To address this problem, we introduce the use of the Haar measure. Finally, we extended CGPA to incorporate several reference shapes. Experimental results on synthetic and real experiments show the benefits of CGPA over GPA.     

一种基于多视立体视觉的多视图直线匹配方法

提出了一种基于多视立体视觉（Multiple view stereo,MVS）进行多视图直线匹配的方法. 本文方法首先利用MVS所得到的三维点云及其可见性信息,建立三维点与图像直线的对应关系. 根据此对应关系,为每条图像直线建立由一个三维点集和一个三维单位向量构成的描述子,用以衡量图像直线之间的相似性及一致性. 之后,本文方法以所有图像直线为顶点建立一个图,并引入了图谱分析来获取统一的顶点距离度量. 最后,本方法对DBSCAN聚类算法进行了修改,并用修改后的算法从图谱分析结果中获取可靠的直线匹配. 实验显示,本方法比已有方法更加鲁棒,并且有更高的准确率.     

潜艇航行训练模拟器视景仿真中的三维场景显示

该文主要论述了在潜艇航行训练模拟器三维视景仿真中，把三维模型投影到屏幕坐标系当中的方法。该方法是利用Direct3D的几何管道，使模型中顶点依次经过世界变换、视变换、投影变换和屏蔽载减最后输出到光栅上。     

A genetic algorithm embedded with a concise chromosome representation for distributed and flexible job-shop scheduling problems

This paper proposes a genetic algorithm \(GA\_JS\) for solving distributed and flexible job-shop scheduling (DFJS) problems. A DFJS problem involves three scheduling decisions: (1) job-to-cell assignment, (2) operation-sequencing, and (3) operation-to-machine assignment. Therefore, solving a DFJS problem is essentially a 3-dimensional solution space search problem; each dimension represents a type of decision. The \(GA\_JS\) algorithm is developed by proposing a new and concise chromosome representation \({\varvec{S}}_{{\varvec{JOB}}}\), which models a 3-dimensional scheduling solution by a 1-dimensional scheme (i.e., a sequence of all jobs to be scheduled). That is, the chromosome space is 1-dimensional (1D) and the solution space is 3-dimensional (3D). In \(GA\_JS\), we develop a 1D-to-3D decoding method to convert a 1D chromosome into a 3D solution. In addition, given a 3D solution, we use a refinement method to improve the scheduling performance and subsequently use a 3D-to-1D encoding method to convert the refined 3D solution into a 1D chromosome. The 1D-to-3D decoding method is designed to obtain a “good” 3D solution which tends to be load-balanced. In contrast, the refinement and 3D-to-1D encoding methods of a 3D solution provides a novel way (rather than by genetic operators) to generate new chromosomes, which are herein called shadow chromosomes. Numerical experiments indicate that \(GA\_JS\) outperforms the IGA developed by De Giovanni and Pezzella (Eur J Oper Res 200:395–408, 2010), which is the up-to-date best-performing genetic algorithm in solving DFJS problems.     

Efficient 3D reconstruction for face recognition

Face recognition with variant pose, illumination and expression (PIE) is a challenging problem. In this paper, we propose an analysis-by-synthesis framework for face recognition with variant PIE. First, an efficient two-dimensional (2D)-to-three-dimensional (3D) integrated face reconstruction approach is introduced to reconstruct a personalized 3D face model from a single frontal face image with neutral expression and normal illumination. Then, realistic virtual faces with different PIE are synthesized based on the personalized 3D face to characterize the face subspace. Finally, face recognition is conducted based on these representative virtual faces. Compared with other related work, this framework has following advantages: (1) only one single frontal face is required for face recognition, which avoids the burdensome enrollment work; (2) the synthesized face samples provide the capability to conduct recognition under difficult conditions like complex PIE; and (3) compared with other 3D reconstruction approaches, our proposed 2D-to-3D integrated face reconstruction approach is fully automatic and more efficient. The extensive experimental results show that the synthesized virtual faces significantly improve the accuracy of face recognition with changing PIE.     

A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization

Registration of 3D data is a key problem in many applications in computer vision, computer graphics and robotics. This paper provides a family of minimal solutions for the 3D-to-3D registration problem in which the 3D data are represented as points and planes. Such scenarios occur frequently when a 3D sensor provides 3D points and our goal is to register them to a 3D object represented by a set of planes. In order to compute the 6 degrees-of-freedom transformation between the sensor and the object, we need at least six points on three or more planes. We systematically investigate and develop pose estimation algorithms for several configurations, including all minimal configurations, that arise from the distribution of points on planes. We also identify the degenerate configurations in such registrations. The underlying algebraic equations used in many registration problems are the same and we show that many 2D-to-3D and 3D-to-3D pose estimation/registration algorithms involving points, lines, and planes can be mapped to the proposed framework. We validate our theory in simulations as well as in three real-world applications: registration of a robotic arm with an object using a contact sensor, registration of planar city models with 3D point clouds obtained using multi-view reconstruction, and registration between depth maps generated by a Kinect sensor.     

一种将羽毛球比赛的2D视频转换到3D视频的算法

文中提出一种羽毛球比赛的2D视频转换到3D视频的算法。在这类视频中,前景是最受关注的部分,准确地从背景中提取出前景对象是获取深度图的关键。文中采用一种改进的图割算法来获取前景,并根据场景结构构建背景深度模型,获取背景深度图;在背景深度图的基础上,根据前景与镜头之间的距离关系为前景对象进行深度赋值,从而得到前景深度图。然后,融合背景深度图和前景深度图,得到完整的深度图。最后,通过基于深度图像的虚拟视点绘制技术DIBR来获取用于3D显示的立体图像对。实验结果表明,最终生成的立体图像对具有较好的3D效果。     

Non-rigid 3D shape tracking from multiview video

We present a fast and efficient non-rigid shape tracking method for modeling dynamic 3D objects from multiview video. Starting from an initial mesh representation, the shape of a dynamic object is tracked over time, both in geometry and topology, based on multiview silhouette and 3D scene flow information. The mesh representation of each frame is obtained by deforming the mesh representation of the previous frame towards the optimal surface defined by the time-varying multiview silhouette information with the aid of 3D scene flow vectors. The whole time-varying shape is then represented as a mesh sequence which can efficiently be encoded in terms of restructuring and topological operations, and small-scale vertex displacements along with the initial model. The proposed method has the ability to deal with dynamic objects that may undergo non-rigid transformations and topological changes. The time-varying mesh representations of such non-rigid shapes, which are not necessarily of fixed connectivity, can successfully be tracked thanks to restructuring and topological operations employed in our deformation scheme. We demonstrate the performance of the proposed method both on real and synthetic sequences.     

2D/3D Isometric Transformation Using Spring-Mass System

２Ｄ／３ＤＩｓｏｍｅｔｒｉｃＴｒａｎｓｆｏｒｍａｔｉｏｎＵｓｉｎｇＳｐｒｉｎｇ＿ＭａｓＳｙｓｔｅｍＦａｎＪｉｎ１ＭａｔｈｅｗＭ．Ｆ．Ｙｕｅｎ２ＱｉｆｕＷａｎｇ１ＺｈｏｕＪｉ１１ＴｈｅＣＡＤＣｅｎｔｅｒｏｆＴｈｅＨｕａｚｈｏｎｇＵｎｖｉｅｒｓｉｔｙｏｆ...     

Shape and Motion Reconstruction from 3D-to-1D Orthographically Projected Data via Object-Image Relations

This paper describes an invariant-based shape- and motion reconstruction algorithm for 3D-to-1D orthographically projected range data taken from unknown viewpoints. The algorithm exploits the object-image relation that arises in echo-based range data and represents a simplification and unification of previous work in the literature. Unlike one proposed approach, this method does not require uniqueness constraints, which makes its algorithmic form independent of the translation removal process (centroid removal, range alignment, etc.). The new algorithm, which simultaneously incorporates every projection and does not use an initialization in the optimization process, requires fewer calculations and is more straightforward than the previous approach. Additionally, the new algorithm is shown to be the natural extension of the approach developed by Tomasi and Kanade for 3D-to-2D orthographically projected data and is applied to a realistic inverse synthetic aperture radar imaging scenario, as well as experiments with varying amounts of aperture diversity and noise.     

基于电场通量特征的3D模型检索方法

对基于内容的三维模型检索的应用背景和现有算法进行了研究。结合物理学的基本原理,提出了一种静电场电通量描述子,在三维模型的模拟电场中设置测试球壳,计算各个测试球壳内部的电通量,用以描述三维模型。该描述子具有平移、旋转和缩放不变的特征,并对噪声有较强的鲁棒性。实验证明该描述子具有较强的检索能力。     

A Direct Manipulation Technique for Specifying 3D Object Transformations with a 2D Input Device

A mechanism is presented for direct manipulation of 3D objects with a conventional 2D input device, such as a mouse. The user can define and modify a model by graphical interaction on a 3D perspective or parallel projection. A gestural interface technique enables the specification of 3D transformations (translation, rotation and scaling) by 2D pick and drag operations. Interaction is not restricted to single objects but can be applied to compound objects as well. The method described in this paper is an easy-to-understand 3D input technique which does not require any special hardware and is compatible with the designer's mental model of object manipulation.