2D vs. 3D Deformable Face Models: Representational Power,Construction, and Real-Time Fitting

Model-based face analysis is a general paradigm with applications that include face recognition, expression recognition, lip-reading, head pose estimation, and gaze estimation. A face model is first constructed from a collection of training data, either 2D images or 3D range scans. The face model is then fit to the input image(s) and the model parameters used in whatever the application is. Most existing face models can be classified as either 2D (e.g. Active Appearance Models) or 3D (e.g. Morphable Models). In this paper we compare 2D and 3D face models along three axes: (1) representational power, (2) construction, and (3) real-time fitting. For each axis in turn, we outline the differences that result from using a 2D or a 3D face model.     

Reconstructing 3D trajectories of independently moving objects using generic constraints

The 3D reconstruction of scenes containing independently moving objects from uncalibrated monocular sequences still poses serious challenges. Even if the background and the moving objects are rigid, each reconstruction is only known up to a certain scale, which results in a one-parameter family of possible, relative trajectories per moving object with respect to the background. In order to determine a realistic solution from this family of possible trajectories, this paper proposes to exploit the increased linear coupling between camera and object translations that tends to appear at false scales. An independence criterion is formulated in the sense of true object and camera motions being minimally correlated. The increased coupling at false scales can also lead to the destruction of special properties such as planarity, periodicity, etc. of the true object motion. This provides us with a second, ‘non-accidentalness’ criterion for the selection of the correct motion among the one-parameter family.     

基于二阶锥规划无标定参照物的手眼标定

为了克服传统机器人手眼标定方法求解手眼关系及机器人坐标系与世界坐标系方位关系对标定参照物的依赖,提出一种基于二阶锥规划的无标定参照物手眼标定改进方法,并搭建相关实验系统进行验证。首先,利用运动恢复结构算法解算定义在一个尺度因子基础上的相机运动矩阵;然后,引入对偶四元数理论参数化标定方程中的旋转矩阵和平移向量;最后,通过二阶锥规划方法同时求解相机运动矩阵尺度因子、手眼关系及机器人坐标系与世界坐标系方位关系的最优解。仿真和实测结果表明,在没有标定参照物作为测量基准的情况下,标定结果中旋转参数相对误差为3.998%,平移参数相对误差为0.117%。与其他标定方法相比,该方法提高了无标定参照物模式下机器人手眼标定精度,扩展了手眼标定方法的应用范围。     

Four Points in Two or Three Calibrated Views: Theory and Practice

Suppose two perspective views of four world points are given and that the intrinsic parameters are known but the camera poses and the world point positions are not. We prove that the epipole in each view is then constrained to lie on a curve of degree ten. We derive the equation for the curve and establish many of the curve’s properties. For example, we show that the curve has four branches through each of the image points and that it has four additional points on each conic of the pencil of conics through the four image points. We show how to compute the four curve points on each conic in closed form. We show that orientation constraints allow only parts of the curve and find that there are impossible configurations of four corresponding point pairs. We give a novel algorithm that solves for the essential matrix given three corresponding points and one of the epipoles. We then use the theory to create the most efficient solution yet to the notoriously difficult problem of solving for the pose of three views given four corresponding points. The solution is a search over a one-dimensional parameter domain, where each point in the search can be evaluated in closed form. The intended use for the solution is in a hypothesise-and-test architecture to solve for structure and motion. Prepared through collaborative participation in the Robotics Consortium sponsored by the U.S. Army Research Laboratory under the collaborative Technology Alliance Program, Cooperative Agreement DAAD19-01-2-0012. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon. Now at the Center for Visualization and Virtual Environments, Computer Science Department, University of Kentucky. Formerly at the Research School of Information Sciences and Engineering, Australian National University.     

Visual Modeling with a Hand-Held Camera

In this paper a complete system to build visual models from camera images is presented. The system can deal with uncalibrated image sequences acquired with a hand-held camera. Based on tracked or matched features the relations between multiple views are computed. From this both the structure of the scene and the motion of the camera are retrieved. The ambiguity on the reconstruction is restricted from projective to metric through self-calibration. A flexible multi-view stereo matching scheme is used to obtain a dense estimation of the surface geometry. From the computed data different types of visual models are constructed. Besides the traditional geometry- and image-based approaches, a combined approach with view-dependent geometry and texture is presented. As an application fusion of real and virtual scenes is also shown.     

Geotensity: Combining Motion and Lighting for 3D Surface Reconstruction

This paper is about automatically reconstructing the full 3D surface of an object observed in motion by a single static camera. Based on the two paradigms, structure from motion and linear intensity subspaces, we introduce the geotensity constraint that governs the relationship between four or more images of a moving object. We show that it is possible in theory to solve for 3D Lambertian surface structure for the case of a single point light source and propose that a solution exists for an arbitrary number point light sources. The surface may or may not be textured. We then give an example of automatic surface reconstruction of a face under a point light source using arbitrary unknown object motion and a single fixed camera.     

大规模室外图像3维重建技术研究进展

基于图像的3维重建旨在从一组2维多视角图像中精确地恢复真实场景的几何形状,是计算机视觉和摄影测量中基础且活跃的研究课题,具有重要的理论研究意义和应用价值,在智慧城市、虚拟旅游、数字遗产保护、数字地图和导航等领域有着广泛应用。随着图像采集系统（智能手机、消费级数码相机和民用无人机等）的普及和互联网的高速发展,通过搜索引擎可以获取大量关于某个室外场景的互联网图像。利用这些图像进行高效鲁棒准确的3维重建,为用户提供真实感知和沉浸式体验已经成为研究热点,引发了学术界和产业界的广泛关注,涌现了多种方法。深度学习的出现为大规模室外图像的3维重建提供了新的契机。首先阐述大规模室外图像3维重建的基本串行过程,包括图像检索、图像特征点匹配、运动恢复结构和多视图立体。然后从传统方法和基于深度学习的方法两个角度,分别系统全面地回顾大规模室外图像3维重建技术在各重建子过程中的发展和应用,总结各子过程中适用于大规模室外场景的数据集和评价指标。最后介绍现有主流的开源和商业3维重建系统以及国内相关产业的发展现状。     

基于三维重建的虚拟现实类鸟飞行系统

随着虚拟现实(virtual reality, VR)技术的飞速发展,各种应用层出不穷。然而目前大多应用仅限于定点的静态全景展示及游览,无法利用VR化不可能为可能的优越性。本文提出了一种融合VR技术与三维重建算法的类鸟飞行交互模拟系统。该系统搭建了结构稳定的硬件控制平台用以改变实际姿态,同时通过软件仿真了虚拟环境中鸟类的飞行,结合硬件驱动及软件模拟实现了深层次的飞行模拟。现有的虚拟环境大多通过人工建模、激光扫描仪或无人机(unmanned aerial vehicle, UAV)航拍构造,其中建模存在费时费力的缺点,激光扫描仪则无法适应大规模重建,而UAV成本高昂且需专业培训。本文引入了基于图像分簇的PMVS (patch based multi-view stereopesis)算法,只需输入特定场景的图片即可利用计算机自动恢复出三维结构,不仅快速而且可对任意场景重构。使用户在足不出户的情况下即可在世界各地体验飞行,为旅游事业及VR产业提供了一种新的可能。     

基于直接法的视觉同时定位与地图构建技术综述

视觉同时定位与地图构建（V-SLAM）在机器人、无人机导航、自动驾驶等领域有着广泛的研究。直接法V-SLAM基于环境亮度不变性假设,跟踪相机的位姿并构建环境地图。针对直接法V-SLAM,首先简述其基本原理;然后分析、比较几种具有代表性的直接法V-SLAM系统;最后讨论直接法的优缺点和发展趋势,并进行了总结和展望。     

Multi-View AAM Fitting and Construction

Active Appearance Models (AAMs) are generative, parametric models that have been successfully used in the past to model deformable objects such as human faces. The original AAMs formulation was 2D, but they have recently been extended to include a 3D shape model. A variety of single-view algorithms exist for fitting and constructing 3D AAMs but one area that has not been studied is multi-view algorithms. In this paper we present multi-view algorithms for both fitting and constructing 3D AAMs. Fitting an AAM to an image consists of minimizing the error between the input image and the closest model instance; i.e. solving a nonlinear optimization problem. In the first part of the paper we describe an algorithm for fitting a single AAM to multiple images, captured simultaneously by cameras with arbitrary locations, rotations, and response functions. This algorithm uses the scaled orthographic imaging model used by previous authors, and in the process of fitting computes, or calibrates, the scaled orthographic camera matrices. In the second part of the paper we describe an extension of this algorithm to calibrate weak perspective (or full perspective) camera models for each of the cameras. In essence, we use the human face as a (non-rigid) calibration grid. We demonstrate that the performance of this algorithm is roughly comparable to a standard algorithm using a calibration grid. In the third part of the paper, we show how camera calibration improves the performance of AAM fitting. A variety of non-rigid structure-from-motion algorithms, both single-view and multi-view, have been proposed that can be used to construct the corresponding 3D non-rigid shape models of a 2D AAM. In the final part of the paper, we show that constructing a 3D face model using non-rigid structure-from-motion suffers from the Bas-Relief ambiguity and may result in a “scaled” (stretched/compressed) model. We outline a robust non-rigid motion-stereo algorithm for calibrated multi-view 3D AAM construction and show how using calibrated multi-view motion-stereo can eliminate the Bas-Relief ambiguity and yield face models with higher 3D fidelity. Electronic Supplementary Material The online version of this article () contains supplementary material, which is available to authorized users.