光度立体三维重建算法

提出一种三维表面重建方法，以便快速准确地获得三维模型．首先获取控制光照和视角条件下的3幅图像，采用光度立体视觉的方法计算表面法向量，并在基于高度的全局优化和局部特征保持的基础上，通过金字塔式多尺度逼近，由表面法向量恢复三维表面信息．实验表明，这是一种简易快速的三维重建算法，在游戏三维建模、交互式动画等方面有着广泛的应用．     

Nonlinearities and Noise Reduction in 3-Source Photometric Stereo

1-D Leap-Frog (L. Noakes, J. Math. Australian Soc. A, Vol. 64, pp. 37–50, 1999) is an iterative scheme for solving a class of nonquadratic optimization problems. In this paper a 2-D version of Leap-Frog is applied to a non optimization problem in computer vision, namely the recovery (so far as possible) of an unknown surface from 3 noisy camera images. This contrasts with previous work on photometric stereo, in which noise is added to the gradient of the height function rather than camera images. Given a suitable initial guess, 2-D Leap-Frog is proved to converge to the maximum-likelihood estimate for the vision problem. Performance is illustrated by examples.     

Multispectral Photometric Stereo for Spatially-Varying Spectral Reflectances

International Journal of Computer Vision - Multispectral photometric stereo (MPS) aims at recovering the surface normal of a scene measured under multiple light sources with different...     

Design Issues for a Colour Photometric Stereo System

This paper presents the detailed performance analysis for a colour photometric stereo system proposed recently. The system recovers surface colour and surface normal for each surface patch separately, in the presence of highlights and shadows. The error analysis presented concerns every step of the algorithm, and it is based on the assumptions that errors may arise due to Gaussian image noise and errors in the accuracy with which the geometry of the illuminating set-up is known. The analysis is confirmed by experiments. The implications of this sensitivity analysis to the design of such a rig and the choice of threshold values for the algorithm are also discussed and recommendations to a designer are given.     

Self-calibrated visual servoing with respect to axial-symmetric 3D objects

A self-calibrated approach to visual servoing with respect to non-planar targets modeled through a pair of coaxial circles plus one point is discussed. Full calibration data (fixed internal parameters) are obtained from two views, and used to recover the Euclidean structure of an auxiliary virtual plane associated to the target, together with the relative pose of the camera. Pose disambiguation is achieved without requiring any real third view of the target. The approach benefits of an off-line planning strategy by which the camera follows a 3D helicoidal path around an arbitrarily chosen axis. A convenient choice for the helicoidal axis is found to be that of the target axis itself. Simulation results demonstrate that the approach is robust with respect to noise both in the off-line and on-line control phases.     

3D立体产业发展前景广阔

1月4日,电影史上的又一个具有标志性意义的里程碑——《阿凡达》（Avatar）在中国的上映,为我们展示了迄今为止3D影像及CG技术最为成功的应用。《阿凡达》的热映让观众体验到3D技术突破视觉极限带来的震撼感受的同时,也拉开了2010年3D立体时代的大幕。     

LED-Based Photometric Stereo: Modeling,Calibration and Numerical Solution

We conduct a thorough study of photometric stereo under nearby point light source illumination, from modeling to numerical solution, through calibration. In the classical formulation of photometric stereo, the luminous fluxes are assumed to be directional, which is very difficult to achieve in practice. Rather, we use light-emitting diodes to illuminate the scene to be reconstructed. Such point light sources are very convenient to use, yet they yield a more complex photometric stereo model which is arduous to solve. We first derive in a physically sound manner this model, and show how to calibrate its parameters. Then, we discuss two state-of-the-art numerical solutions. The first one alternatingly estimates the albedo and the normals, and then integrates the normals into a depth map. It is shown empirically to be independent from the initialization, but convergence of this sequential approach is not established. The second one directly recovers the depth, by formulating photometric stereo as a system of nonlinear partial differential equations (PDEs), which are linearized using image ratios. Although the sequential approach is avoided, initialization matters a lot and convergence is not established either. Therefore, we introduce a provably convergent alternating reweighted least-squares scheme for solving the original system of nonlinear PDEs. Finally, we extend this study to the case of RGB images.     

3D Deformable Super-Resolution for Multi-Camera 3D Face Scanning

Low-cost and high-accuracy 3D face measurement is becoming increasingly important in many computer vision applications including face recognition, facial animation, games, orthodontics and aesthetic surgery. In most cases fringe projection based systems are used to overcome the relatively uniform appearance of skin. These systems employ a structured light camera/projector device and require explicit user cooperation and controlled lighting conditions. In this paper, we propose a 3D acquisition solution with a 3D space-time non-rigid super-resolution capability, using three calibrated cameras coupled with a non calibrated projector device, which is particularly suited to 3D face scanning, i.e. rapid, easily movable and robust to ambient lighting variation. The proposed solution is a hybrid stereovision and phase-shifting approach, using two shifted patterns and a texture image, which not only takes advantage of stereovision and structured light, but also overcomes their weaknesses. The super-resolution scheme involves a shape+texture 3D non-rigid registration for 3D artifacts correction in the presence of small non-rigid deformations as facial expressions.     

Near-lighting Photometric Stereo for unknown scene distance and medium attenuation

Photometric Stereo in murky water is subject to light attenuation and near-field illumination, and the resulting image formation model is complex. Apart from the scene normals and albedo, the incident illumination varies per-pixel and it depends on the scene depth and the attenuation coefficient of the medium. When these are unknown, e.g. in a realistic scenario where a robotic platform explores an underwater scene (unknown shape and distance) within the dynamic subsea environment (unknown scattering level), Photometric Stereo becomes ambiguous. Previous approaches have tackled the problem by assuming distant-lighting and resorting to external hardware for estimating the unknown model variables. In our work, we show that the Photometric Stereo problem can be determined as soon as some additional constraints regarding the scene albedo and the presence of pixels with local intensity maxima within the image are incorporated into the optimization framework. Our proposed solution leads to effective Photometric Stereo and yields detailed 3D reconstruction of objects in murky water when the scene distance and the medium attenuation are unknown. We evaluate our work using both numerical simulations and real experiments in the controlled environment of a water tank and real port water using a remotely operated vehicle.     

Direct regularized surface reconstruction from gradients for Industrial Photometric Stereo

This paper addresses the issue of regularization in the surface reconstruction from gradients problem in Industrial Photometric Stereo. Regularization of the solution is a necessary step in an industrial environment, where algorithms must cope with non-Gaussian noise, such as outliers, or non-Lambertian textures such as corrosion. Introducing Tikhonov regularization into the global least squares solution suppresses the influence of outliers in the reconstruction. Viable methods should both minimize a global least squares cost function and also introduce some form of regularization into the solution; state-of-the-art methods to this end are grossly inefficient and are severely limited in the size of surface they can reconstruct. We present a new algorithm which can reconstruct a surface of 1200×1200$1200\times 1200$, (i.e., greater than 1 M-pixel) in a few seconds. This is orders of magnitude faster than state-of-the-art methods incorporating regularization, and hence presents the first method viable for regularized reconstructions in practical applications.     

一种鲁棒的基于光度立体视觉的表面重建方法

提出一种基于先进的凸优化技术的光度立体视觉重建框架. 首先通过鲁棒的主成分分析(Robust principle component analysis, RPCA)祛除图像噪声, 得到低秩矩阵和物体表面向量场, 然后再通过表面重建算法从向量场来恢复物体形状. 相对于先前的一些使用最小二乘或者一些启发式鲁棒技术的方法, 该方法使用了所有可用的信息, 可以同时修复数据中的丢失和噪声数据, 显示出了较高的计算效率以及对于大的稀疏噪声的鲁棒性. 实验结果表明, 本文提出的框架大大提高了在噪声存在情况下物体表面的重建精度.     

Using Stereo Matching with General Epipolar Geometry for 2D Face Recognition across Pose

Face recognition across pose is a problem of fundamental importance in computer vision. We propose to address this problem by using stereo matching to judge the similarity of two, 2D images of faces seen from different poses. Stereo matching allows for arbitrary, physically valid, continuous correspondences. We show that the stereo matching cost provides a very robust measure of similarity of faces that is insensitive to pose variations. To enable this, we show that, for conditions common in face recognition, the epipolar geometry of face images can be computed using either four or three feature points. We also provide a straightforward adaptation of a stereo matching algorithm to compute the similarity between faces. The proposed approach has been tested on the CMU PIE data set and demonstrates superior performance compared to existing methods in the presence of pose variation. It also shows robustness to lighting variation.     

Stereo geometry from 3D ego-motion streams

This paper addresses the problem of geometry determination of a stereo rig that undergoes general rigid motions. Neither known reference objects nor stereo correspondence are required. With almost no exception, all existing online solutions attempt to recover stereo geometry by first establishing stereo correspondences. We first describe a mathematical framework that allows us to solve for stereo geometry, i.e., the rotation and translation between the two cameras, using only motion correspondence that is far easier to acquire than stereo correspondence. Second, we show how to recover the rotation and present two linear methods, as well as a nonlinear one to solve for the translation. Third, we perform a stability study for the developed methods in the presence of image noise, camera parameter noise, and ego-motion noise. We also address accuracy issues. Experiments with real image data are presented. The work allows the concept of online calibration to be broadened, as it is no longer true that only single cameras can exploit structure-from-motion strategies; even the extrinsic parameters of a stereo rig of cameras can do so without solving stereo correspondence. The developed framework is applicable for estimating the relative three-dimensional (3D) geometry associated with a wide variety of mounted devices used in vision and robotics, by exploiting their scaled ego-motion streams.     

基于立体视觉的水下三维测量系统研究

随着我国海洋战略的提出,对于海洋观测技术和装备的需求日趋迫切。针对现有水下成像系统无法实现精确三维测量这一难题,该文提出了一种基于双目立体视觉原理的水下三维测量系统研究方法,并对其可行性进行了验证。针对水下成像过程存在的水体界面折射问题,该文提出了相应的相机成像模型及系统参数标定方法,建立了防水深度达 30 m 的双目水下测量及照明装置,并在水池、近海条件下进行了实地测试。实验结果显示,在水体条件较好的情况下,系统观测距离可达 8 m 以上,有效测量距离为 0.5～4.5 m,在 0.5 m 和 4.5 m 距离处的测量误差分别为 2 mm 和 20 mm。实验验证了 水下双目成像模型、立体标定、测量模型等方法的有效性和精确性,可为水下检修作业等海洋工程行业提供一种有效的三维测量技术手段。     

虚拟世界中立体声合成与播放技术研究

研究虚拟世界中多声道立体声合成与播放的简化算法。文中首先分析多声道立体声的合成原理,以及混声效果的评价标准及评价声源对整个声音所作贡献的公式,然后给出多声道立体声实时合成和播放的简化算法,整个算法简单容易实现。     

Anthropometric 3D Face Recognition

We present a novel anthropometric three dimensional (Anthroface 3D) face recognition algorithm, which is based on a systematically selected set of discriminatory structural characteristics of the human face derived from the existing scientific literature on facial anthropometry. We propose a novel technique for automatically detecting 10 anthropometric facial fiducial points that are associated with these discriminatory anthropometric features. We isolate and employ unique textural and/or structural characteristics of these fiducial points, along with the established anthropometric facial proportions of the human face for detecting them. Lastly, we develop a completely automatic face recognition algorithm that employs facial 3D Euclidean and geodesic distances between these 10 automatically located anthropometric facial fiducial points and a linear discriminant classifier. On a database of 1149 facial images of 118 subjects, we show that the standard deviation of the Euclidean distance of each automatically detected fiducial point from its manually identified position is less than 2.54 mm. We further show that the proposed Anthroface 3D recognition algorithm performs well (equal error rate of 1.98% and a rank 1 recognition rate of 96.8%), out performs three of the existing benchmark 3D face recognition algorithms, and is robust to the observed fiducial point localization errors.     

基于视频的三维人体运动跟踪

提出一种结合多种图像特征,在多摄像机环境下跟踪人体运动的方法．通过定义人体模型、摄像机投影模型以及相似性度量模型来得到优化框架下的目标函数,并使用牛顿-高斯优化算法对其进行求解．模拟数据和实际数据的实验表明,文中方法比仅仅使用灰度特征,跟踪结果得到了改善,对比实验结果也优于基于概率算法的退火粒子滤波．     

3D立体投影ViewSonic PJD6531w

ViewSonic在率先推出了令人印象深刻的全球第一款3D投影机PJD6210-3D之后,ViewSonic随后又相继发布了多款3D投影机新品,率先引领了业界的3D投影风潮。其中,同时兼具3D投影效果和短焦投影功能的PJD6381成为ViewSonic开拓3D立体投影市场的又一重大突破,一经问世即获得了2009     

面向三维街景重构的立体平行拼图自动生成算法

针对目前已有的多视点拼图技术只注重其二维展示效果,而未考虑三维重构的问题,提出一种面向三维街景重构并结合纹理与深度信息的立体平行拼图(SPM)技术及其生成算法.通过深度信息获得图像场景的平面拟合结果;以最大涵盖建筑物正面和侧面为选取标准,根据平面的参数与位置选出用于拼接的关键视图;基于双缝合线与泊松融合技术对相邻关键视图进行拼接和融合,得到平滑无缝的SPM.实验结果证明,SPM中的纹理拼图具有较好的二维街景展示效果,更主要的是其结合深度信息能够完成街景的三维重构,可为用户提供自由漫游的虚拟城市环境.     

Stereo vision system for precision dimensional inspection of 3D holes

This paper presents a machine vision system for the high-accuracy 3D measurement of holes on the surface of industrial components. This is a very important application for inline quality inspection in assembly plants. A CAD-based stereo vision approach is adopted. The introduction of several novel techniques enables the system to achieve high robustness in versatile industrial environments, rapid response, and accuracy below 0.1 mm. These are demonstrated by extensive experiments with synthetic and real data.Received: 10 May 2002, Accepted: 25 April 2003, Published online: 7 October 2003