Use of non-photorealistic rendering and photometric stereo in making bas-reliefs from photographs

Automatic bas-relief generation from 2D photographs potentially has applications to coinage, commemorative medals and souvenirs. However, current methods are not yet ready for real use in industry due to insufficient artistic effect, noticeable distortion, and unbalanced contrast. We previously proposed a shape-from-shading (SFS) based method to automatically generate bas-reliefs from single frontal photographs of human faces; however, suppression of unwanted details remained a problem. Here, we give experimental results showing how incorporating non-photorealistic rendering (NPR) into our previous framework enables us to both suppress unwanted detail, and yet also emphasize important features. We have consider an alternative approach to recovering relief shape, based on photometric stereo instead of SFS for surface orientation estimation. This can effectively reduce the computational time.     

Photometric stereo using LCD displays

This paper considers the problem of shape-from-shading using nearby extended light sources. The paper reviews a number of methods that employ nearby illuminants, and describes a new technique that assumes a rectangular planar nearby distributed uniform isotropic illuminant. It is shown that such a light source illuminating a small Lambertian surface patch is equivalent to a single isotropic point light source at infinity, in the absence of shadowing. A closed-form solution is given for the equivalent point light source direction in terms of the illuminant corner locations. Equivalent point light sources can be obtained for distinct illuminant patterns allowing standard photometric stereo algorithms to be used. An extension is given to the case of a rectangular planar illuminant with arbitrary radiance distribution. Experimental results are shown demonstrating the application of the theory to photometric stereo using illumination from a LCD computer monitor. Details on the photometric calibration of the illumination source and image acquisition device are provided.     

Bayesian shape estimation: Shape-from-shading and photometric stereo revisited

We present a Bayesian approach to the machine vision processes of shape-from-shading and photometric stereo, also considering the associated question of the detection of shape discontinuities. The shape reconstruction problem is formulated as a maximum a posteriori (MAP) estimation from probability distributions of Gibbs form, and is solved via simulated annealing. In shape-from-shading, our formulation leads to a constrained optimization problem, where the constraints come from the image irradiance equation and from the incorporation of the necessary boundary conditions. In photometric stereo, we are able to estimate shape directly from degraded input images. We also propose an edge-detection algorithm that works cooperatively with the reconstruction process, employing the shape estimates to locate the discontinuities of the reconstructed surface. We show results of the application of our framework both to synthetic and to real imagery.     

Real-time recovery of moving 3D faces for emerging applications

Existing face imaging systems are not suitable to meet the face representation and recognition demands for emerging applications in areas such as interactive gaming, enhanced learning environments and directed advertising. This is mainly due to the poor capture and characterisation of facial data that compromises their spatial and temporal precision. For emerging applications it is not only necessary to have a high level of precision for the representation of facial data, but also to characterise dynamic faces as naturally as possible and in a timely manner. This study proposes a new framework for capturing and recovering dynamic facial information in real-time at significantly high order of spatial and temporal accuracy to capture and model subtle facial changes for enhanced realism in 3D face visualisation and higher precision for face recognition applications. We also present a novel, fast, and robust correspondence mapping approach for 3D registration of moving 3D faces.     

一种基于二维图像的三维仿真方法研究

针对二维的图像数据,综合运用计算机视觉、计算机图像处理和计算机辅助几何设计理论与技术,提出了一种实用的三维仿真方法.采用阴影恢复形状的方法获取三维形貌信息,利用图像分割技术提取物体的二维轮廓信息,通过三维形貌信息与二维轮廓信息的融合技术建立物体的数字化模型,通过B样条的万法实现数字化模型的重构.实例表明,方法可行有效,能够有效地去除图像的背景信息,改善重构模型的分辨力,为基于二维图像的三维仿真技术的广泛应用提供了条件.     

A multimodal approach for 3D face modeling and recognition using 3D deformable facial mask

We present a multimodal approach for face modeling and recognition. The algorithm uses three cameras to capture stereo images, two frontal and one profile, of the face. 2D facial features are extracted from one of the frontal images and a dense disparity map is computed from the two frontal images. Using the extracted 2D features and their corresponding disparities, we compute their 3D coordinates. We next align a low resolution 3D mesh model to the 3D features, re-project its vertices onto the frontal 2D image and adjust its profile silhouette vertices using the profile view image. We increase the resolution of the resulting 2D model at its center region to obtain a facial mask model covering distinctive features of the face. The 2D coordinates of the vertices, along with their disparities, result in a deformed 3D mask model specific to a given subject’s face. Our method integrates information from the extracted facial features from the 2D image modality with information from the 3D modality obtained from the stereo images. Application of the models in 3D face recognition, for 112 subjects, validates the algorithm with a 95% identification rate and 92% verification rate at 0.1% false acceptance rate.

Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition

We present a fully automated algorithm for facial feature extraction and 3D face modeling from a pair of orthogonal frontal and profile view images of a person's face taken by calibrated cameras. The algorithm starts by automatically extracting corresponding 2D landmark facial features from both view images, then compute their 3D coordinates. Further, we estimate the coordinates of the features that are hidden in the profile view based on the visible features extracted in the two orthogonal face images. The 3D coordinates of the selected feature points obtained from the images are used first to align, then to locally deform the corresponding facial vertices of the generic 3D model. Preliminary experiments to assess the applicability of the resulted models for face recognition show encouraging results.     

Constructing a 3D individualized head model from two orthogonal views

A new scheme for constructing a 3D individualized head model automatically from only a side view and the front view of the face is presented. The approach instantiates a generic 3D head model based on a set of the individual's facial features extracted by a local maximum-curvature tracking (LMCT) algorithm that we have developed. A distortion vector field that deforms the generic model to that of the individual is computed by correspondence matching and interpolation. The input of the two facial images are blended and texture-mapped onto the 3D head model. Arbitrary views of a person can be generated from two orthogonal images and can be implemented efficiently on a low-cost, PC-based platform.     

Estimation of search-space in 3D coronary artery reconstruction using angiographic biplane images

The three-dimensional reconstruction of heart arteries, using two radiographic images, is an ambiguous problem and results in multiple solutions. The correct solution can only be identified using a priori information. This work proposes a method to determine the number of possible solutions, which may be useful for the development of algorithms to solve this ambiguity problem. Our method proceeds by generating and counting all the possible solutions. Using real data, it was found that the number of solutions is moderate and an extensive search could be a feasible option at certain levels of the search process. The currently available computation power should be able to handle this solution space.     

Shadow Graphs and 3D Texture Reconstruction

We present methods for recovering surface height fields such as geometric details of 3D textures by incorporating shadow constraints. We introduce shadow graphs which give a new graph-based representation for shadow constraints. It can be shown that the shadow graph alone is sufficient to solve the shape-from-shadow problem from a dense set of images. Shadow graphs provide a simpler and more systematic approach to represent and integrate shadow constraints from multiple images. To recover height fields from a sparse set of images, we propose a method for integrated shadow and shading constraints. Previous shape-from-shadow algorithms do not consider shading constraints while shape-from-shading usually assumes there is no shadow. Our method is based on collecting a set of images from a fixed viewpoint as a known light source changes its position. It first builds a shadow graph from shadow constraints from which an upper bound for each pixel can be derived if the height values of a small number of pixels are initialized correctly. Finally, a constrained optimization procedure is designed to make the results from shape-from-shading consistent with the height bounds derived from the shadow constraints. Our technique is demonstrated on both synthetic and real imagery.     

用单数码相机实现物体表面的三维重建

根据由运动重建物体结构的原理,设计了一个简便易操作的三维重建系统,具体做法是:先用张氏标定法求得内参数矩阵,然后在两个不同的未知位置拍摄物体得到两幅图像,经立体匹配后,利用图像特征点的对应关系求解基本矩阵和本质矩阵,分解本质矩阵获得两个拍摄位置确定的摄像机运动参数(旋转矩阵和平移向量),进而求出相机在两个位置的投影矩阵,最后用三角法计算出物体表面特征点的三维坐标并在OpenGL中重建物体表面.和传统的立体视觉系统相比,本系统只需要一台数码相机和平面方格模板就可以实现三维重建,因此适用于普通相机用户.     

Kinect动作捕捉技术在三维动画教学中的应用与实践

针对传统三维角色动画制作成本高、时间长的问题,文章介绍了一种应用Kinect动作捕捉技术实现高效制作三维角色动画的方法。该方法借助Kinect体感摄像机捕捉真人的动作生成骨骼关节的关键帧数据并输出bvh动作路径文件,然后把bvh动作路径数据导入C4D软件中,就可以驱动角色模型完成角色动画的制作。把这种方法运用到三维角色动画教学实践中,有利于提高学生学习的兴趣和效率。     

平行双目视觉系统的三维重建研究

随着图像处理,模式识别的快速发展,人们对双目视觉系统和三维创建越来越重视.本文研究和设计一套双目视觉系统目的是获得生动的立体图像.从二维图像恢复到三维图像实现了三维物体的可视化.     

Robust 3D face recognition based on resolution invariant features

A novel resolution invariant local feature based method is proposed for 3D face recognition. Scale space extrema on shape index images and texture images are detected and matched, through which resolution and noise insensitive face matching is achieved without complex preprocessing and normalization. An outlier removal strategy is designed to eliminate incorrect matching points while keeping relevant ones. Six different scale invariant similarity measures are proposed and fused at the score level, which increases the robustness against expression variations. Systematical experiments are conducted on the FRGC v2.0 database, achieving in the neutral vs. all experiment a verification rate of 90.7% with un-normalized similarity scores, and 96.3% with normalized similarity scores at False Acceptance Rate (FAR) of 0.1%, and 96.2% rank-1 identification rate, which are comparable to the state of the art, and promising considering the significantly reduced preprocessing requirement.     

应用神经网络隐式视觉模型进行立体视觉的三维重建

针对传统的基于精确数学模型的立体视觉方法过程繁琐的不足,提出一种应用BP神经网络隐式视觉模型进行三维重建的算法。该算法将多个标定平面放置在有效视场内,用神经网络模拟立体视觉由两个二维图像重建三维几何的过程,经过网络训练建模后,无须摄像机标定即可进行三维重建。仿真实验结果证明,该算法比较简单,且能保持较高的精度。     

Static and dynamic 3D facial expression recognition: A comprehensive survey

Automatic facial expression recognition constitutes an active research field due to the latest advances in computing technology that make the user's experience a clear priority. The majority of work conducted in this area involves 2D imagery, despite the problems this presents due to inherent pose and illumination variations. In order to deal with these problems, 3D and 4D (dynamic 3D) recordings are increasingly used in expression analysis research. In this paper we survey the recent advances in 3D and 4D facial expression recognition. We discuss developments in 3D facial data acquisition and tracking, and present currently available 3D/4D face databases suitable for 3D/4D facial expressions analysis as well as the existing facial expression recognition systems that exploit either 3D or 4D data in detail. Finally, challenges that have to be addressed if 3D facial expression recognition systems are to become a part of future applications are extensively discussed.     

An efficient 3D face recognition approach using local geometrical signatures

This paper presents a computationally efficient 3D face recognition system based on a novel facial signature called Angular Radial Signature (ARS) which is extracted from the semi-rigid region of the face. Kernel Principal Component Analysis (KPCA) is then used to extract the mid-level features from the extracted ARSs to improve the discriminative power. The mid-level features are then concatenated into a single feature vector and fed into a Support Vector Machine (SVM) to perform face recognition. The proposed approach addresses the expression variation problem by using facial scans with various expressions of different individuals for training. We conducted a number of experiments on the Face Recognition Grand Challenge (FRGC v2.0) and the 3D track of Shape Retrieval Contest (SHREC 2008) datasets, and a superior recognition performance has been achieved. Our experimental results show that the proposed system achieves very high Verification Rates (VRs) of 97.8% and 88.5% at a 0.1% False Acceptance Rate (FAR) for the “neutral vs. nonneutral” experiments on the FRGC v2.0 and the SHREC 2008 datasets respectively, and 96.7% for the ROC III experiment of the FRGC v2.0 dataset. Our experiments also demonstrate the computational efficiency of the proposed approach.     

Silhouette and stereo fusion for 3D object modeling

In this paper, we present a new approach to high quality 3D object reconstruction. Starting from a calibrated sequence of color images, the algorithm is able to reconstruct both the 3D geometry and the texture. The core of the method is based on a deformable model, which defines the framework where texture and silhouette information can be fused. This is achieved by defining two external forces based on the images: a texture driven force and a silhouette driven force. The texture force is computed in two steps: a multi-stereo correlation voting approach and a gradient vector flow diffusion. Due to the high resolution of the voting approach, a multi-grid version of the gradient vector flow has been developed. Concerning the silhouette force, a new formulation of the silhouette constraint is derived. It provides a robust way to integrate the silhouettes in the evolution algorithm. As a consequence, we are able to recover the contour generators of the model at the end of the iteration process. Finally, a texture map is computed from the original images for the reconstructed 3D model.     

Pose-invariant face recognition using a 3D deformable model

The paper proposes a novel, pose-invariant face recognition system based on a deformable, generic 3D face model, that is a composite of: (1) an edge model, (2) a color region model and (3) a wireframe model for jointly describing the shape and important features of the face. The first two submodels are used for image analysis and the third mainly for face synthesis. In order to match the model to face images in arbitrary poses, the 3D model can be projected onto different 2D viewplanes based on rotation, translation and scale parameters, thereby generating multiple face-image templates (in different sizes and orientations). Face shape variations among people are taken into account by the deformation parameters of the model. Given an unknown face, its pose is estimated by model matching and the system synthesizes face images of known subjects in the same pose. The face is then classified as the subject whose synthesized image is most similar. The synthesized images are generated using a 3D face representation scheme which encodes the 3D shape and texture characteristics of the faces. This face representation is automatically derived from training face images of the subject. Experimental results show that the method is capable of determining pose and recognizing faces accurately over a wide range of poses and with naturally varying lighting conditions. Recognition rates of 92.3% have been achieved by the method with 10 training face images per person.