Efficient constrained local model fitting for non-rigid face alignment

Active appearance models (AAMs) have demonstrated great utility when being employed for non-rigid face alignment/tracking. The “simultaneous” algorithm for fitting an AAM achieves good non-rigid face registration performance, but has poor real time performance (2–3 fps). The “project-out” algorithm for fitting an AAM achieves faster than real time performance (>200 fps) but suffers from poor generic alignment performance. In this paper we introduce an extension to a discriminative method for non-rigid face registration/tracking referred to as a constrained local model (CLM). Our proposed method is able to achieve superior performance to the “simultaneous” AAM algorithm along with real time fitting speeds (35 fps). We improve upon the canonical CLM formulation, to gain this performance, in a number of ways by employing: (i) linear SVMs as patch-experts, (ii) a simplified optimization criteria, and (iii) a composite rather than additive warp update step. Most notably, our simplified optimization criteria for fitting the CLM divides the problem of finding a single complex registration/warp displacement into that of finding N simple warp displacements. From these N simple warp displacements, a single complex warp displacement is estimated using a weighted least-squares constraint. Another major advantage of this simplified optimization lends from its ability to be parallelized, a step which we also theoretically explore in this paper. We refer to our approach for fitting the CLM as the “exhaustive local search” (ELS) algorithm. Experiments were conducted on the CMU MultiPIE database.     

LDA算法及其在人脸识别中的应用

线性特征提取在人脸识别中的应用非常广泛,LDA是其主要方法之一,它基于Fisher 判别准则,然而,当人脸训练样本数小于人脸样本向量的维数时,变换矩阵将无法直接得到,因此线性判别分析过程失效。采用了一种改进的基于Fisher 准则的LDA方法,针对小样本问题提出了一种有效地解决类内散布矩阵奇异的方法,而且用ORL人脸数据进行了实验验证。实验证明该方法在正确识别率方面表现突出。     

Voxel-based 3D face reconstruction and its application to face recognition using sequential deep learning

Multimedia Tools and Applications - In this paper, a novel 3D face reconstruction technique is proposed along with a sequential deep learning-based framework for face recognition. It uses the...     

基于改进三维形变模型的三维人脸重建和密集人脸对齐方法

针对现在广泛使用的三维形变模型表达能力不够,导致重建出的三维人脸模型泛化性能不佳的问题,提出了一种在姿态、表情和光照未知的条件下的基于单张人脸图片的三维人脸重建和密集人脸对齐的新方法。首先,通过卷积神经网络对现有的三维形变模型进行改进,以提高三维人脸模型的表达能力;然后,基于人脸光滑性和图像相似性,在特征点和像素层面提出新的损失函数,并使用弱监督学习训练卷积神经网络模型;最后,通过训练出的网络模型进行三维人脸重建和密集人脸对齐。实验结果表明,对于三维人脸重建任务,所提模型在AFLW2000-3D上实现了2.25的归一化平均误差;对于密集人脸对齐任务,所提模型在AFLW2000-3D和AFLW-LFPA上分别实现了3.80和3.34的归一化平均误差。与原始使用三维形变模型的方法相比,所提模型在三维人脸重建和密集人脸对齐上的归一化平均误差分别降低了7.4%和7.8%。针对不同光照环境以及角度的人脸图片,该网络模型的重建准确,鲁棒性好,且具有较高的三维人脸重建和密集人脸对齐质量。     

基于改进三维形变模型的三维人脸重建和密集人脸对齐方法

针对现在广泛使用的三维形变模型表达能力不够，导致重建出的三维人脸模型泛化性能不佳的问题，提出了一种在姿态、表情和光照未知的条件下的基于单张人脸图片的三维人脸重建和密集人脸对齐的新方法。首先，通过卷积神经网络对现有的三维形变模型进行改进，以提高三维人脸模型的表达能力；然后，基于人脸光滑性和图像相似性，在特征点和像素层面提出新的损失函数，并使用弱监督学习训练卷积神经网络模型；最后，通过训练出的网络模型进行三维人脸重建和密集人脸对齐。实验结果表明，对于三维人脸重建任务，所提模型在AFLW2000-3D上实现了2.25的归一化平均误差；对于密集人脸对齐任务，所提模型在AFLW2000-3D和AFLW-LFPA上分别实现了3.80和3.34的归一化平均误差。与原始使用三维形变模型的方法相比，所提模型在三维人脸重建和密集人脸对齐上的归一化平均误差分别降低了7.4%和7.8%。针对不同光照环境以及角度的人脸图片，该网络模型的重建准确，鲁棒性好，且具有较高的三维人脸重建和密集人脸对齐质量。     

结合多尺度融合特征和残差注意力机制的联合三维人脸重建及密集对齐算法

针对三维人脸重建和密集对齐算法精度不足的问题,引入密集连接的多尺度特征融合模块和残差注意力机制设计了一种性能强大的网络.在编码器结构前,引入密集连接的多尺度特征融合模块获得多尺度融合特征,使编码器获得更丰富的信息;在解码器模块中引入残差注意力机制,强化网络对重要特征的关注同时抑制不必要的噪声.实验结果表明,相较其他算法,该算法取得了较显著的改进;相对PRNet,该算法以更少的参数量在各项指标上取得7.7％～12.1％的性能提升.     

Dense 3D face alignment from 2D video for real-time use

To enable real-time, person-independent 3D registration from 2D video, we developed a 3D cascade regression approach in which facial landmarks remain invariant across pose over a range of approximately 60°. From a single 2D image of a person's face, a dense 3D shape is registered in real time for each frame. The algorithm utilizes a fast cascade regression framework trained on high-resolution 3D face-scans of posed and spontaneous emotion expression. The algorithm first estimates the location of a dense set of landmarks and their visibility, then reconstructs face shapes by fitting a part-based 3D model. Because no assumptions are required about illumination or surface properties, the method can be applied to a wide range of imaging conditions that include 2D video and uncalibrated multi-view video. The method has been validated in a battery of experiments that evaluate its precision of 3D reconstruction, extension to multi-view reconstruction, temporal integration for videos and 3D head-pose estimation. Experimental findings strongly support the validity of real-time, 3D registration and reconstruction from 2D video. The software is available online at http://zface.org.     

3D face reconstruction and dense alignment with a new generated dataset

Face alignment and reconstruction are classical problems in the computer vision field, one of the greatest difficulties of which is the limited number of facial images with landmark points. The 300 W-LP dataset is the most commonly used for the existing methods of single-view 3D Morphable Model (3DMM)-based reconstruction; however, the model performance is limited by the small variety of facial images in this dataset. In this work, a 3D facial image dataset with landmark points generated by the rotate-and-render method is proposed. The key innovation of the proposed method is that the back-and-forth rotation of faces in 3D space and then re-rendering them to the 2D plane can provide strong self-supervision. The recent advances in 3D face modeling and high-resolution generative adversarial networks (GANs) are leveraged to constitute the blocks. To obtain more precise facial landmark points, the 3D dense face alignment (3DDFA) model is used to label the generated images and filter the landmark points. Finally, the 3DDFA model is retrained using the proposed dataset, and an improved result is achieved.     

Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition

We present a fully automated algorithm for facial feature extraction and 3D face modeling from a pair of orthogonal frontal and profile view images of a person's face taken by calibrated cameras. The algorithm starts by automatically extracting corresponding 2D landmark facial features from both view images, then compute their 3D coordinates. Further, we estimate the coordinates of the features that are hidden in the profile view based on the visible features extracted in the two orthogonal face images. The 3D coordinates of the selected feature points obtained from the images are used first to align, then to locally deform the corresponding facial vertices of the generic 3D model. Preliminary experiments to assess the applicability of the resulted models for face recognition show encouraging results.     

A robust face super-resolution algorithm and its application in low-resolution face recognition system

Multimedia Tools and Applications - In real-world surveillance scenario, the face recognition (FR) systems pose a lot of challenges due to the captured low-resolution (LR) and noisy probe images. A...     

Robust3D: a robust 3D face reconstruction application

In the process of reconstructing a historical event such as a rock concert only from video, the reconstruction of faces and expressions of the musicians is obviously important. However, in the process of rebuilding appearance, because of the low quality of the video of the recorded concert, the result of the reconstruction may be far from the real appearance. In this paper, a robust 3D face reconstruction application is described that can be applied to a video recording. The application first uses DeblurGAN program to run anti-ambiguity calculation and removes the ambiguity in the concert video. Then, the super-resolution program is used to enlarge every frame of the concert video by four times, thus making every frame of the video clearer. Finally, the 3D faces are obtained after 3D reconstruction of the processed video frames via the 3DMM_CNN program.

     

DBSCAN算法在公路选线中的应用

基于密度的聚类算法是聚类分析算法中的一种主要技术,它对空间数据库聚类有着很好的性能,然而,对大规模数据库聚类时,DBSCAN算法需要大量的内存支持并伴随着I/O开销.提出了一种带有矢量性的密度聚类算法,具有约束聚类方向,减少候选点的特点.以地理信息系统(GIS)为应用背景,成功应用于高速公路选线,得到了良好的效果.     

改进的显式形状回归人脸特征点定位算法

针对显式形状回归（ESR）人脸特征点定位精度低的问题，提出了改进的显式形状回归人脸特征点定位算法。首先定位出三点人脸形状代替人脸检测框作为初始形状的映射标准来得到更精确的初始人脸形状，然后采用像素块特征代替像素特征对抗光照变化来提高算法的鲁棒性，最后采用多假设融合策略代替平均法对多个定位结果进行最佳融合来进一步提高算法的定位精度。仿真实验结果表明，在LFPW、HELEN和300-W人脸库上，与显式形状回归算法相比，定位精度分别提高了7.96%、5.36%和1.94%。     

一种改进的三维人脸重建方法

以径向基函数散乱数据插值算法为基础,针对该算法在三维人脸建模中的不足,提出一种改进的算法：在三维人脸建模中,根据特征点和非特征点的不同特性对基函数选择不同的参数值,使通用人脸模型的调整变换更加合理;然后应用二元三次多项式插值方法对特征点局部区域进行二次平滑,避免了畸变模型的产生,最终得到完整的三维人脸模型。实验结果表明,使用该算法构建的三维人脸模型生成速度快,且具有较高的精确度。     

基于分层策略的三维非刚性模型配准算法

在三维非刚性模型分析中,通常需要对不同姿态下的模型进行配准。针对传统配准算法存在复杂度高、计算量大、精确度低等问题,提出一种新的基于分层策略的三维非刚性模型配准算法。首先,定义热核签名函数为模型的标量域,使用同源聚类算法提取模型的特征点和特征区域,进而提出三维几何模型的树形表示方法:它的根节点为三维几何模型,内部节点为模型的特征区域,叶节点为包含在相应区域的特征点。然后,根据三维几何模型的树形表示提出模型的分层配准算法。在SHREC 2010模型配准数据集上对比分析了分层配准算法、推广的多维尺度变换算法（GMDS）和博弈论方法在等距变换、孔洞、小孔洞、尺度变换、局部尺度变换、重采样、噪声、散粒噪声以及拓扑变换等情况下的性能。实验结果表明,在以上三维几何模型数据受干扰的情况下,分层配准算法的准确性明显优于GMDS方法和博弈论方法,同时具有较低的计算复杂度。     

Reconstruction of non-rigid 3D shapes from stereo-motion

Several non-rigid structure from motion methods have been proposed so far in order to recover both the motion and the non-rigid structure of an object. However, these monocular algorithms fail to give reliable 3D shape estimates when the overall rigid motion of the sequence is small. Aiming to overcome this limitation, in this paper we propose a novel approach for the 3D Euclidean reconstruction of deformable objects observed by an uncalibrated stereo rig. Using a stereo setup drastically improves the 3D model estimation when the observed 3D shape is mostly deforming without undergoing strong rigid motion. Our approach is based on the following steps. Firstly, the stereo system is automatically calibrated and used to compute metric rigid structures from pairs of views. Afterwards, these 3D shapes are aligned to a reference view using a RANSAC method in order to compute the mean shape of the object and to select the subset of points which have remained rigid throughout the sequence. The selected rigid points are then used to compute frame-wise shape registration and to robustly extract the motion parameters from frame to frame. Finally, all this information is used as initial estimates of a non-linear optimization which allows us to refine the initial solution and also to recover the non-rigid 3D model. Exhaustive results on synthetic and real data prove the performance of our proposal estimating motion, non-rigid models and stereo camera parameters even when there is no rigid motion in the original sequence.     

Intraclass retrieval of nonrigid 3D objects: application to face recognition

As the size of the available collections of 3D objects grows, database transactions become essential for their management with the key operation being retrieval (query). Large collections are also precategorized into classes so that a single class contains objects of the same type (e.g., human faces, cars, four-legged animals). It is shown that general object retrieval methods are inadequate for intraclass retrieval tasks. We advocate that such intraclass problems require a specialized method that can exploit the basic class characteristics in order to achieve higher accuracy. A novel 3D object retrieval method is presented which uses a parameterized annotated model of the shape of the class objects, incorporating its main characteristics. The annotated subdivision-based model is fitted onto objects of the class using a deformable model framework, converted to a geometry image and transformed into the wavelet domain. Object retrieval takes place in the wavelet domain. The method does not require user interaction, achieves high accuracy, is efficient for use with large databases, and is suitable for nonrigid object classes. We apply our method to the face recognition domain, one of the most challenging intraclass retrieval tasks. We used the Face Recognition Grand Challenge v2 database, yielding an average verification rate of 95.2 percent at 10-3 false accept rate. The latest results of our work can be found at http://www.cbl.uh.edu/UR8D/.     

An interpolation algorithm fitted for dynamic 3D face reconstruction

In order to solve the problem of low recognition accuracy in later period which is caused by the too few extracted parameters in the 3D face recognition, and the incapable formation of completed point cloud structure. An automatic iterative interpolation algorithm is proposed. The new and more accurate 3D face data points are obtained by automatic iteration. This algorithm can be used to restore the data point cloud information of 3D facial feature in 2D images by means of facial three-legged structure formed by 3D face and automatic interpolation. Thus, it can realize to shape the 3D facial dynamic model which can be recognized and has high saturability. Experimental results show that the interpolation algorithm can achieve the complete the construction of facial feature based on the facial feature after 3D dynamic reconstruction, and the validity is higher.