基于红外结构光的三维人脸建模

在基于结构光的双目三维人脸重建中,容易丢失细节处数据和建模精度较低,导致三维人脸数据完整度不高,对三维人脸识别较差。本文研究了基于红外条纹的双目三维照相机系统,通过投射红外条纹结构光,根据相移法将生成包裹相位,利用三频法得到绝对相位,生成视差图,得到三维人脸模型。实验表明,基于红外的哑铃规球心测距误差在0.1%以内,人脸精度在0.1mm内。在眼睛,眉毛等弱纹理区数据缺失较少,优于基于可见光的三维建模,并且人脸模型更平滑更能反映人脸的真实三维形状。通过对比投射红外与可见光的性能,为三维人脸重建未来研究分析提供技术和算法上的参考。     

基于LED与肌肉模型的特定人脸建模与动画

特定三维人脸的建模与动画是计算机图形学中一个非常令人感兴趣的领域.本文提出了一种新的从两幅正交照片建立特定人脸的模型以及动画方法,首先以主动轮廓跟踪技术snake自动获取人脸特征点的准确位置,然后以文中的局部弹性变形(local elastic deformation)方法进行通用人脸模型到特定人脸的定制,并辅以采用图像镶嵌技术生成的大分辨率纹理图像施行纹理绘制,该方法以特征点的位移和非特征点与特征点的相对位置为基础计算局部人脸面部的变形,同时还能够实现人脸剧烈的面部变化和动作,与肌肉模型相结合,可很好地实时完成人脸的动画,具有快速高效的特点.最后,给出了所得到的实验结果.     

一种基于先验生成对抗网络的人脸超分辨率重建方法

针对人脸超分辨率算法中图像失真大、缺乏细节特征等问题,提出了一种基于先验知识的人脸超分辨率重建模型。通过在超分网络中加入纹理辅助分支,为重建过程提供额外纹理结构先验,以生成精细的面部纹理,恢复高分辨率纹理图。同时引入级联叠加模块对纹理辅助分支进行反馈。设计特征融合模块,将纹理特征图与超分分支特征图融合,获得更好的纹理细节;将纹理损失融入损失函数,以提高网络恢复纹理细节的能力。4倍放大因子下,该方法的峰值信噪比(Peak Signal-to-Noise Ratio, PSNR)、结构相似性指数(Structural Similarity Index, SSIM)比现有方法至少提升1.082 5 dB和0.036,无参考图像质量评价(Natural Image Quality Evaluator, NIQE)至少降低1.690 2;8倍放大因子下,该方法的PSNR与SSIM值分别至少提升0.787 5 dB和0.046 85,NIQE值最小降低3.92。     

基于稀疏形变模型的三维卡通人脸生成

 随着数字娱乐产业的发展,由照片生成卡通人脸的技术将取得广泛应用.此前的方法主要集中在平面卡通化的领域,风格较为单一.对于三维人脸,尽管形变模型方法可以由照片合成各种属性的三维人脸,但它计算量较大,不适用于实时应用场合.本文提出了一种基于稀疏形变模型的三维卡通人脸生成方法,提高了计算速度,且只需要单幅正面人脸照片.首先由稀疏形变模型拟合照片人脸获得特定的稀疏人脸模型;然后将一个一般人脸模型变形到特定人脸并合成纹理;最后对三维人脸进行卡通化.实验结果证明本文方法能够快速自动地合成生动的三维卡通人脸.     

一种无标记点人脸表情捕捉与重现算法

提出了一种无标记点的人脸表情捕捉方法.首先根据ASM（Active Shape Model）人脸特征点生成了覆盖人脸85%面部特征的人脸均匀网格模型;其次,基于此人脸模型提出了一种表情捕捉方法,使用光流跟踪特征点的位移变化并辅以粒子滤波稳定其跟踪结果,以特征点的位移变化驱动网格整体变化,作为网格跟踪的初始值,使用网格的形变算法作为网格的驱动方式.最后,以捕捉到的表情变化数据驱动不同的人脸模型,根据模型的维数不同使用不同的驱动方法来实现表情动画重现,实验结果表明,提出的算法能很好地捕捉人脸表情,将捕捉到的表情映射到二维卡通人脸和三维虚拟人脸模型都能取得较好的动画效果.     

基于照片的人脸建模

研究由一张正面照片和一张侧面照片合成头部三维模型的方法.从照片中提取面部特征点后.得到人脸在两个坐标系统中的三维空间数据坐标,从两张照片可以合成任意角度视图.然后进行数据拟合得到非特征点的数据,最后根据三维数据重建得到三维人脸模型.     

一种基于共振峰分析的语音驱动人脸动画方法

快速、高效地实现语音驱动下的唇形自动合成,以及优化语音与唇动的同步是语音驱动人脸动画的重点。提出了一种基于共振峰分析的语音驱动人脸动画的方法。对语音信号进行加窗分帧,DFT变换,再对短时音频信号的频谱进行第一、第二共振峰分析,将分析结果映射为一组控制序列,并对控制序列进行去奇异点等后处理。设定三维人脸模型的动态基本口形,以定时方式将控制序列导入模型,完成人脸动画驱动。实验结果表明,该方法简单快速,有效实现了语音和唇形的同步,动画效果连贯自然,可广泛用于各类虚拟角色的配音,缩短虚拟人物的制作周期。     

基于PSO的人脸姿态估计

提出了一种人脸姿态估计的新方法。根据三维扫描预先生成一个通用的模型,基于三维形态模型采用粒子群优化算法对人脸姿态进行初步估计,由特征点检测确定姿态大致范围,再在初步估计的结果上进行修正迭代,从而对人脸姿态进行精确估计。实验表明,该方法在简化数学运算的基础上,具有较好的估计效果,平衡了计算复杂度与结果精确度。     

多视图三维人脸重建方法研究

人脸信息包含每个人的外貌、表情信息。对人脸进行三维重建,重建后的人脸信息能够广泛地应用于人脸识别门禁系统、人脸动画角色以及医美整形等行业,具有重要的研究价值。目前,三维人脸重建存在一些遮挡和大姿态等问题,其中大部分人脸重建方法是从单张图片入手,模型效果较差,而多张图片的信息综合利用更有助于人脸信息的重建。为此,本文围绕基于传统方法的多视图人脸重建和基于深度学习方法的多视图人脸重建展开详细阐述,并对多视图人脸重建领域进行总结与展望。     

基于深度卷积网络的3D人脸重构算法

基于单张人脸图片的3D人脸模型重构,无论是在计算机图形领域还是可见光成像领域都是一个极具挑战性的研究方向,对于人脸识别、人脸成像、人脸动画等实际应用更是具有重要意义。针对目前算法复杂度较高、运算量较大且存在局部最优解和初始化不良等问题,本文提出了一种基于深度卷积神经网络的单张图片向3D人脸自动重构算法。该算法首先基于3D转换模型来提取2D人脸图像的密集信息,然后构建深度卷积神经网络架构、设计总体损失函数,直接学习2D人脸图像从像素到3D坐标的映射,从而实现了3D人脸模型的自动构建。算法对比与仿真实验表明,该算法在3D人脸重建上的归一化平均误差更低,且仅需一张2D人脸图像便可自动重构生成3D人脸模型。所生成的3D人脸模型鲁棒性好,重构准确,完整保留表情细节,并且对不同姿态的人脸也具有较好的重建效果,能够在三维空间中无死角自由呈现,将满足更多实际应用需求。     

2D facial expression recognition via 3D reconstruction and feature fusion

In this paper, a novel feature extraction method is proposed for facial expression recognition by extracting the feature from facial depth and 3D mesh alongside texture. Accordingly, the 3D Facial Expression Generic Elastic Model (3D FE-GEM) method is used to reconstruct an expression-invariant 3D model from the human face. Then, the texture, depth and mesh are extracted from the reconstructed face model. Afterwards, the Local Binary Pattern (LBP), proposed 3D High-Low Local Binary Pattern (3DH-LLBP) and Local Normal Binary Patterns (LNBPs) are applied to texture, depth and mesh of the face, respectively, to extract the feature from 2D images. Finally, the final feature vectors are generated through feature fusion and are classified by the Support Vector Machine (SVM). Convincing results are acquired for facial expression recognition on the CK+, CK, JAFFE and Bosphorus image databases compared to several state-of-the-art methods.     

An interactive 3D video system for human facial reconstruction and expression modeling

A 3D facial reconstruction and expression modeling system which creates 3D video sequences of test subjects and facilitates interactive generation of novel facial expressions is described. Dynamic 3D video sequences are generated using computational binocular stereo matching with active illumination and are used for interactive expression modeling. An individual’s 3D video set is annotated with control points associated with face subregions. Dragging a control point updates texture and depth in only the associated subregion so that the user generates new composite expressions unseen in the original source video sequences. Such an interactive manipulation of dynamic 3D face reconstructions requires as little preparation on the test subject as possible. Dense depth data combined with video-based texture results in realistic and convincing facial animations, a feature lacking in conventional marker-based motion capture systems.     

Multi-pose 3D face recognition based on 2D sparse representation

The increasing availability of 3D facial data offers the potential to overcome the difficulties inherent with 2D face recognition, including the sensitivity to illumination conditions and head pose variations. In spite of their rapid development, many 3D face recognition algorithms in the literature still suffer from the intrinsic complexity in representing and processing 3D facial data. In this paper, we propose the intrinsic 3D facial sparse representation (I3DFSR) algorithm for multi-pose 3D face recognition. In this algorithm, each 3D facial surface is first mapped homeomorphically onto a 2D lattice, where the value at each site is the depth of the corresponding vertex on the 3D surface. Each 2D lattice is then interpolated and converted into a 2D facial attribute image. Next, the sparse representation is applied to those attribute images. Finally, the identity of each query face can be obtained by using the corresponding sparse coefficients. The innovation of our approach lies in the strategy of converting irregular 3D facial surfaces into regular 2D attribute images such that 3D face recognition problem can be solved by using the sparse representation of those attribute images. We compare the proposed algorithm to three widely used 3D face recognition algorithms in the GavabDB database, to six state-of-the-art algorithms in the FRGC2.0 database, and to three baseline algorithms in the NPU3D database. Our results show that the proposed I3DFSR algorithm can substantially improve the accuracy and efficiency of multi-pose 3D face recognition.     

基于激光扫描的人脸三维重建方法

针对人脸三维重建,基于RGB 激光扫描系统,提出了一种彩色三维重建方法。首先,以直接标定的方法,建立了数码相机采样图片的像素坐标系与世界坐标系的映射关系;利用相机拍摄的三色激光照明下的标准色板图像,标定了数码相机对三原色的感光曲线,即建立了相机三原色记录值与标准值之间的关系。然后,以人脸为目标,提出了从获取扫描数据到进行三维重建的完整方法。最后,经迭代法匹配,输出为通用的三维模型格式,实现了人脸在毫米量级上的三维重建。该方法简化了标定过程,算法简单,系统造价低,可推广至各种物件的三维扫描重建。     

基于色彩和纹理特征融合的模糊人脸识别方法

基于纹理特征的方法被广泛应用于人脸识别。然而纹理特征依赖于图像的高频细节信息,当图像出现模糊时,单纯利用纹理特征的识别方法的识别精度会急剧下降。为了克服纹理特征的在模糊人脸识别中的不足,提出了一种基于色彩特征和纹理特征融合的识别方法。首先参照人类的对立色感知机制提取人脸的色彩特征;然后,将该色彩特征和纹理特征分别用于识别分类;最后,将二者的识别相似度进行融合,得到最终的识别结果。该色彩特征描述了图像的低频信息,其对图像模糊不敏感,并且与描述图像高频信息的纹理特征具有良好的互补性。在FERET 和AR 人脸库上的实验表明,融合色彩特征和纹理特征有效地提高了模糊人脸的识别精度。     

人脸显性特征的融合构造方法及识别

 目前的人脸识别研究中,面部几何特征没有得到很好的利用.本文阐述了几何特征对于人脸识别的重要性,在此基础上提出了一种提取面部几何特征的新方法;通过融合几何信息和纹理信息构造出一种面部显性特征,并给出了相应的人脸识别方法.这种新的人脸识别方法相对于基于统计学习的子空间方法具有一定的优势,同时也可作为后者的有益补充.实验表明,本文提出的人脸表示特征及识别方法对人脸表情变化和环境光照变化均有一定的鲁棒性.     

Real-time Stereo Face Recognition by Fusing Appearance and Depth Fisherfaces

Reported 3D face recognition techniques assume the use of active 3D measurement for 3D facial capture. However, active method employ structured illumination (structure projection, phase shift, gray-code demodulation, etc) or laser scanning, which is not desirable in many applications. A major problem of using passive stereo is its lower 3D face resolution and thus no passive method for 3D face recognition has been reported. In this paper, a real-time passive stereo face recognition system is presented. Entire face detection, tracking, pose estimation and face recognition are investigated. We used SRI Stereo engine that outputs sub-pixel disparity automatically. An investigation is carried out in combining 3D and 2D information for face recognition. The straightforward two-stage principal component analysis plus linear discriminant analysis is carried out in appearance and depth face images respectively. A probe face is identified using sum of the weighted appearance and depth linear discriminant distances. We investigate the complete range of linear combinations to reveal the interplay between these two paradigms. The improvement of the face recognition rate using this combination is verified. The recognition rate by the combination is higher than that of either appearance alone or depth alone. We then discuss the implementation of the algorithm on a stereo vision system. A hybrid face and facial features detection/tracking approach is proposed which collects near-frontal views for face recognition. Our face detection/tracking approach automatically initializes without user intervention and can be re-initialized automatically if the tracking of the 3D face pose is lost. The experiments include two parts. Firstly, the performance of the proposed algorithm is verified on XM2VTS database; Secondly, the algorithm is demonstrated on a real-time stereo vision system. It is able to detect, track and recognize a person while walking toward a stereo camera.

三维人脸特征提取方法综述

有效提取人脸特征是人脸识别技术的关键组成部分。传统的二维图像容易受到光照、姿态及表情的影响,而三维数据被认为具有光照姿态不变性。文中从局部特征和整体特征两个角度,对三维人脸特征提取进行综述,对部分方法进行比较,并分析了方法的有效性,总结了三维人脸特征提取方法的优势和困难。     

A visual analysis/synthesis feedback loop for accurate face tracking

We propose a novel approach for face tracking, resulting in a visual feedback loop: instead of trying to adapt a more or less realistic artificial face model to an individual, we construct from precise range data a specific texture and wireframe face model, whose realism allows the analysis and synthesis modules to visually cooperate in the image plane, by directly using 2D patterns synthesized by the face model. Unlike other feedback loops found in the literature, we do not explicitly handle the 3D complex geometric data of the face model, to make real-time manipulations possible. Our main contribution is a complete face tracking and pose estimation framework, with few assumptions about the face rigid motion (allowing large rotations out of the image plane), and without marks or makeup on the user's face. Our framework feeds the feature-tracking procedure with synthesized facial patterns, controlled by an extended Kalman filter. Within this framework, we present original and efficient geometric and photometric modelling techniques, and a reformulation of a block-matching algorithm to make it match synthesized patterns with real images, and avoid background areas during the matching. We also offer some numerical evaluations, assessing the validity of our algorithms, and new developments in the context of facial animation. Our face-tracking algorithm may be used to recover the 3D position and orientation of a real face and generate a MPEG-4 animation stream to reproduce the rigid motion of the face with a synthetic face model. It may also serve as a pre-processing step for further facial expression analysis algorithms, since it locates the position of the facial features in the image plane, and gives precise 3D information to take into account the possible coupling between pose and expressions of the analysed facial images.