Three-dimensional cartoon facial animation based on art rules

Three-dimensional (3D) cartoon facial animation is one step further than the challenging 3D caricaturing which generates 3D still caricatures only. In this paper, a 3D cartoon facial animation system is developed for a subject given only a single frontal face image of a neutral expression. The system is composed of three steps consisting of 3D cartoon face exaggeration, texture processing, and 3D cartoon facial animation. By following caricaturing rules of artists, instead of mathematical formulations, 3D cartoon face exaggeration is accomplished at both global and local levels. As a result, the final exaggeration is capable of depicting the characteristics of an input face while achieving artistic deformations. In the texture processing step, texture coordinates of the vertices of the cartoon face model are obtained by mapping the parameterized grid of the standard face model to a cartoon face template and aligning the input face to the face template. Finally, 3D cartoon facial animation is implemented in the MPEG-4 animation framework. In order to avoid time-consuming construction of a face animation table, we propose to utilize the tables of existing models through model mapping. Experimental results demonstrate the effectiveness and efficiency of our proposed system.     

基于形变模型的真实感三维人脸动画研究

论文提出了一种新的基于三维人脸形变模型,并兼容于MPEG-4的三维人脸动画模型。采用基于均匀网格重采样的方法建立原型三维人脸之间的对齐,应用MPEG-4中定义的三维人脸动画规则,驱动三维模型自动生成真实感人脸动画。给定一幅人脸图像,三维人脸动画模型可自动重建其真实感的三维人脸,并根据FAP参数驱动模型自动生成人脸动画。     

基于MPEG-4的人脸表情图像变形研究

为了实时地生成自然真实的人脸表情,提出了一种基于MPEG-4人脸动画框架的人脸表情图像变形方法。该方法首先采用face alignment工具提取人脸照片中的88个特征点;接着在此基础上,对标准人脸网格进行校准变形,以进一步生成特定人脸的三角网格;然后根据人脸动画参数（FAP）移动相应的面部关键特征点及其附近的关联特征点,并在移动过程中保证在多个FAP的作用下的人脸三角网格拓扑结构不变;最后对发生形变的所有三角网格区域通过仿射变换进行面部纹理填充,生成了由FAP所定义的人脸表情图像。该方法的输入是一张中性人脸照片和一组人脸动画参数,输出是对应的人脸表情图像。为了实现细微表情动作和虚拟说话人的合成,还设计了一种眼神表情动作和口内细节纹理的生成算法。基于5分制（MOS）的主观评测实验表明,利用该人脸图像变形方法生成的表情脸像自然度得分为3.67。虚拟说话人合成的实验表明,该方法具有很好的实时性,在普通PC机上的平均处理速度为66.67 fps,适用于实时的视频处理和人脸动画的生成。     

基于Kinect 的人脸表情捕捉及动画模拟系统研究

三维人脸表情动画技术是一个具有巨大应用前景和研究意义的学科方向。在研究 现有表情捕捉和动画合成技术的基础上,提出了一种基于Microsoft Kinect 设备的人脸表情动画 系统。该系统首先利用Kinect 设备捕捉人脸并提取相关表情参数,同时利用Autodesk Maya 动 画软件建立对应的人脸三维模型,之后将模型导入到OGRE 动画引擎并把提取的表情参数传递 给OGRE,渲染生成实时人脸表情动画。实验结果表明该技术方案可行,实时表情动画效果达 到实际应用水平,相比于现有其他表情动画技术,系统采用通用的Kinect 设备,成本更大众化 且更容易进行操作。     

面向影视动画的真实感三维人脸快速建模算法

针对影视动画领域,利用LS_5000型三维激光扫描仪,提出了一套真实感三维人脸快速建模算法。只需输入真实演员人脸的三维扫描点云和未定标的照片,以及极少的人工交互,即可生成虚拟演员真实感的三维人脸模型（包括几何模型、纹理模型和面向动画的可变模型）。实验结果表明,算法输出的模型简洁规范,可直接应用于实际的影视动画制作,提高了人脸模型建模效率。     

AU-aware Algorithm for 3D Facial Reconstruction

As a critical task in computer vision and animation, 3D face reconstruction can provide 3D model structures and rich semantic information for multi-modal facial applications. However, monocular 2D facial images lack depth information and the parameters of the predicted facial model are not reliable, which causes poor reconstruction results. We propose to employ facial Action Unit (AU) and key facial points which are highly correlated with model parameters as a bridge to guide the regression of model-related parameters and thus solve the ill-posed monocular face reconstruction. Based on existing face reconstruction datasets, we provide a complete semi-automatic labeling scheme for facial AUs and construct a 300W-LP-AU dataset. Furthermore, a 3D face reconstruction algorithm based on AU awareness is put forward to realize end-to-end multi-tasking learning and reduce the overall training difficulty. Experimental results show that the algorithm can improve the face reconstruction performance, with high fidelity of the rebuilt facial model.     

视频驱动人脸动画合成(英文)

In this paper, we present a system for real-time performance-driven facial animation. With the system, the user can control the facial expression of a digital character by acting out the desired facial action in front of an ordinary camera. First,we create a muscle-based 3D face model. The muscle actuation parameters are used to animate the face model. To increase the reality of facial animation, the orbicularis oris in our face model is divided into the inner part and outer part. We also establish the relationship between jaw rotation and facial surface deformation. Second, a real-time facial tracking method is employed to track the facial features of a performer in the video. Finally, the tracked facial feature points are used to estimate muscle actuation parameters to drive the face model. Experimental results show that our system runs in real time and outputs realistic facial animations.Compared with most existing performance-based facial animation systems, ours does not require facial markers, intrusive lighting,or special scanning equipment, thus it is inexpensive and easy to use.     

视频驱动人脸动画合成

描述了一种实时的视频驱动的人脸动画合成系统.通过该系统,用户只要在摄像头前面表演各种脸部动作,就可以控制虚拟人脸的表情.首先,建立一个基于肌肉的三维人脸模型,并使用肌肉激励参数控制人脸形变.为提高人脸动画的真实感,将口轮匝肌分解为外圈和内圈两部分,同时建立脸部形变与下颌转动的关系.然后,使用一种实时的特征点跟踪算法跟踪视频中人脸的特征点.最后,将视频跟踪结果转换为肌肉激励参数以驱动人脸动画.实验结果表明,该系统能实时运行,合成的动画也具有较强真实感.与大部分现有的视频驱动的人脸动画方法相比,该系统不需要使用脸部标志和三维扫描设备,极大地方便了用户使用.     

Building highly realistic facial modeling and animation: a survey

This paper provides a comprehensive survey on the techniques for human facial modeling and animation. The survey is carried out from two different perspectives: facial modeling, which concerns how to produce 3D face models, and facial animation, which regards how to synthesize dynamic facial expressions. To generate an individual face model, we can either perform individualization of a generic model or combine face models from an existing face collection. With respect to facial animation, we have further categorized the techniques into simulation-based, performance-driven and shape blend-based approaches. The strength and weakness of these techniques within each category are discussed, alongside with the applications of these techniques to various exploitations. In addition, a brief historical review of the technique evolution is provided. Limitations and future trend are discussed. Conclusions are drawn at the end of the paper.     

基于单张图像的三维人脸建模与表情动画

在人机交互、数字娱乐等领域,传统的表情合成技术难以稳定地生成具有真实感的个性化人脸表情动画.为此,提出一种基于单张图像的三维人脸建模和表情动画系统.自动检测人脸在图像中的位置,自动定位人脸上的关键点,基于这些关键点和形变模型重建个性化三维人脸模型,对重建的人脸模型进行扩展得到完整的人脸网格,采用稀疏关键点控制的动画数据映射方法来驱动重建的人脸模型生成动态表情动画.实验结果表明,该方法稳定性强、自动化程度高,人脸模型与表情动画比较逼真.     

Learning a generic 3D face model from 2D image databases using incremental Structure-from-Motion

Over the last decade 3D face models have been extensively used in many applications such as face recognition, facial animation and facial expression analysis. 3D Morphable Models (MMs) have become a popular tool to build and fit 3D face models to images. Critical to the success of MMs is the ability to build a generic 3D face model. Major limitations in the MMs building process are: (1) collecting 3D data usually involves the use of expensive laser scans and complex capture setups, (2) the number of available 3D databases is limited, and typically there is a lack of expression variability and (3) finding correspondences and registering the 3D model is a labor intensive and error prone process.     

将高解析度动作撷取资料校正应用在脸部动画系统

提出一个新的方法来产生脸部动画,即利用动作撷取系统捕捉真人脸上的细微动作,再将动态资料用来驱动脸部模型产生动画,首先,利用Ｏｘｆｏｒｄ Ｍｅｔｒｉｃｓ’ＶＩＣＯＮ８系统,在真人的脸上贴了２３个反光标记物,用以进行动作撷取,得到三维动态资料后,必须经过后继处理才能使用,因此,提出了消除头部运动的方法,并估计人头的旋转支点,经过处理后,剩余的动态资料代表脸部表情的变化,因此,可以直接运用到脸部模型。用     

将高解析度动作撷取资料校正应用在脸部画系统

提出一个新的方法来产生脸部动画,即利用动作撷取系统捕捉真人脸上的细微动作,再将动态资料用来驱动脸部模型产生动画.首先,OXfor Metrics'VICON8系统,在真人的脸上贴了23全反光标记物,用以进行动作撷取.得到三维动态资料后,必须经过后继处理才能使用,因此,提出了消除头部运动的方法,并估计人头的旋转支点,经过处理后,剩余的动态资料代表脸部表情的变化,因此,可以直接运用到脸部模型.用2.5D的脸模型来实作系统,这样可兼得二维模型与三维模型的优点:简单、在小角度旋转时显得生动、自然.在脸部动务的制作中,利用一个特殊的内差公式来计算非特征点的位移,并将脸部分成数个区域,用以限制模型上三维点的移动,使动画更加自然,此动画系统在Pentium Ⅲ500MHz的机器上,并配有OpenGL的加速卡,更新率可以超过每秒30张.     

Craniofacial reconstruction based on multi-linear subspace analysis

Craniofacial reconstruction aims to estimate an individual’s facial appearance from its skull. It can be applied in many multimedia services such as forensic medicine, archaeology, face animation etc. In this paper, a statistical learning based method is proposed for 3D craniofacial reconstruction. In order to well represent the craniofacial shape variation and better utilize the relevance between different local regions, two tensor models are constructed for the skull and the face skin respectively, and multi-linear subspace analysis is used to extract the craniofacial subspace features. A partial least squares regression (PLSR) based mapping from skull subspace to skin subspace is established with the attributes such as age and BMI being considered. For an unknown skull, the 3D face skin is reconstructed using the learned mapping with the help of the skin tensor model. Compared with some other statistical learning based method in literature, the proposed method more directly and properly reflects the shape relationship between the skull and the face. In addition, the proposed method has little manual intervention. Experimental results show that the proposed method is valid.     

一个实用的人脸定制和表情动画编辑系统

介绍了一个简单而行之有效的特定人脸定制和表情动画的编辑系统，首先给定一个内嵌肌肉向量的一般人脸三维多边形网络模型并提供特定人脸的正侧面正交图象，然后采用snake技术自动适配人脸特征线，变分一般模型定制出三维虚拟人脸；接着再用多分辨率样条技术产生无缝的人脸纹理镶嵌图，进而生成高度真实感的虚拟特定人脸，由内嵌肌肉向量的参数化表示，通过编辑参数能赋予三维虚拟人脸各种丰富的表情。由于变分后人脸模型的拓扑不变，因此可用基于三角化的图象metamorphosis实现不同特定人脸间的三维morph，该系统能在廉价的PC平台上实现，其快速、简单而且具有真实感，具有很大的实用价值。     

Physically-based facial modelling,analysis, and animation

We develop a new 3D hierarchical model of the human face. The model incorporates a physically-based approximation to facial tissue and a set of anatomically-motivated facial muscle actuators. Despite its sophistication, the model is efficient enough to produce facial animation at interactive rates on a high-end graphics workstation. A second contribution of this paper is a technique for estimating muscle contractions from video sequences of human faces performing expressive articulations. These estimates may be input as dynamic control parameters to the face model in order to produce realistic animation. Using an example, we demonstrate that our technique yields sufficiently accurate muscle contraction estimates for the model to reconstruct expressions from dynamic images of faces.     

Modeling and animation of facial expressions based on B-Splines

This paper describes a technique for the automatic adaptation of a canonical facial model to data obtained by a 3D laser scanner. The facial model is a B-spline surface with 13×16 control points. We introduce a technique by which this canonical model is fit to the scanned data and that takes into consideration the requirements for the animation of facial expressions. The animation of facial expressions is based on the facial action coding system (FACS). Using B-splines in combination with FACS, we automatically create the impression of a moving skin. To increase the realism of the animation we map textural information onto the B-spline surface.     

改进的统计模型三维人脸特征点标定算法框架

自动三维人脸特征点标定是计算机视觉领域的研究热点,其广泛应用于人脸识别,人脸模型配准,表情识别,脸部动画等领域。通过对三维人脸样本统计建模,采用遗传算法对待匹配模型的生成数目进行参数优化,利用模型相似性匹配方法及其映射关系对三维人脸特征点进行自动标定。首先,对三维人脸数据预处理,然后对其统计建模并通过模型形变得到有映射关系的基准模型和待匹配模型。利用遗传算法对待匹配模型中的待匹配模型生成数目参数进行优化,生成与之对应的待匹配模型数;接着计算待测模型与待匹配模型的相似度。最后,利用模型相似度和模型映射关系,间接得到待测模型的特征点。实验结果表明,提出的算法是可行的,能够在一定程度上提高原有算法的效率。该算法可以自动标定三维人脸模型的特征点,当距离阈值为10像素时,39个三维人脸特征点定位的准确率都可以达到100%,并有效解决了传统方法中三维人脸模型平滑区域特征点精度不高的问题。     

3D Deformable Super-Resolution for Multi-Camera 3D Face Scanning

Low-cost and high-accuracy 3D face measurement is becoming increasingly important in many computer vision applications including face recognition, facial animation, games, orthodontics and aesthetic surgery. In most cases fringe projection based systems are used to overcome the relatively uniform appearance of skin. These systems employ a structured light camera/projector device and require explicit user cooperation and controlled lighting conditions. In this paper, we propose a 3D acquisition solution with a 3D space-time non-rigid super-resolution capability, using three calibrated cameras coupled with a non calibrated projector device, which is particularly suited to 3D face scanning, i.e. rapid, easily movable and robust to ambient lighting variation. The proposed solution is a hybrid stereovision and phase-shifting approach, using two shifted patterns and a texture image, which not only takes advantage of stereovision and structured light, but also overcomes their weaknesses. The super-resolution scheme involves a shape+texture 3D non-rigid registration for 3D artifacts correction in the presence of small non-rigid deformations as facial expressions.