Performance‐driven animation of hand‐drawn cartoon faces

We present a novel performance‐driven approach to animating cartoon faces starting from pure 2D drawings. A 3D approximate facial model automatically built from front and side view master frames of character drawings is introduced to enable the animated cartoon faces to be viewed from angles different from that in the input video. The expressive mappings are built by artificial neural network (ANN) trained from the examples of the real face in the video and the cartoon facial drawings in the facial expression graph for a specific character. The learned mapping model makes the resultant facial animation to properly get the desired expressiveness, instead of a mere reproduction of the facial actions in the input video sequence. Furthermore, the lit sphere, capturing the lighting in the painting artwork of faces, is utilized to color the cartoon faces in terms of the 3D approximate facial model, reinforcing the hand‐drawn appearance of the resulting facial animation. We made a series of comparative experiments to test the effectiveness of our method by recreating the facial expression in the commercial animation. The comparison results clearly demonstrate the superiority of our method not only in generating high quality cartoon‐style facial expressions, but also in speeding up the animation production of cartoon faces. Copyright © 2011 John Wiley & Sons, Ltd.     

漫画风格的人脸肖像生成算法

为模拟艺术家画出的具有夸张效果的肖像漫画,由用户输入正面人脸照片,通过交互获取关键特征点,根据特征点计算面部特征值来判断需要夸张的部分和各自的夸张方式;然后根据特征点自动生成不同层级的网格并分层实施夸张变形;最后进行图像处理以获得不同艺术风格的图像.该算法从研究漫画家的作画风格入手,总结了肖像漫画特定的夸张规律,所生成的漫画风格人脸肖像效果较好,能够应用于非真实感图形学和数字娱乐等领域.     

基于MPEG-4的人脸表情图像变形研究

为了实时地生成自然真实的人脸表情,提出了一种基于MPEG-4人脸动画框架的人脸表情图像变形方法。该方法首先采用face alignment工具提取人脸照片中的88个特征点;接着在此基础上,对标准人脸网格进行校准变形,以进一步生成特定人脸的三角网格;然后根据人脸动画参数（FAP）移动相应的面部关键特征点及其附近的关联特征点,并在移动过程中保证在多个FAP的作用下的人脸三角网格拓扑结构不变;最后对发生形变的所有三角网格区域通过仿射变换进行面部纹理填充,生成了由FAP所定义的人脸表情图像。该方法的输入是一张中性人脸照片和一组人脸动画参数,输出是对应的人脸表情图像。为了实现细微表情动作和虚拟说话人的合成,还设计了一种眼神表情动作和口内细节纹理的生成算法。基于5分制（MOS）的主观评测实验表明,利用该人脸图像变形方法生成的表情脸像自然度得分为3.67。虚拟说话人合成的实验表明,该方法具有很好的实时性,在普通PC机上的平均处理速度为66.67 fps,适用于实时的视频处理和人脸动画的生成。     

Example-Based Rendering of Eye Movements

This paper describes a model for example-based, photo-realistic rendering of eye movements in 3D facial animation. Based on 3D scans of a face with different gaze directions, the model captures the motion of the eyeball along with the deformation of the eyelids and the surrounding skin. These deformations are represented in a 3D morphable model.
Unlike the standard procedure in facial animation, the eyeball is not modeled as a rotating 3D sphere located behind the skin surface. Instead, the visible region of the eyeball is part of a continuous face mesh, and displacements of the iris as well as occlusions by the lids are modeled in a texture mapping approach. The algorithm avoids artifacts that are widely encountered in 3D facial animation, and it presents a new concept of handling occlusions and discontinuities in morphing algorithms.     

基于标准肤色的人脸图像纹理合成与三维重建应用

本文提出一种基于单幅人脸图像并结合标准肤色的人脸图像纹理合成和三维重建算法.首先,利用ASM算法提取人脸特征点,并通过基于局部线性嵌入算法的编辑传播实现颜色转换,使图像人脸色调与三维人脸模型标准肤色一致.接着,将人脸图像五官区域与标准肤色图进行泊松融合,并考虑眉毛遮挡情况,利用人脸对称性或眉毛模板还原眉毛.尤其对于半遮挡眉毛,采用Li模型和角点检测相结合的方法重建眉毛轮廓,得到最终人脸纹理图.最后通过纹理映射将人脸纹理图映射到三维人脸模型上,得到较好的个性化三维人脸重建效果.实验表明,本文算法能够适用于不同复杂背景和光照条件下拍摄的人脸图像,具有较快的处理速度,能够应用于人脸实时重建产品中.     

基于人脸特征分析的卡通人脸动画系统

给出一个基于人脸特征分析的卡通人脸动画系统,其输入是一张二维真实人脸照片和一段文本,输出为具有娱乐效果的卡通人脸动画。该文采用基于人脸特征分析的卡通人脸肖像生成方法,在卡通肖像的基础上结合文本驱动产生卡通人脸动画。将系统移植到移动平台,在手机上生成卡通人脸动画。实验结果表明该系统在PDA上能产生较好的娱乐效果。     

Frontal and Semi-Frontal Facial Caricature Synthesis Using Non-Negative Matrix Factorization

In this paper, we present a novel approach to synthesizing frontal and semi-frontal cartoon-like facial caricatures from an image. The caricature is generated by warping the input face from the original feature points to the corresponding exaggerated feature points. A 3D mean face model is incorporated to facilitate face to caricatures by inferring the depth of 3D feature points and the spatial transformation. Then the 3D face is deformed by using non-negative matrix factorization and projected back to image plane for future warping. To efficiently solve the nonlinear spatial transformation, we propose a novel initialization scheme to set up Levenberg-Marquardt optimization. According to the spatial transformation, exaggeration is applied to the most salient features by exaggerating their normalized difference from the mean. Non-photorealistic rendering (NPR) based stylization completes the cartoon caricature. Experiments demonstrate that our method outperforms existing methods in terms of view angles and aesthetic visual quality.     

Modelling and animating faces using scanned data

This paper extends a physically-based approach to the realistic modelling and animation of human heads using scanned data. A three-step procedure is proposed which begins with geometry and colour data acquired from a subject's head using a radial laser scanner, continues with the construction of a 3D model of the head with complete colour texture mapping, and culminates in real-time, interactive facial animation. First, an adaptive meshing technique is described that creates non-uniform facial meshes from the densely sampled data. These geometrically accurate facial meshes adapt to features of interest in the data, increasing polygon density over articulate face regions and areas of high curvature. Secondly, the mesh is transformed into a physically-based head model of the subject. This incorporates a physical approximation to facial tissue and a set of anatomically-motivated facial muscle actuators. Finally, these components are integrated in a facial modelling and animation system.     

面向移动数字娱乐的卡通人脸动画系统

给出了一个移动平台下的卡通人脸动画系统,其输入是一张二维真实人脸照片和一段文本,输出为具有娱乐效果的手机卡通人脸动画。首先,根据输入的照片生成人物的卡通肖像,采用了基于特征发现的卡通人脸肖像生成方法。其次,在卡通肖像的基础上结合文本驱动产生卡通人脸动画。最后,将系统移植到移动平台,在手机上播放卡通人脸动画,该系统在局域网和PDA上具有很好的娱乐动画效果。     

3D cartoon face generation by local deformation mapping

We present a data-driven method for automatically generating a 3D cartoon of a real 3D face. Given a sparse set of 3D real faces and their corresponding cartoon faces modeled by an artist, our method models the face in each subspace as the deformation of its nearby exemplars and learn a mapping between the deformations defined by the real faces and their cartoon counterparts. To reduce the exemplars needed for learning, we regress a collection of linear mappings defined locally in both face geometry and identity spaces and develop a progressive scheme for users to gradually add new exemplars for training. At runtime, our method first finds the nearby exemplars of an input real face and then constructs the result cartoon face from the corresponding cartoon faces of the nearby real face exemplars and the local deformations mapped from the real face subspace. Our method greatly simplifies the cartoon generation process by learning artistic styles from a sparse set of exemplars. We validate the efficiency and effectiveness of our method by applying it to faces of different facial features. Results demonstrate that our method not only preserves the artistic style of the exemplars, but also keeps the unique facial geometric features of different identities.     

基于单张图像的三维人脸建模与表情动画

在人机交互、数字娱乐等领域,传统的表情合成技术难以稳定地生成具有真实感的个性化人脸表情动画.为此,提出一种基于单张图像的三维人脸建模和表情动画系统.自动检测人脸在图像中的位置,自动定位人脸上的关键点,基于这些关键点和形变模型重建个性化三维人脸模型,对重建的人脸模型进行扩展得到完整的人脸网格,采用稀疏关键点控制的动画数据映射方法来驱动重建的人脸模型生成动态表情动画.实验结果表明,该方法稳定性强、自动化程度高,人脸模型与表情动画比较逼真.     

3D face computational photography using PCA spaces

In this paper, we present a 3D face photography system based on a facial expression training dataset, composed of both facial range images (3D geometry) and facial texture (2D photography). The proposed system allows one to obtain a 3D geometry representation of a given face provided as a 2D photography, which undergoes a series of transformations through the texture and geometry spaces estimated. In the training phase of the system, the facial landmarks are obtained by an active shape model (ASM) extracted from the 2D gray-level photography. Principal components analysis (PCA) is then used to represent the face dataset, thus defining an orthonormal basis of texture and another of geometry. In the reconstruction phase, an input is given by a face image to which the ASM is matched. The extracted facial landmarks and the face image are fed to the PCA basis transform, and a 3D version of the 2D input image is built. Experimental tests using a new dataset of 70 facial expressions belonging to ten subjects as training set show rapid reconstructed 3D faces which maintain spatial coherence similar to the human perception, thus corroborating the efficiency and the applicability of the proposed system.     

面向影视动画的真实感三维人脸快速建模算法

针对影视动画领域,利用LS_5000型三维激光扫描仪,提出了一套真实感三维人脸快速建模算法。只需输入真实演员人脸的三维扫描点云和未定标的照片,以及极少的人工交互,即可生成虚拟演员真实感的三维人脸模型（包括几何模型、纹理模型和面向动画的可变模型）。实验结果表明,算法输出的模型简洁规范,可直接应用于实际的影视动画制作,提高了人脸模型建模效率。     

The squash-and-stretch stylization for character motions

The squash-and-stretch describes the rigidity of the character. This effect is the most important technique in traditional cartoon animation. In this paper, we introduce a method that applies the squash-and-stretch effect to character motion. Our method exaggerates the motion by sequentially applying the spatial exaggeration technique and the temporal exaggeration technique. The spatial exaggeration technique globally deforms the pose in order to make the squashed or stretched pose by modeling it as a covariance matrix of joint positions. Then, the temporal exaggeration technique computes a time-warping function for each joint, and applies it to the position of the joint allowing the character to stretch its links appropriately. The motion stylized by our method is a sequence of squashed and stretched poses with stretching limbs. By performing a user survey, we prove that the motion created using our method is similar to that used in 2D cartoon animation and is funnier than the original motion for human observers who are familiar with 2D cartoon animation.     

一个实用的人脸定制和表情动画编辑系统

介绍了一个简单而行之有效的特定人脸定制和表情动画的编辑系统，首先给定一个内嵌肌肉向量的一般人脸三维多边形网络模型并提供特定人脸的正侧面正交图象，然后采用snake技术自动适配人脸特征线，变分一般模型定制出三维虚拟人脸；接着再用多分辨率样条技术产生无缝的人脸纹理镶嵌图，进而生成高度真实感的虚拟特定人脸，由内嵌肌肉向量的参数化表示，通过编辑参数能赋予三维虚拟人脸各种丰富的表情。由于变分后人脸模型的拓扑不变，因此可用基于三角化的图象metamorphosis实现不同特定人脸间的三维morph，该系统能在廉价的PC平台上实现，其快速、简单而且具有真实感，具有很大的实用价值。     

Expressive facial animation synthesis by learning speech coarticulation and expression spaces

Synthesizing expressive facial animation is a very challenging topic within the graphics community. In this paper, we present an expressive facial animation synthesis system enabled by automated learning from facial motion capture data. Accurate 3D motions of the markers on the face of a human subject are captured while he/she recites a predesigned corpus, with specific spoken and visual expressions. We present a novel motion capture mining technique that "learns" speech coarticulation models for diphones and triphones from the recorded data. A phoneme-independent expression eigenspace (PIEES) that encloses the dynamic expression signals is constructed by motion signal processing (phoneme-based time-warping and subtraction) and principal component analysis (PCA) reduction. New expressive facial animations are synthesized as follows: First, the learned coarticulation models are concatenated to synthesize neutral visual speech according to novel speech input, then a texture-synthesis-based approach is used to generate a novel dynamic expression signal from the PIEES model, and finally the synthesized expression signal is blended with the synthesized neutral visual speech to create the final expressive facial animation. Our experiments demonstrate that the system can effectively synthesize realistic expressive facial animation     

一种参数化的表情映射方法*

人脸表情细节的合成是生成高度真实感人脸动画的重要环节,传统的表情映射技术仅考虑了人脸特征点的位置变化,无法处理表情细节;最新的研究考虑了表情纹理细节的合成,但仅能根据已有的表情样本一对一地获取纹理细节,应用范围很有限。有效地结合小波金字塔分解和表情比例图像两种技术,提出了一种进行表情合成的新算法。该算法既可以映射表情纹理细节,又可以生成表情动画,且表情夸张程度可参数化地控制,算法简单有效。     

具有真实感的三维虚拟特定人脸生成方法

三维人脸的计算机生成目前是一个具有挑战性的课题。如何在人脸复杂的、不规则的表面上建模以及如何反映出特定人脸间的个体差异是现真实人脸模拟的两大主要困难所在。本文针对后者,提出了一种新的三维特定人脸生成方法,该方法基于人脸模型变形技术,允许模拟者在交互方式下将一般人脸几何模型和豫先提供的特定人脸多方面图像之间进形特征校准,进而得到特定人脸三维模型,该模型将精确地反映特定人脸的诸特征,同时,这种变形技术     

一种自动鲁棒的三维人脸重建方法

三维人脸模型已经广泛应用到视频电话、视频会议、影视制作、电脑游戏、人脸识别等多个领域。目前三维人脸建模一般使用多幅图像，且要求表情中性。本文提出了基于正、侧面任意表情三维人脸重建方法。首先对二维图像中的人脸进行特征提取，然后基于三维人脸统计模型，通过缩放、平移、旋转等方法，及全局和局部匹配，获得特定的三维人脸。基于二维图像中的人脸纹理信息，通过纹理映射，获得完整的三维人脸。通过对大量实际二维人脸图像的三维人脸重建，证实了该方法的有效性和鲁棒性。     

Character animation creation using hand-drawn sketches

To create a character animation, a 3D character model is often needed. However, since humanlike characters are not rigid bodies, to deform the character model to fit each animation frame is tedious work. Therefore, we propose an easy-to-use method for creating a set of consistent 3D character models from some hand-drawn sketches while keeping the projected silhouettes and features of the created models consistent with the input sketches. Since the character models possess vertexwise correspondences, they can be used for frame-consistent texture mapping or for making character animations. In our system, the user only needs to annotate the correspondence of the features among the input-vector-based sketches; the remaining processes are all performed automatically.