人机交互中的人脸表情识别研究进展

随着人机交互与情感计算技术的快速发展,人脸表情识别已成为人们研究的热点。为了阐明人机交互中人脸表情识别的研究方向及进展,该文从人脸表情数据库、表情特征提取、表情分类方法、鲁棒的表情识别、精细的表情识别、混合表情识别、非基本表情识别等方面对人脸表情识别的研究现状进行了分析。最后总结了人脸表情识别研究的热点及趋势,同时指出了人脸表情识别研究存在的局限性,并对人脸表情识别的发展进行了展望。     

人脸表情运动跟踪与表情识别算法

对于人脸视频中的每一帧,提出一种静态人脸表情识别算法,人脸表情运动参数被提取出来后,根据表情生理知识来分类表情;为了应对知识的不足,提出一种静态表情识别和动态表情识别相结合的算法,以基于多类表情马尔可夫链和粒子滤波的统计框架结合生理知识来同时提取人脸表情运动和识别表情.实验证明了算法的有效性.     

A Video-Based Facial Motion Tracking and Expression Recognition System

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.     

拟人Agent面部复杂运动合成的研究与实现

在多维面部运动的形式化描述方法的基础上,分析了面部肌肉的运动特征和控制约束,提出了面部运动单元的概念,采用正向运动学和示教再现两种方法实现拟人Agent的基本面部运动,并通过运动合成产生复杂的面部运动。实验表明,正向运动学方法能精确控制面部运动并降低非自然表情的出现;示教再现方法自动产生面部运动简化了操作细节;运动合成则通过运动重用提高了面部运动生成的效率。     

基于Gabor特征和混合高斯模型的人脸表情分析

首先对人脸表情的特点进行分析,提出了利用Gabor小波特征、主分量分析（PCA）结合混合高斯模型的人脸表情分析方法,并在人脸表情数据库JAFFE进行了实验。通过对不同表情的分布规律进行实验分析,实现了对表情的定性/定量分析。实验结果表明,提出的人脸表情分析方法能够对人脸表情进行恰当的表达和描述。     

A Unified Probabilistic Framework for Spontaneous Facial Action Modeling and Understanding

Facial expression is a natural and powerful means of human communication. Recognizing spontaneous facial actions, however, is very challenging due to subtle facial deformation, frequent head movements, and ambiguous and uncertain facial motion measurements. Because of these challenges, current research in facial expression recognition is limited to posed expressions and often in frontal view. A spontaneous facial expression is characterized by rigid head movements and nonrigid facial muscular movements. More importantly, it is the coherent and consistent spatiotemporal interactions among rigid and nonrigid facial motions that produce a meaningful facial expression. Recognizing this fact, we introduce a unified probabilistic facial action model based on the Dynamic Bayesian network (DBN) to simultaneously and coherently represent rigid and nonrigid facial motions, their spatiotemporal dependencies, and their image measurements. Advanced machine learning methods are introduced to learn the model based on both training data and subjective prior knowledge. Given the model and the measurements of facial motions, facial action recognition is accomplished through probabilistic inference by systematically integrating visual measurements with the facial action model. Experiments show that compared to the state-of-the-art techniques, the proposed system yields significant improvements in recognizing both rigid and nonrigid facial motions, especially for spontaneous facial expressions.     

基于图像的面部表情识别方法综述

近年来,面部表情识别在教育、医学、心理分析以及商业领域得到了广泛关注。针对目前表情识别方法不够系统、概念模糊的问题,对面部表情识别的步骤及其方法进行了综述探讨。首先,介绍了目前常用的人脸表情数据集,并回顾了面部表情识别的发展历程;然后,介绍了人脸表情识别的面部表情编码和面部表情识别过程这两个方面,归纳了人脸面部表情识别的四个过程,重点总结了特征提取和表情分类两个过程中的经典算法以及这些算法的基本原理和优劣比较;最后,指出了目前面部表情识别存在的问题和未来可能的发展趋势。     

Prediction of facial attractiveness from facial proportions

This study investigates the relationship between facial attractiveness and facial proportions. Here, we generated facial images with different proportions using computer software, hence avoided the influence of hairstyle, facial expression as well as skin tone and texture on the perception of facial attractiveness. By analyzing the relationship between the facial proportions of 432 computer generated facial images and their attractiveness ratings, here we identified the optimum proportions for an attractive female face and further established a model of predicting facial attractiveness from four principle components of facial proportions with good predictability (R²=0.64).     

Expression modeling—a boundary element approach

Popular techniques for modeling facial expression usually rely on the shape blending of a series of pre-defined facial models, the use of feature parameters, or the use of an anatomy based facial model. This requires extensive user interaction to construct the pre-defined facial model, the deformation functions, or the anatomy based facial model. Besides, existing anatomy based facial modeling techniques are targeted for human facial model and may not be used directly for non-human like character models. This paper presents an intuitive technique for the design of facial expressions using a physics based deformation approach. The technique does not require specifying the deformation function associated with facial feature parameters, and does not require a detail anatomical model of the head. By adjusting the contraction or relaxation of a set of facial muscles, different facial expressions can be obtained. Facial muscles and skin are assumed to be linearly elastic. The boundary element method (BEM) is adopted for evaluating deformation of the facial skin. This avoids the use of volumetric elements as in the case of finite element method (FEM) or the setting up of complex mass–spring models. Given a polygon mesh of a facial model, a closed volume of the facial mesh is obtained by offsetting the polygon mesh according to a user defined depth map. Each facial muscle is approximated with a series of muscle polygons on the mesh surface. Deformation of the facial mesh is attained by stretching or compressing the muscle polygons. By pre-computing the inverse of the stiffness matrix, interactive editing of facial expression can be achieved.     

基于综合技术的人脸检测与识别系统

本文提出了一个完整的人脸检测与识别系统。首先利用“参考白”技术进行了光线补偿,然後利用肤色及人脸结构特徵实现了人脸检测,并考虑了类似肤色的影响,最後根据人脸图像的灰度特性和几何特性,采用投影法和人脸五官的比例关系进行定位和提取,并在多媒体数据库中实现了人脸照片的直接查询。     

基于流水分割的面部特征定位

在人脸的计算机自动识别过程中，人脸特征的定位以及人脸特征的抽取起着至关重要的作用。文章提出了一种基于流水分割的面部特征定位方法，可以将面部轮廓从面部图像中剥离出去，进而利用剩余信息进行面部特征的定位。实验结果表明，该方法能够准确地进行面部特征定位，并选用于某些特殊的情况。     

利用图像变形生成个性化人脸卡通

提出了一种利用图像变形技术生成个性化人脸卡通的方法,该方法以真实人脸图像、卡通人脸图像和中性标准人脸网格为输入。根据提取的特征点,在人脸上分区域构建多个径向基神经网络;将标准人脸网格变形,分别和人脸以及卡通脸配准,得到人脸和卡通脸的个性化网格;将卡通脸图像作为纹理映射到个性化人脸网格,并做色调分离提取原始人脸的光照信息,得到个性化卡通人脸图像。     

面部表情的仿真研究

随着ＣＧ技术的发展,人的身体形状和动作画像,可以通过电脑生成。人类逐步进入信息社会,人与机器的对话加入人物像后,可以使对话稳定和顺利进行,该文对人物面部表情的仿真,特别是自然、多彩的表情生成,从技术角度做进一步探讨。     

基于人脸特征分析的卡通人脸动画系统

给出一个基于人脸特征分析的卡通人脸动画系统,其输入是一张二维真实人脸照片和一段文本,输出为具有娱乐效果的卡通人脸动画。该文采用基于人脸特征分析的卡通人脸肖像生成方法,在卡通肖像的基础上结合文本驱动产生卡通人脸动画。将系统移植到移动平台,在手机上生成卡通人脸动画。实验结果表明该系统在PDA上能产生较好的娱乐效果。     

基于生成式对抗网络的鲁棒人脸表情识别

人们在自然情感交流中经常伴随着头部旋转和肢体动作,它们往往导致较大范围的人脸遮挡,使得人脸图像损失部分表情信息.现有的表情识别方法大多基于通用的人脸特征和识别算法,未考虑表情和身份的差异,导致对新用户的识别不够鲁棒.本文提出了一种对人脸局部遮挡图像进行用户无关表情识别的方法.该方法包括一个基于Wasserstein生成式对抗网络（Wasserstein generative adversarial net,WGAN）的人脸图像生成网络,能够为图像中的遮挡区域生成上下文一致的补全图像;以及一个表情识别网络,能够通过在表情识别任务和身份识别任务之间建立对抗关系来提取用户无关的表情特征并推断表情类别.实验结果表明,我们的方法在由CK+,Multi-PIE和JAFFE构成的混合数据集上用户无关的平均识别准确率超过了90%.在CK+上用户无关的识别准确率达到了96%,其中4.5%的性能提升得益于本文提出的对抗式表情特征提取方法.此外,在45°头部旋转范围内,本文方法还能够用于提高非正面表情的识别准确率.     

全自动的基于MPEG-4的任意拓扑结构人脸动画

构造任意拓扑结构人脸网格的人脸动画定义表是基于MPEG-4的任意拓扑结构人脸动画系统的关键．通过搜索三维模型的二维纹理图像特征，提出一种自动地在任意拓扑结构三维人脸模型上定位特征点的方法．通过利用任意拓扑结构人脸模型上的三维特征点变形标准人脸模型，并根据标准人脸模型的动画定义表，实现了自动、准确地构造任意拓扑结构人脸模型动画定义表的方法．给定一个任意拓扑结构三维人脸模型，通过文中方法可以全自动地驱动所给人脸模型做动画．     

视频驱动人脸动画合成

描述了一种实时的视频驱动的人脸动画合成系统.通过该系统,用户只要在摄像头前面表演各种脸部动作,就可以控制虚拟人脸的表情.首先,建立一个基于肌肉的三维人脸模型,并使用肌肉激励参数控制人脸形变.为提高人脸动画的真实感,将口轮匝肌分解为外圈和内圈两部分,同时建立脸部形变与下颌转动的关系.然后,使用一种实时的特征点跟踪算法跟踪视频中人脸的特征点.最后,将视频跟踪结果转换为肌肉激励参数以驱动人脸动画.实验结果表明,该系统能实时运行,合成的动画也具有较强真实感.与大部分现有的视频驱动的人脸动画方法相比,该系统不需要使用脸部标志和三维扫描设备,极大地方便了用户使用.     

视频驱动人脸动画合成(英文)

In this paper, we present a system for real-time performance-driven facial animation. With the system, the user can control the facial expression of a digital character by acting out the desired facial action in front of an ordinary camera. First,we create a muscle-based 3D face model. The muscle actuation parameters are used to animate the face model. To increase the reality of facial animation, the orbicularis oris in our face model is divided into the inner part and outer part. We also establish the relationship between jaw rotation and facial surface deformation. Second, a real-time facial tracking method is employed to track the facial features of a performer in the video. Finally, the tracked facial feature points are used to estimate muscle actuation parameters to drive the face model. Experimental results show that our system runs in real time and outputs realistic facial animations.Compared with most existing performance-based facial animation systems, ours does not require facial markers, intrusive lighting,or special scanning equipment, thus it is inexpensive and easy to use.     

Bilinear interpolation for facial expression and metamorphosis in real-time animation

This paper describes a new method for generating facial animation in which facial expression and shape can be changed simultaneously in real time. A 2D parameter space independent of facial shape is defined, on which facial expressions are superimposed so that the expressions can be applied to various facial shapes. A facial model is transformed by a bilinear interpolation, which enables a rapid change in facial expression with metamorphosis. The practical efficiency of this method has been demonstrated by a real-time animation system based on this method in live theater.     

Facial beauty analysis based on features prediction and beautification models

This paper studies facial beauty analysis by using feature-based computer models in prediction and beautification. In order to assess the facial beauty index, a facial beauty prediction model is established, which can quickly and effectively estimate beauty indexes for new images. In the facial beautification model, we mainly achieve three functions: (1) propose an effective method to adjust the positions of the landmark points to beautify the facial geometry; (2) improve the multi-level median filtering method to perform facial skin beautification; and (3) design a method to achieve average facial beautification. Experimental results showed that the proposed models can significantly analyze the facial attractiveness of face images.