人脸表情的形变线性拟合方法

提出了用于人脸表情合成的形变线性拟合方法. 该方法利用人脸图像形变模型线性组合逼近的基本思想, 确定合成表情图像的形状信息和纹理信息, 其步骤简单, 容易实现. 该方法能有效地从中性表情人脸图像合成出具有表情的图像, 并且得到的人脸表情自然、逼真、具有说服力. 更为重要的是, 该方法能从闭着嘴的中性表情人脸图像合成出具有张开嘴露出牙齿效果的人脸表情图像, 克服了当前大多数人脸表情合成方法不能实现这一效果的不足.     

基于Gabor直方图特征和MVBoost的人脸表情识别

提出采用Gabor变换与分级直方图统计相结合的方法来提取表情特征,以分层次反映局部区域内纹理变化的信息.这比仅用一维的Gabor系数具有更强的特征表示能力.借助直方图特征,还设计了向量输入、多类连续输出的弱分类器,并嵌入到多类连续AdaBoost的算法框架中,得到了向量输入、多类输出的MVBoost方法.该方法直接对特征进行多类的判决以满足多类时分类的需求,而不必训练多个二分类的AdaBoost分类器,从而使训练过程和分类过程都得到简化.     

Facial Expression Synthesis Using Manifold Learning and Belief Propagation

Given a person’s neutral face, we can predict his/her unseen expression by machine learning techniques for image processing. Different from the prior expression cloning or image analogy approaches, we try to hallucinate the person’s plausible facial expression with the help of a large face expression database. In the first step, regularization network based nonlinear manifold learning is used to obtain a smooth estimation for unseen facial expression, which is better than the reconstruction results of PCA. In the second step, Markov network is adopted to learn the low-level local facial feature’s relationship between the residual neutral and the expressional face image’s patches in the training set, then belief propagation is employed to infer the expressional residual face image for that person. By integrating the two approaches, we obtain the final results. The experimental results show that the hallucinated facial expression is not only expressive but also close to the ground truth.     

基于偏最小二乘回归的人脸身份和表情同步识别方法

将偏最小二乘回归方法用于人脸身份和表情的同步识别。首先,对每幅人脸图像进行脸部特征提取以及相应的语义特征定义。在脸部特征提取方面,从每幅图像中标定出若干脸部关键点位置,并提取图像在该关键点处的Gabor小波系数（Gabor特征）以及关键点的坐标值（几何特征）,作为该图像的输入特征。语义特征则定义为该人脸图像所属的表情类别信息以及所对应的人脸身份信息。其次,利用核主成分分析(KPCA)方法对脸部Gabor特征和几何特征进行融合,使得输入特征具有更好的识别特性;最后,运用偏最小二乘回归(PLSR)方法建立脸部特征和语义特征之间的关系模型,并运用此模型对某一测试人脸图像进行表情和身份的同步识别。通过在JAFFE国际表情数据库和AR人脸数据库上的对比实验,证实了所提方法的有效性。     

基于形状外观关联映射的动态脸部纹理生成

脸部表情的变化细节(例如皱纹)是很重要的视觉线索,但是它们难以建模与合成.这是由于人们说话和作表情时,脸部纹理也在动态地改变.与传统的纹理映射方法不同,提出一种根据脸部特征点运动来生成脸部动态纹理的新方法.基于形状和外观在表情脸部图像上高度相关这个观察,设计建立了从形状到外观的映射关系.这个映射被称作"形状外观的关联映射(shape-appearance dependence mapping,简称SADM)".实验结果表明用,SADM合成出的人脸与真实的人脸十分近似.提出的SADM方法可以集成到基于线框模型的人头模型中产生真实的动画效果,也可以应用于基于模型的视频编码来进一步节省传输带宽.     

基于Boost改进的嵌入式隐马尔可夫模型的实时表情识别

在人机交互过程中，理解人类的情绪是计算机和人进行交流必备的技能之一。最能表达人类情绪的就是面部表情。设计任何现实情景中的人机界面，面部表情识别是必不可少的。在本文中，我们提出了交互式计算环境中的一种新的实时面部表情识别框架。文章对这个领域的研究主要有两大贡献：第一，提出了一种新的网络结构和基于AdaBoost的嵌入式HMM的参数学习算法。第二，将这种优化的嵌入式HMM用于实时面部表情识别。本文中，嵌入式HMM把二维离散余弦变形后的系数作为观测向量，这和以前利用像素深度来构建观测向量的嵌入式HMM方法不同。因为算法同时修正了嵌入式HMM的网络结构和参数，大大提高了分类的精确度。该系统减少了训练和识别系统的复杂程度，提供了更加灵活的框架，且能应用于实时人机交互应用软件中。实验结果显示该方法是一种高效的面部表情识别方法。     

基于对称双线性模型的光照鲁棒性人脸表情识别

针对传统的光照预处理方法降低原始图像质量、丢失部分有效辨识信息的缺点,提出一种新颖的应用对称双线性模型来对人脸表情图像进行光照预处理的光照鲁棒性人脸表情识别方法.首先通过对称双线性模型将训练集图像分解为相互独立的光照因子和表情因子,并提取其光照因子.接下来提取含有未知光照的测试集表情图像的表情因子,并将其转换到训练集的若干个已知光照上,这样处理能够将任意光照的测试图像转换到相同的光照平台上,令所有测试图像的特征具有归一化特性.实验结果表明, 本文所提光照预处理方 法在识别性能上优于传统的光照预处理方法,应用在光照处理后的JAFFE表情库上识别率达到92.37%, 表明其适用于光照鲁棒性人脸表情识别.     

基于特征运动的表情人脸识别

人脸像的面部表情识别一直是人脸识别的一个难点，为了提高表情人脸识别的鲁棒性，提出了一种基于特征运动的人脸识别方法，该方法首先利用块匹配的方法来确定表情人脸和无表情人脸之间的运动向量，然后利用主成分分析方法（PCA）从这些运动向量中，产生低维子空间，称之为特征运动空间，测试时，先将测试人脸与无表情人脸之间的运动向量投影到特征运动空间，再根据这个运动向量在特征运动空间里的残差进行人脸识别，同时还介绍了基于特征运动的个人模型方法和公共模型方法，实验结果证明，该新算法在表情人脸的识别上，优于特征脸方法，有非常高的识别率。     

多姿态人脸照片的姿态估计

提出了一种基于面部图像的新的匹配系统。在这个系统中,输入的图像与各种人脸姿态的数据库图像进行比较,然后,匹配的图像给出了人脸姿态。图像数据库不仅包括各种人脸姿态,而且也包括不同的光照条件,如此,这个人脸姿态评价系统适用于不同的光照条件。对于收集各种不同面部图像,这里是通过计算机自动产生,而不是拍摄实际的照片。特征空间方法被用于寻找与输入面部图像匹配的图像。因为不同的光照图像被收集在面部图像数据库中,故提取的主特征向量主要依靠人脸姿态。由于通过选用主特征向量而减少了向量的维数,故这个匹配过程是很快的。这个姿态评价系统能够继续跟踪在不同的光照条件下不同人的人脸姿态。     

Unified model in identity subspace for face recognition

Human faces have two important characteristics: (1) They are similar objects and the specific variations of each face are similar to each other; (2) They are nearly bilateral symmetric. Exploiting the two important properties, we build a unified model in identity subspace (UMIS) as a novel technique for face recognition from only one example image per person. An identity subspace spanned by bilateral symmetric bases, which compactly encodes identity information, is presented. The unified model, trained on an obtained training set with multiple samples per class from a known people group A, can be generalized well to facial images of unknown individuals, and can be used to recognize facial images from an unknown people group B with only one sample per subject. Extensive experimental results on two public databases (the Yale database and the Bern database) and our own database (the ICT-JDL database) demonstrate that the UMIS approach is significantly effective and robust for face recognition.     

Emotional facial expression model building

Human face plays a crucial role in interpersonal communication. If we synthesize vivid expressional face in cyberspace, we could make the interaction between computer and human more natural and friendly. In this paper, we present a simple methodology for mimicking realistic face by manipulating emotional states. Compared with traditional methods of facial expression synthesis, our approach takes three advantages at the same time. They are (1) generating facial expressions under quantitative control of emotional states, (2) rendering shape and illumination changes on face simultaneously and (3) synthesizing expressional face for any new person by only utilizing a neutral face image. We have discussed the implementation method in the paper and demonstrated the effects of our approach by using a series of interesting experiments, such as predicting unseen expressions for an unfamiliar person, simulating one’s facial expressions with someone else’s style, extracting pure emotional expressions from the admixtures.     

一种基于个人身份认证的正面人脸识别算法

利用小波分解提取人脸特征技术和支持向量机 (SVM)分类模型 ,提出了一种基于个人身份认证的正面人脸识别算法 (或称为人脸认证方法 ) .针对 M个用户的人脸认证算法包括二个阶段 :(1)训练阶段 :使用小波分解方法对脸像训练集中的人脸图象进行特征提取 ,并用所提取的人脸特征向量训练 M个 SVM(对应 M个用户 ) ;(2 )认证阶段 :先由待认证者所声称的用户身份 (姓名或密码等 )确定对应的一训练好的 SVM,然后用这一 SVM对小波分解方法提取的待认证人的脸像特征向量进行分类 ,分类结果将显示待认证人所声称的身份是否真实 .利用 ORL人脸图象库对该算法的实验测试结果 ,以及与径向基函数神经网络作为分类器时的实验结果比较表明了该算法性能的优越性     

Face recognition under arbitrary illumination using illuminated exemplars

Recently, the importance of face recognition has been increasingly emphasized since popular CCD cameras are distributed to various applications. However, facial images are dramatically changed by lighting variations, so that facial appearance changes caused serious performance degradation in face recognition. Many researchers have tried to overcome these illumination problems using diverse approaches, which have required a multiple registered images per person or the prior knowledge of lighting conditions. In this paper, we propose a new method for face recognition under arbitrary lighting conditions, given only a single registered image and training data under unknown illuminations. Our proposed method is based on the illuminated exemplars which are synthesized from photometric stereo images of training data. The linear combination of illuminated exemplars can represent the new face and the weighted coefficients of those illuminated exemplars are used as identity signature. We make experiments for verifying our approach and compare it with two traditional approaches. As a result, higher recognition rates are reported in these experiments using the illumination subset of Max-Planck Institute face database and Korean face database.     

Statistical learning based facial animation

To synthesize real-time and realistic facial animation, we present an effective algorithm which combines image- and geometry-based methods for facial animation simulation. Considering the numerous motion units in the expression coding system, we present a novel simplified motion unit based on the basic facial expression, and construct the corresponding basic action for a head model. As image features are difficult to obtain using the performance driven method, we develop an automatic image feature recognition method based on statistical learning, and an expression image semi-automatic labeling method with rotation invariant face detection, which can improve the accuracy and efficiency of expression feature identification and training. After facial animation redirection, each basic action weight needs to be computed and mapped automatically. We apply the blend shape method to construct and train the corresponding expression database according to each basic action, and adopt the least squares method to compute the corresponding control parameters for facial animation. Moreover, there is a pre-integration of diffuse light distribution and specular light distribution based on the physical method, to improve the plausibility and efficiency of facial rendering. Our work provides a simplification of the facial motion unit, an optimization of the statistical training process and recognition process for facial animation, solves the expression parameters, and simulates the subsurface scattering effect in real time. Experimental results indicate that our method is effective and efficient, and suitable for computer animation and interactive applications.     

Individual Kernel Tensor-subspaces for robust face recognition: a computationally efficient tensor framework without requiring mode factorization.

Facial images change appearance due to multiple factors such as different poses, lighting variations, and facial expressions. Tensors are higher order extensions of vectors and matrices, which make it possible to analyze different appearance factors of facial variation. Using higher order tensors, we can construct a multilinear structure and model the multiple factors of face variation. In particular, among the appearance factors, the factor of a person's identity modeled by a tensor structure can be used for face recognition. However, this tensor-based face recognition creates difficulty in factorizing the unknown parameters of a new test image and solving for the person-identity parameter. In this paper, to break this limitation of applying the tensor-based methods to face recognition, we propose a novel tensor approach based on an individual-modeling method and nonlinear mappings. The proposed method does not require the problematic tensor factorization and is more efficient than the traditional TensorFaces method with respect to computation and memory. We set up the problem of solving for the unknown factors as a least squares problem with a quadratic equality constraint and solve it using numerical optimization techniques. We show that an individual-multilinear approach reduces the order of the tensor so that it makes face-recognition tasks computationally efficient as well as analytically simpler. We also show that nonlinear kernel mappings can be applied to this optimization problem and provide more accuracy to face-recognition systems than linear mappings. In this paper, we show that the proposed method, Individual Kernel TensorFaces, produces the better discrimination power for classification. The novelty in our approach as compared to previous work is that the Individual Kernel TensorFaces method does not require estimating any factor of a new test image for face recognition. In addition, we do not need to have any a priori knowledge of or assumption about the factors of a test image when using the proposed method. We can apply Individual Kernel TensorFaces even if the factors of a test image are absent from the training set. Based on various experiments on the Carnegie Mellon University Pose, Illumination, and Expression database, we demonstrate that the proposed method produces reliable results for face recognition.     

2DFLD与LPP相结合的人脸和表情识别方法

提出一种二维Fisher线性判别分析(2DFLD)与局部保持投影(LPP)相结合的人脸和表情识别方法.首先,将训练集图像用2DFLD投影,使其按身份分离.然后,用LPP进行二次投影提取出它的表情流形.最后,给出概率度量,得到待测图像属于各类身份和表情的概率,从而识别出它的人脸和表情的类别.在CMU-AMP和JAFFE人脸库上的实验表明,该方法简便有效、识别效果好.     

Semi‐supervised classification of facial expression using a mixture of multivariate t distributions

Abstract: This paper addresses the semi‐supervised classification of facial expression images using a mixture of multivariate t distributions. The facial expression features are first extracted into labelled graph vectors using the Gabor wavelet transformation. We then learn a mixture of multivariate t distributions by using the labelled graph vectors, and set correspondence between the component distributions and the basic facial emotions. According to this correspondence, the classification of a given testing image is implemented in a probabilistic way according to its fitted posterior probabilities of component memberships. Specifically, we perform hard classification of the testing image by assigning it into an emotional class that the corresponding mixture component has the highest posterior probability, or softly use the posterior probabilities as the estimates of the semantic ratings of expressions. The experimental results on the Japanese female facial expression database, Ekman's Pictures of Facial Affect database and the AR database demonstrate the effectiveness of the proposed method.     

单幅人脸图像的全景纹理图生成方法

目的 针对从单幅人脸图像中恢复面部纹理图时获得的信息不完整、纹理细节不够真实等问题,提出一种基于生成对抗网络的人脸全景纹理图生成方法。方法 将2维人脸图像与3维人脸模型之间的特征关系转换为编码器中的条件参数,从图像数据与人脸条件参数的多元高斯分布中得到隐层数据的概率分布,用于在生成器中学习人物的头面部纹理特征。在新创建的人脸纹理图数据集上训练一个全景纹理图生成模型,利用不同属性的鉴别器对输出结果进行评估反馈,提升生成纹理图的完整性和真实性。结果 实验与当前最新方法进行了比较,在CelebA-HQ和LFW （labled faces in the wild）数据集中随机选取单幅正面人脸测试图像,经生成结果的可视化对比及3维映射显示效果对比,纹理图的完整度和显示效果均优于其他方法。通过全局和面部区域的像素量化指标进行数据比较,相比于UVGAN,全局峰值信噪比（peak signal to noise ratio,PSNR）和全局结构相似性（structural similarity index,SSIM）分别提高了7.9 dB和0.088,局部PSNR和局部SSIM分别提高了2.8 dB和0.053;相比于OSTeC,全局PSNR和全局SSIM分别提高了5.45 dB和0.043,局部PSNR和局部SSIM分别提高了0.4 dB和0.044;相比于MVF-Net （multi-view 3D face network）,局部PSNR和局部SSIM分别提高了0.6和0.119。实验结果证明,提出的人脸全景纹理图生成方法解决了从单幅人脸图像中重建面部纹理不完整的问题,改善了生成纹理图的显示细节。结论 本文提出的人脸全景纹理图生成方法,利用人脸参数和网络模型的特性,使生成的人脸纹理图更完整,尤其是对原图不可见区域,像素恢复自然连贯,纹理细节更真实。     

细节特征保持的三维面部表情迁移方法

在三维面部表情迁移中,针对保持目标模型丰富的细节信息以使生成的新表情真实自然,以及减少表情迁移的学习训练时间这2个热点问题,提出一种细节特征保持的三维面部表情迁移方法.首先提取三维面部模型的细节特征,获得滤掉细节后的基本表情;然后利用改进的有参无监督回归方法将源模型的基本表情传递给目标模型;最后利用提出的细节特征向量调...     

基于物理模型的人脸表情动画技术研究

用计算机建立人脸表情动画是当前计算机图形学研究领域的一个富有挑战性的课题，该文在总结了国内外有关该课题研究方法的基础上，提出了一种基于物理模型的人脸表情画生成算法，并依该算法计和开发了一个实际的人脸表情动画系统HUFACE。该算法将人的脸部模拟为一个弹性体，为使计算简化，又将人脸表面依其生理特性分为八个子块，脸部表情所产生的五官动作模拟为弹性体的形变，并建立相应的弹性形变模型，当脸部表情引起脸部各子块形变时，每个子块上的各点将发生位移，于是利用该模型计算这些点的位移量，由此获得表情动画中的每一帧画面，由于脸部动作由该形变模型控制，且计算简单，速度快，因此不需存储表情动画中的各个画面，提高了系统的效率，实验结果表明，由HUFACE系统生成的人脸表情真实，自然。