Facial expression recognition using ASM-based post-processing technique

Facial expression recognition is a challenging field in numerous researches, and impacts important applications in many areas such as human-computer interaction and data-driven animation, etc. Therefore, this paper proposes a facial expression recognition system using active shape model (ASM) landmark information and appearance-based classification algorithm, i.e., embedded hidden Markov model (EHMM). First, we use ASM landmark information for facial image normalization and weight factors of probability resulted from EHMM. The weight factor is calculated through investigating Kullback-Leibler (KL) divergence of best feature with high discrimination power. Next, we introduce the appearance-based recognition algorithm for classification of emotion states. Here, appearance-based recognition means the EHMM algorithm using two-dimensional discrete cosine transform (2D-DCT) feature vector. The performance evaluation of proposed method was performed with the CK facial expression database and the JAFFE database. As a result, the method using ASM information showed performance improvements of 6.5 and 2.5% compared to previous method using ASM-based face alignment for CK database and JAFFE database, respectively.     

Facial expression recognition using constructive feedforward neural networks

A new technique for facial expression recognition is proposed, which uses the two-dimensional (2D) discrete cosine transform (DCT) over the entire face image as a feature detector and a constructive one-hidden-layer feedforward neural network as a facial expression classifier. An input-side pruning technique, proposed previously by the authors, is also incorporated into the constructive learning process to reduce the network size without sacrificing the performance of the resulting network. The proposed technique is applied to a database consisting of images of 60 men, each having five facial expression images (neutral, smile, anger, sadness, and surprise). Images of 40 men are used for network training, and the remaining images of 20 men are used for generalization and testing. Confusion matrices calculated in both network training and generalization for four facial expressions (smile, anger, sadness, and surprise) are used to evaluate the performance of the trained network. It is demonstrated that the best recognition rates are 100% and 93.75% (without rejection), for the training and generalizing images, respectively. Furthermore, the input-side weights of the constructed network are reduced by approximately 30% using our pruning method. In comparison with the fixed structure back propagation-based recognition methods in the literature, the proposed technique constructs one-hidden-layer feedforward neural network with fewer number of hidden units and weights, while simultaneously provide improved generalization and recognition performance capabilities.     

类间学习神经网络的人脸表情识别*

针对目前表情识别类间信息无关状态,提出了一种表情类间学习的神经网络分类识别算法。该算法首先构建一个BP网络学习对和一个距离判据单元,该距离判据单元仅用来计算类间的实际距离,类间期望距离是根据大量实验结果获得的;然后通过类内实际输出和类间期望距离来修正该网络;最后给出一组实例样本进行表情分类识别。实验结果表明,该算法能有效地识别人脸表情,能紧密地将各类表情间的信息联系起来,效率和准确性均有明显提高。     

基于VGG-NET的特征融合面部表情识别

为了解决在面部表情特征提取过程中卷积神经网络CNN和局部二值模式LBP只能提取面部表情图像的单一特征,难以提取与面部变化高度相关的精确特征的问题,提出了一种基于深度学习的特征融合的表情识别方法。该方法将LBP特征和CNN卷积层提取的特征通过加权的方式结合在改进的VGG-16网络连接层中,最后将融合特征送入Softmax分类器获取各类特征的概率,完成基本的6种表情分类。实验结果表明,所提方法在CK+和JAFFE数据集上的平均识别准确率分别达到了97.5%和97.62%,利用融合特征得到的识别结果明显优于利用单一特征识别的效果。与其他方法相比较,该方法能有效提高表情识别准确率,对光照变化更加鲁棒。     

基于生物启发特征和SVM的人脸表情识别

将C1特征应用于静态图像人脸表情识别,提出了一种新的基于生物启发特征和SVM的表情识别算法。提取人脸图像的C1特征,利用PCA+LDA方法对特征进行降维,用SVM进行分类。在JAFFE和Extended Cohn-Kanade（CK+）人脸表情数据库上的实验结果表明,该算法具有较高的识别率,是一种有效的人脸表情识别方法。     

人脸性别约束下的深度随机森林表情识别

由于人脸表情类内变化和类间干扰因素的存在,人脸表情识别仍面临着巨大挑战.提出一种基于性别条件约束随机森林的深度人脸表情识别方法,解决人脸表情识别中噪声、性别等变化和干扰问题.首先,采用深度多示例学习方法提取鲁棒性人脸特征,解决人脸光照、遮挡和低分辨率等图像变化问题;其次,采用性别条件随机森林分类方法进行人脸表情分类器设...     

Facial expression recognition using iterative universum twin support vector machine

Facial expressions are one of the most important characteristics of human behaviour. They are very useful in applications on human computer interaction. To classify facial emotions, different feature extraction methods are used with machine learning techniques. In supervised learning, information about the distribution of data is given by data points not belonging to any of the classes. These data points are known as universum data. In this work, we use universum data to perform multiclass classification of facial emotions from human facial images. Moreover, the existing universum based models suffer from the drawback of high training cost, so we propose an iterative universum twin support vector machine (IUTWSVM) using Newton method. Our IUTWSVM gives good generalization performance with less computation cost. To solve the optimization problem of proposed IUTWSVM, no optimization toolbox is required. Further, improper selection of universum points always leads to degraded performance of the model. For generating better universum, a novel scheme is proposed in this work based on information entropy of data. To check the effectiveness of proposed IUTWSVM, several numerical experiments are performed on benchmark real world datasets. For multiclass classification of facial emotions, the performance of IUTWSVM is compared with existing algorithms using different feature extraction techniques. Our proposed algorithm shows better generalization performance with less training cost in both binary as well as multiclass classification problems.     

Facial expression recognition using kernel canonical correlation analysis (KCCA)

In this correspondence, we address the facial expression recognition problem using kernel canonical correlation analysis (KCCA). Following the method proposed by Lyons et al. and Zhang et al. , we manually locate 34 landmark points from each facial image and then convert these geometric points into a labeled graph (LG) vector using the Gabor wavelet transformation method to represent the facial features. On the other hand, for each training facial image, the semantic ratings describing the basic expressions are combined into a six-dimensional semantic expression vector. Learning the correlation between the LG vector and the semantic expression vector is performed by KCCA. According to this correlation, we estimate the associated semantic expression vector of a given test image and then perform the expression classification according to this estimated semantic expression vector. Moreover, we also propose an improved KCCA algorithm to tackle the singularity problem of the Gram matrix. The experimental results on the Japanese female facial expression database and the Ekman's "Pictures of Facial Affect" database illustrate the effectiveness of the proposed method.     

Facial expression recognition using tracked facial actions: Classifier performance analysis

In this paper, we address the analysis and recognition of facial expressions in continuous videos. More precisely, we study classifiers performance that exploit head pose independent temporal facial action parameters. These are provided by an appearance-based 3D face tracker that simultaneously provides the 3D head pose and facial actions. The use of such tracker makes the recognition pose- and texture-independent. Two different schemes are studied. The first scheme adopts a dynamic time warping technique for recognizing expressions where training data are given by temporal signatures associated with different universal facial expressions. The second scheme models temporal signatures associated with facial actions with fixed length feature vectors (observations), and uses some machine learning algorithms in order to recognize the displayed expression. Experiments quantified the performance of different schemes. These were carried out on CMU video sequences and home-made video sequences. The results show that the use of dimension reduction techniques on the extracted time series can improve the classification performance. Moreover, these experiments show that the best recognition rate can be above 90%.     

自适应加权完全局部二值模式的表情识别

为了有效地提取局部特征和全局特征以提高表情识别的性能,提出自适应加权的完全局部二值模式（Adaptively Weighted Compound Local Binary Pattern,AWCLBP）的人脸表情识别算法。首先对人脸表情图像进行预处理分离出表情子区域,与此同时生成表情子区域的贡献度图谱（Contribution Map,CM）;然后对表情子区域和整幅表情图像做完全局部二值模式变换提取三种特征（差值符号特征CLBP_S、差值幅值特征CLBP_M、中心像素特征CLBP_C）并连接三种特征生成级联直方图,并根据CM对表情子区域的级联直方图进行加权和整张图像的直方图进行融合;最后用卡方距离和最近邻方法进行分类识别。本算法在JAFFE库上做了实验并和LBP、Gabor小波、活动外观模型进行了比较,验证了本算法的有效性。     

基于卷积神经网络的人脸表情识别

传统的神经网络表情识别系统由特征提取和神经网络分类器组成,利用人的经验来获取模式特征,很容易丢失表征表情特征的细节信息。提出一种基于卷积神经网络的识别方法,避免了对图像进行复杂的特征提取,直接把图像数据作为输入。通过在Cohn-Kanade表情库上的实验结果表明,该方法能够取得很好的表情分类效果。     

Facial expression recognition on multiple manifolds

Manifold learning has been successfully applied to facial expression recognition by modeling different expressions as a smooth manifold embedded in a high dimensional space. However, the assumption of single manifold is still arguable and therefore does not necessarily guarantee the best classification accuracy. In this paper, a generalized framework for modeling and recognizing facial expressions on multiple manifolds is presented which assumes that different expressions may reside on different manifolds of possibly different dimensionalities. The intrinsic features of each expression are firstly learned separately and the genetic algorithm (GA) is then employed to obtain the nearly optimal dimensionality of each expression manifold from the classification viewpoint. Classification is performed under a newly defined criterion that is based on the minimum reconstruction error on manifolds. Extensive experiments on both the Cohn-Kanade and Feedtum databases show the effectiveness of the proposed multiple manifold based approach.     

基于CBP-TOP特征的人脸表情识别*

针对人脸表情时空域特征信息的有效提取,本文提出了一种CBP-TOP（Centralized Binary Patterns From Three Orthogonal Panels）特征和SVM分类器相结合的人脸表情识别新方法。该方法首先将原始图像序列进行图像预处理,包括人脸检测、图像截取和图像尺度归一化,然后用CBP-TOP算子对图像序列进行分块提取特征,最后采用SVM分类器进行表情识别。实验结果表明,该方法能更有效提取图像序列的运动特征和动态纹理信息,提高了表情识别的准确率。和VLBP特征相比, CBP-TOP特征在表情识别中具有更高的识别率和更快的识别速度。     

疲劳驾驶面部表情识别算法

针对疲劳驾驶的六种表情 ,提出几何规范化结合 Gabor滤波提取表情特征 ,使用支持向量机对疲劳驾驶的面部表情分类识别的系统。首先对视频图像预处理进行几何规范化 ,利用二维 Gabor核函数构造最优滤波器 48个,获取 48个面部表情特征点 ,最后利用支持向量机进行面部表情分类识别。实验结果表明径向基函数的 SVM性能最好。     

持续学习改进的人脸表情识别

目的 大量标注数据和深度学习方法极大地提升了图像识别性能。然而,表情识别的标注数据缺乏,训练出的深度模型极易过拟合,研究表明使用人脸识别的预训练网络可以缓解这一问题。但是预训练的人脸网络可能会保留大量身份信息,不利于表情识别。本文探究如何有效利用人脸识别的预训练网络来提升表情识别的性能。方法 本文引入持续学习的思想,利用人脸识别和表情识别之间的联系来指导表情识别。方法指出网络中对人脸识别整体损失函数的下降贡献最大的参数与捕获人脸公共特征相关,对表情识别来说为重要参数,能够帮助感知面部特征。该方法由两个阶段组成：首先训练一个人脸识别网络,同时计算并记录网络中每个参数的重要性;然后利用预训练的模型进行表情识别的训练,同时通过限制重要参数的变化来保留模型对于面部特征的强大感知能力,另外非重要参数能够以较大的幅度变化,从而学习更多表情特有的信息。这种方法称之为参数重要性正则。结果 该方法在RAF-DB（real-world affective faces database）,CK+（the extended Cohn-Kanade database）和Oulu-CASIA这3个数据集上进行了实验评估。在主流数据集RAF-DB上,该方法达到了88.04%的精度,相比于直接用预训练网络微调的方法提升了1.83%。其他数据集的实验结果也表明了该方法的有效性。结论 提出的参数重要性正则,通过利用人脸识别和表情识别之间的联系,充分发挥人脸识别预训练模型的作用,使得表情识别模型更加鲁棒。     

Facial expression recognition: A clustering-based approach

This paper describes a new clustering based feature extraction method for facial expression recognition. We demonstrate the effectiveness of this method and compare it with commonly used principal component analysis method and linear discriminant analysis method.     

判别性完全局部二值模式人脸表情识别

针对完全局部二值模式（CLBP）存在直方图维数过高和特征冗余,会导致识别速度降低和识别率低的问题,提出基于有判别力的完全局部二值模式（Discriminative completed LBP,disCLBP）的人脸表情识别算法。首先,对人脸表情图像进行预处理获得表情子区域;然后提取表情子区域和整幅图像的disCLBP特征,针对不同的表情筛选出不同的表情特征,再将筛选出的表情子区域特征直方图融合;最后用最近邻分类器进行分类识别。该算法在CK人脸表情库上进行实验的平均识别率为97.3%。     

基于矩阵模式的人脸表情识别

提出了一种基于矩阵模式的人脸表情识别方法。该算法直接将人脸表情图像矩阵作为矩阵模式,并结合传统PCA和FLDA进行表情特征提取,称之为MatPCA和MatFLDA。与2DPCA等不同,该算法既利用图像矩阵中的行向量间的信息,也充分利用列向量间的信息,尽可能地保留了原始的表情信息。基于JAFFE和CED-WYU（1.0）两个表情数据库的识别结果表明,基于矩阵模式的特征提取方法能有效地提高识别率并节省计算时间。