Multi-scale attention guided network for end-to-end face alignment and recognition

Attention modules embedded in deep networks mediate the selection of informative regions for object recognition. In addition, the combination of features learned from different branches of a network can enhance the discriminative power of these features. However, fusing features with inconsistent scales is a less-studied problem. In this paper, we first propose a multi-scale channel attention network with an adaptive feature fusion strategy (MSCAN-AFF) for face recognition (FR), which fuses the relevant feature channels and improves the network’s representational power. In FR, face alignment is performed independently prior to recognition, which requires the efficient localization of facial landmarks, which might be unavailable in uncontrolled scenarios such as low-resolution and occlusion. Therefore, we propose utilizing our MSCAN-AFF to guide the Spatial Transformer Network (MSCAN-STN) to align feature maps learned from an unaligned training set in an end-to-end manner. Experiments on benchmark datasets demonstrate the effectiveness of our proposed MSCAN-AFF and MSCAN-STN.     

Entropy-based feature selection for improved 3D facial expression recognition

Facial expressions contain most of the information on human face which is essential for human–computer interaction. Development of robust algorithms for automatic recognition of facial expressions with high recognition rates has been a challenge for the last 10 years. In this paper, we propose a novel feature selection procedure which recognizes basic facial expressions with high recognition rates by utilizing three-Dimensional (3D) geometrical facial feature positions. The paper presents a system of classifying expressions in one of the six basic emotional categories which are anger, disgust, fear, happiness, sadness, and surprise. The paper contributes on feature selections for each expression independently and achieves high recognition rates with the proposed geometric facial features selected for each expression. The novel feature selection procedure is entropy based, and it is employed independently for each of the six basic expressions. The system’s performance is evaluated using the 3D facial expression database, BU-3DFE. Experimental results show that the proposed method outperforms the latest methods reported in the literature.     

人脸显性特征的融合构造方法及识别

 目前的人脸识别研究中,面部几何特征没有得到很好的利用.本文阐述了几何特征对于人脸识别的重要性,在此基础上提出了一种提取面部几何特征的新方法;通过融合几何信息和纹理信息构造出一种面部显性特征,并给出了相应的人脸识别方法.这种新的人脸识别方法相对于基于统计学习的子空间方法具有一定的优势,同时也可作为后者的有益补充.实验表明,本文提出的人脸表示特征及识别方法对人脸表情变化和环境光照变化均有一定的鲁棒性.     

Automatic facial expression recognition: feature extraction and selection

In this paper, we investigate feature extraction and feature selection methods as well as classification methods for automatic facial expression recognition (FER) system. The FER system is fully automatic and consists of the following modules: face detection, facial detection, feature extraction, selection of optimal features, and classification. Face detection is based on AdaBoost algorithm and is followed by the extraction of frame with the maximum intensity of emotion using the inter-frame mutual information criterion. The selected frames are then processed to generate characteristic features using different methods including: Gabor filters, log Gabor filter, local binary pattern (LBP) operator, higher-order local autocorrelation (HLAC) and a recent proposed method called HLAC-like features (HLACLF). The most informative features are selected based on both wrapper and filter feature selection methods. Experiments on several facial expression databases show comparisons of different methods.     

Ensemble convolutional neural networks for automatic fusion recognition of multi‐platform radar emitters

Presently, the extraction of hand‐crafted features is still the dominant method in radar emitter recognition. To solve the complicated problems of selection and updation of empirical features, we present a novel automatic feature extraction structure based on deep learning. In particular, a convolutional neural network (CNN) is adopted to extract high‐level abstract representations from the time‐frequency images of emitter signals. Thus, the redundant process of designing discriminative features can be avoided. Furthermore, to address the performance degradation of a single platform, we propose the construction of an ensemble learning‐based architecture for multi‐platform fusion recognition. Experimental results indicate that the proposed algorithms are feasible and effective, and they outperform other typical feature extraction and fusion recognition methods in terms of accuracy. Moreover, the proposed structure could be extended to other prevalent ensemble learning alternatives.     

Facial expression recognition in image sequences using geometric deformation features and Support Vector Machines.

In this paper, two novel methods for facial expression recognition in facial image sequences are presented. The user has to manually place some of Candide grid nodes to face landmarks depicted at the first frame of the image sequence under examination. The grid-tracking and deformation system used, based on deformable models, tracks the grid in consecutive video frames over time, as the facial expression evolves, until the frame that corresponds to the greatest facial expression intensity. The geometrical displacement of certain selected Candide nodes, defined as the difference of the node coordinates between the first and the greatest facial expression intensity frame, is used as an input to a novel multiclass Support Vector Machine (SVM) system of classifiers that are used to recognize either the six basic facial expressions or a set of chosen Facial Action Units (FAUs). The results on the Cohn-Kanade database show a recognition accuracy of 99.7% for facial expression recognition using the proposed multiclass SVMs and 95.1% for facial expression recognition based on FAU detection.     

Multimodal emotion recognition based on peak frame selection from video

We present a fully automatic multimodal emotion recognition system based on three novel peak frame selection approaches using the video channel. Selection of peak frames (i.e., apex frames) is an important preprocessing step for facial expression recognition as they contain the most relevant information for classification. Two of the three proposed peak frame selection methods (i.e., MAXDIST and DEND-CLUSTER) do not employ any training or prior learning. The third method proposed for peak frame selection (i.e., EIFS) is based on measuring the “distance” of the expressive face from the subspace of neutral facial expression, which requires a prior learning step to model the subspace of neutral face shapes. The audio and video modalities are fused at the decision level. The subject-independent audio-visual emotion recognition system has shown promising results on two databases in two different languages (eNTERFACE and BAUM-1a).     

面向人脸表情识别的迁移卷积神经网络研究

人脸表情识别是模式识别研究的一个重要领域,现实环境中人脸表情识别容易受到光照、姿态、个体表情差异等因素的影响,识别效果仍有待提高。为了取得更好的人脸表情识别效果,本文提出一种基于迁移卷积神经网络的人脸表情识别方法,本文在训练得到人脸识别网络模型的基础上,采用迁移学习方法将所得人脸识别模型迁移到人脸表情识别任务上,并提出Softmax-MSE损失函数和双激活层(Double Activate Layer, DAL)结构,以提高模型的识别能力。在FER2013数据库和SFEW2.0数据库上的实验表明,本文所提方法分别取得了61.59%和47.23%的主流识别效果。      

基于稀疏特征挑选和概率线性判别分析的表情识别研究

提出一种基于稀疏特征挑选（Sparse selection）和概率线性判别分析（Probabilistic linear discriminant analysis）的表情识别方法SS-PLDA.该方法由两部分构成：第一部分是使用稀疏的方法挑选出人脸与表情相关的区域,构造表情的完备特征集;第二部分是针对构造的表情完备特征集里仍含有一些其他信息,运用概率线性判别分析实现表情特征与干扰信息的分离,学习出一个只含有表情信息的子空间,最后基于该表情子空间进行表情识别分析.通过在CK+和JAFFE这两个数据库上面的实验,证实了基于稀疏特征挑选的方法可以得到识别性能的改善,且先使用特征挑选再对所挑选结果应用概率线性判别分析可以达到更好的提升效果.     

Discriminative Wavelet Packet Filter Bank Selection for Pattern Recognition

This paper addresses the problem of discriminative wavelet packet (WP) filter bank selection for pattern recognition. The problem is formulated as a complexity regularized optimization criterion, where the tree-indexed structure of the WP bases is explored to find conditions for reducing this criterion to a type of minimum cost tree pruning, a method well understood in regression and classification trees (CART). For estimating the conditional mutual information, adopted to compute the fidelity criterion of the minimum cost tree-pruning problem, a nonparametric approach based on product adaptive partitions is proposed, extending the Darbellay-Vajda data-dependent partition algorithm. Finally, experimental evaluation within an automatic speech recognition (ASR) task shows that proposed solutions for the WP decomposition problem are consistent with well understood empirically determined acoustic features, and the derived feature representations yield competitive performances with respect to standard feature extraction techniques.     

Speaker identification based on adaptive discriminative vector quantisation

A novel adaptive discriminative vector quantisation technique for speaker identification (ADVQSI) is introduced. In the training mode of ADVQSI, for each speaker, the speech feature vector space is divided into a number of subspaces. The feature space segmentation is based on the difference between the probability distribution of the speech feature vectors from each speaker and that from all speakers in the speaker identification (SI) group. Then, an optimal discriminative weight, which represents the subspace's role in SI, is calculated for each subspace of each speaker by employing adaptive techniques. The largest template differences between speakers in the SI group are achieved by using optimal discriminative weights. In the testing mode of ADVQSI, discriminative weighted average vector quantisation (VQ) distortions are used for SI decisions. The performance of ADVQSI is analysed and tested experimentally. The experimental results confirm the performance improvement employing the proposed technique in comparison with existing VQ techniques for SI and recently reported discriminative VQ techniques for SI (DVQSI)     

多流形判别分析在特征提取中的应用研究

为了更好地提高人脸识别率及其识别效率,提出了一种基于多流形判别分析(MMDA)的图像特征提取方法.在MM-DA方法中,为了寻求能够同时最大化类间散布矩阵和最小化类内散布矩阵的判别矩阵,类间、类内分布图分别被用来描述类间和类内的分离性,类内图可以表示子流形的信息,而类间图可以代表多流形的信息,从而更好地实现分类.在ORL及FERET人脸数据库上进行实验,结果表明了MMDA方法在特征提取中的有效性.     

基于改进二次误差测度及网格参数化的三维人脸本征属性描述方法

3维人脸特征描述是3维人脸配准及识别的关键技术。该文针对3维人脸高分辨率模型特征分布不均匀且存在信息冗余的问题,提出一种基于模型简化和网格参数化的3维人脸特征描述方法。采用半边折叠及自适应收缩代价加权等手段对基于二次误差测度的网格简化方法进行改进,克服原算法中存在重叠三角形和丢失细节特征的问题。同时,基于多分辨分析思想,利用特征约束的保形同构映射对简化后的3维人脸模型在2维平面进行保形展开,并由此构造多分辨2维本征属性图。该方法将3维空间运算问题简化为简单的2维图像运算,显著降低了计算复杂度。对GavabDB 3维人脸库的识别实验表明,该文方法能有效描述3维人脸的本征属性,同时对数据缺失具有较强的鲁棒性。     

基于时空注意力网络的中国手语识别

手语识别广泛应用于聋哑人与正常人之间的交流中。针对手语识别任务中时空特征提取不充分而导致识别率低的问题,提出了一种新颖的基于时空注意力的手语识别模型。首先提出了基于残差3D卷积网络(Residual 3D Convolutional Neural Network,Res3DCNN)的空间注意力模块,用来自动关注空间中的显著区域；随后提出了基于卷积长短时记忆网络(Convolutional Long Short-Term Memory,ConvLSTM)的时间注意力模块,用来衡量视频帧的重要性。所提算法的关键在于在空间中关注显著区域,并且在时间上自动选择关键帧。最后,在CSL手语数据集上验证了算法的有效性。     

Structural similarity classifier for facial expression recognition

A novel Gabor filter structural similarity algorithm (GFSSIM) is proposed for facial expression recognition (FER) on noisy images. Low-resolution facial images with low SNRs are specifically dealt with FER system. The features are extracted using 40 Gabor filters, and a feature subset is selected for classification. The test image is classified based on proposed GFSSIM algorithm. The experimental results show that the recognition rate for heavily deteriorated images outperforms the conventional classifier method. In addition, the proposed method is more efficient from the computational complexity point of view.     

基于GLCM、CGA和Adaboost的人脸表情识别方法研究

提出基于灰度共生矩阵(GLCM)和混沌遗传优化算法(CGA)的人脸表情识别方法(FER)。为了消除遗传算法中个体在解空间内分布的不均匀性,利用混沌的随机性、遍历性和规律性,将混沌引入到遗传算法中,由此得到了混沌遗传优化算法(CGA);通过灰度共生矩阵提取出的特征和改进后的混沌遗传优化算法,将人脸表情识别的寻找感兴趣区域(ROI)和特征提取结合成一步;利用Adaboost算法进行图像分类。经过理论和实验证明,该方法实现简单、切实可行。     

LVCSR系统中一种基于区分性和自适应瓶颈深度置信网络的特征提取方法

大词汇量连续语音识别系统中,为了进一步增强网络的鲁棒性、提升瓶颈深度置信网络的识别准确率,本文提出一种基于区分性和自适应瓶颈深度置信网络的特征提取方法。该方法首先使用鲁棒性较强的瓶颈深度置信网络进行初步特征提取,进而进行区分性训练,使网络的区分性更强、识别准确率更高,在此基础上引入说话人自适应技术对网络进行调整,提高系统的鲁棒性。本文利用提出的声学特征在多个噪声较强、主题风格较为随意的多个公共连续语音数据库上进行了测试,识别准确率取得了6.9%的提升。实验结果表明所提出的特征提取方法相对于传统方法的优越性。      

改进非负矩阵分解的神经网络人脸识别

为了提高人脸识别正确率,提出基于改进非负矩阵分解的神经网络人脸识别算法。首先利用改进的非负矩阵分解对人脸图像进行特征提取,提高非负矩阵分解速度。接着将提取出的特征信息作为神经网络学习入口进行特征训练,由于神经网络在学习过程中,容易出现局部最小值且收敛速度慢等问题,为此采用改进的遗传算法对神经网络进行优化处理,获得最终的人脸识别结果。实验结果表明:利用改进的非负矩阵分解方法能够降低神经网络的分类训练负荷量和运算量,提高人脸识别识别率。通过和各种方法比较可知,本方法的人脸识别率都较高。本方法人脸特征分解速度快,提高了神经网络训练前期精度和收敛速度,使得人脸识别正确率高。当特征向量个数达到40以上时,人脸识别正确率保持95%以上。     

Low-resolution expression recognition based on central oblique average CS-LBP with adaptive threshold

In order to solve the problem of low recognition rate of traditional feature extraction operators under low-resolution images, a novel algorithm of expression recognition is proposed, named central oblique average center-symmetric local binary pattern (CS-LBP) with adaptive threshold (ATCS-LBP). Firstly, the features of face images can be extracted by the proposed operator after pretreatment. Secondly, the obtained feature image is divided into blocks. Thirdly, the histogram of each block is computed independently and all histograms can be connected serially to create a final feature vector. Finally, expression classification is achieved by using support vector machine (SVM) classifier. Experimental results on Japanese female facial expression (JAFFE) database show that the proposed algorithm can achieve a recognition rate of 81.9% when the resolution is as low as 16×16, which is much better than that of the traditional feature extraction operators.     

基于多尺度细节增强的面部表情识别方法

人类面部表情是其心理情绪变化的最直观刻画,不同人的面部表情具有很大差异,现有表情识别方法均利用面部统计特征区分不同表情,其缺乏对于人脸细节信息的深度挖掘。根据心理学家对面部行为编码的定义可以看出,人脸的局部细节信息决定了其表情意义。因此该文提出一种基于多尺度细节增强的面部表情识别方法,针对面部表情受图像细节影响较大的特点,提出利用高斯金字塔提取图像细节信息,并对图像进行细节增强,从而强化人脸表情信息。针对面部表情的局部性特点,提出利用层次结构的局部梯度特征计算方法,描述面部特征点局部形状特征。最后,使用支持向量机(SVM)对面部表情进行分类。该文在CK+表情数据库中的实验结果表明,该方法不仅验证了图像细节对面部表情识别过程的重要作用,而且在小规模训练数据下也能够得到非常好的识别结果,表情平均识别率达到98.19%。