融合局部特征与两阶段注意力权重学习的面部表情识别

面部的局部细节信息在面部表情识别中扮演重要角色,然而现有的方法大多只关注面部表情的高层语义信息而忽略了局部面部区域的细粒度信息。针对这一问题,提出一种融合局部特征与两阶段注意力权重学习的深度卷积神经网络FLF-TAWL(deep convolutional neural network fusing local feature and two-stage attention weight learning),它能自适应地捕捉重要的面部区域从而提升面部表情识别的有效性。该FLF-TAWL由双分支框架构成,一个分支从图像块中提取局部特征,另一个分支从整个表情图像中提取全局特征。首先提出了两阶段注意力权重学习策略,第一阶段粗略学习全局和局部特征的重要性权重,第二阶段进一步细化注意力权重,并将局部和全局特征进行融合;其次,采用一种区域偏向损失函数鼓励最重要的区域以获得较高的注意力权重。在FERPlus、Cohn-Kanada(CK+)以及JAFFE三个数据集上进行了广泛实验,分别获得90.92%、98.90%、97.39%的准确率,实验结果验证了FLF-TAWL模型的有效性和可行性。

Deep convolutional neural network fusing local feature and two-stage attention weight learning for facial expression recognition

Facial local detail information plays an important role in facial expression recognition(FER).However, most of the existing methods only focus on the high-level semantic information of facial expressions, while ignoring the fine-grained information of local facial regions.To solve this problem, this paper proposed a deep convolutional neural network fusing local feature and two-stage attention weight learning(FLF-TAWL),which could adaptively capture important facial regions to improve the effectiveness of facial expression recognition.The FLF-TAWL model was composed of a dual-branch framework, one branch extracted local features from image blocks, and the other branch extracted global features from the entire expression image.Firstly, this paper proposed a two-stage attention weight learning strategy.In the first stage, it roughly learned the importance weights of global and local features, in the second stage, it further refined the attention weight, and fused the local and global features.Secondly, the model used a region-biased loss function to encourage the most important regions to obtain higher attention weights.Finally, this paper carried out extensive experiments on FERPlus, Cohn-Kanada(CK+) and JAFFE datasets to obtain accuracy rates of 90.92%,98.90% and 97.39% respectively.The experimental results verify the effectiveness and feasibility of the FLF-TAWL model.