TP-FER：基于优化卷积神经网络的三通道人脸表情识别方法

针对人脸五官在表达不同情绪时所起的作用不同,利用单一的卷积神经网络对人脸面部特征进行特征提取和表情识别可能会导致提取表情关键特征信息时聚焦性不够,而仅对眼部或者嘴部等重点部位进行特征提取,又有可能造成特征提取不够充分的问题,提出了一种基于优化卷积神经网络的三通道人脸表情识别方法TP-FER(tri-path networks for facial expression recognition).该方法基于构建的卷积神经网络训练,采用三个输入渠道,分别聚焦面部、眼部和嘴部区域进行特征提取和表情判别,最后采用基于决策层的融合技术将三个渠道的识别结果进行相对多数投票决策,获取整体最优识别率.将此方法应用于JCK+数据集和自建数据集上进行了实验判别分析,结果表明该方法在两个数据集上均提高了整体表情识别率.该方法既考虑了脸部整体特征的提取,又兼顾了某些表情主要聚焦在眼部、嘴部表达的特性,相互辅助,整体提高了表情的识别率;该方法也能对神经心理学研究提供数据支持.

TP-FER:facial expression recognition method of tri-path networks based on optimal convolutional neural network

Facial features play different functions in expressing different emotions, the only use of convolutional neural network for feature extraction and expression recognition of facial features may not lead to enough focus on the key feature information of facial expressions. However, the feature information only extracting from the key parts such as eyes or mouth may not be sufficient. This paper proposed a facial expression recognition method of tri-path networks based on the optimal convolutional neural network called TP-FER. Based on the convolutional neural network, this method used three input channels to focus on the face, eyes and mouth regions respectively for feature extraction and expression discrimination. Then it used the fusion technology based on the decision layer to make the final decision. At last'' it adopted relative majority voting method based on the recognition results of the three channels to obtain the overall optimal recognition rate. This paper conducted several experiments on JCK+dataset and self-built dataset with this method. The results show that the overall expression recognition rate is improved in both datasets. This method can make full use of the different channel characteristics that focus on not only the key features such as eyes and mouth but also the comprehensive information of the whole face, and improve the expression recognition rate effectively. In addition, it can also provide data support for neuropsychological related research.