双流网络信息交互机制下的微表情识别

针对深度学习的方法用于微表情识别时微表情识别的实验数据库非常稀缺,导致神经网络在学习的过程中知识获取有限而难以提高精度及泛化能力的问题,提出基于双流网络信息交互的微表情识别方法.通过改进的深度互学习策略引导图像序列不同模态之间的交互训练,提高网络的识别率.方法基于RGB图像序列建立主体网络,基于光流建立辅助网络;在训练阶段,通过设计互学习损失中的有监督学习损失和拟态损失,优化训练过程,使得每一种模态都能学习正确地预测训练样本的真实标识,同时能与其他模态的预测相匹配;在测试阶段,由于互学习机制增强了RGB分支的判别能力,因此可对光流分支进行剪裁,在保证精度的前提下提高识别速度.在CASME,CASMEⅡ和SMIC数据库上的实验结果表明,该方法有效地提高了识别精度,整体性能优于已有方法.

Micro-Expression Recognition Based on Dual-Stream Networks Information Interaction

Scarcity of experimental databases for deep learning based micro-expression recognition leads to limited knowledge acquisition in learning process and increasing difficulty for improving accuracy and generalization capabilities. In view of the problem, a micro-expression algorithm based on dual-stream network information interaction is proposed. An improved deep mutual learning strategy is designed to guide the interactive training between different modalities of the image sequence to improve the recognition rate of the network, in which the main network is built based on the RGB image sequence, and the auxiliary network is built based on the optical flow. In the training phase, the training process is optimized via a mutual learning loss which includes the supervised learning loss and mimic loss, which ensures the prediction of each mode is consistent with the true identity of the training sample and the predictions of other modalities. In the testing phase, since the mutual learning mechanism enhances the discrimination ability of RGB branches, the optical flow branches can be tailored to improve the recognition speed while ensuring accuracy. The experimental results on the CASME, CASME Ⅱ and SMIC data sets show that the algorithm effectively improves the recognition accuracy and the overall performance is better than existing algorithms.