基于生成式对抗网络的鲁棒人脸表情识别

人们在自然情感交流中经常伴随着头部旋转和肢体动作，它们往往导致较大范围的人脸遮挡，使得人脸图像损失部分表情信息.现有的表情识别方法大多基于通用的人脸特征和识别算法，未考虑表情和身份的差异，导致对新用户的识别不够鲁棒.本文提出了一种对人脸局部遮挡图像进行用户无关表情识别的方法.该方法包括一个基于Wasserstein生成式对抗网络（Wasserstein generative adversarial net，WGAN）的人脸图像生成网络，能够为图像中的遮挡区域生成上下文一致的补全图像；以及一个表情识别网络，能够通过在表情识别任务和身份识别任务之间建立对抗关系来提取用户无关的表情特征并推断表情类别.实验结果表明，我们的方法在由CK+，Multi-PIE和JAFFE构成的混合数据集上用户无关的平均识别准确率超过了90%.在CK+上用户无关的识别准确率达到了96%，其中4.5%的性能提升得益于本文提出的对抗式表情特征提取方法.此外，在45°头部旋转范围内，本文方法还能够用于提高非正面表情的识别准确率.

Robust Facial Expression Recognition With Generative Adversarial Networks

In natural communication, people would express their expressions with head rotation and body movement, which may result in partial occlusion of face and a consequent information loss regarding facial expression. Also, most of the existing approaches to facial expression recognition are not robust enough to unseen users because they rely on general facial features or algorithms without considering differences between facial expression and facial identity. In this paper, we propose a person-independent recognition method for partially-occluded facial expressions. Based on Wasserstein generative adversarial net (WGAN), a generative network of facial image is trained to perform context-consistent image completion for partially-occluded facial expression images. With an adversarial learning strategy, furthermore, a facial expression recognition network and a facial identity recognition network are established to improve the accuracy and robustness of facial expression recognition via inhibition of intra-class variation. Extensive experimental results demonstrate that 90% average recognition accuracy of facial expression has been reached on a mixed dataset composed of CK+, Multi-PIE, and JAFFE. Moreover, our method achieves 96% accuracy of user-independent recognition on CK+. A 4.5% performance gain is achieved with the novel identity-inhibited expression feature. Our method is also capable of improving recognition accuracy for non-frontal facial expressions within a range of 45-degree head rotation.