基于交互注意力机制的多模态情感识别算法

在多模态语音情感识别中,现有的研究通过提取大量特征来识别情感,但过多的特征会导致关键特征被淹没在相对不重要特征里,造成关键信息遗漏.为此提出了一种模型融合方法,通过两种注意力机制来寻找可能被遗漏的关键特征.本方法在IEMOCAP数据集上的四类情感识别准确率相比现有文献有明显提升;在注意力机制可视化下,两种注意力机制分别找到了互补且对人类情感识别重要的关键信息,从而证明了所提方法相比传统方法的优越性.

Multimodal emotion recognition algorithm based on interactive attention mechanism

In multimodal emotion recognition, the proposed researches focus on extracting more and more features to classify emotion. But too many features which is relatively not important may flood the important feature and due to key information lost. This paper proposed a model fusion algorithm(GATASA) which based on multi-attention mechanism. It used two attention mechanisms to find out important feature which might be neglect. The proposed interactive attention algorithm can effectively improve recognition accuracy in the four emotions classification on IEMOCAP dataset. By using attention visualization, those two attention mechanisms find complementary and important information from human perspective which proves that the proposed GATASA is advanced than traditional algorithm.