基于非文本模态强化和门控融合方法的多模态情感分析

针对各模态之间信息密度存在差距和融合过程中可能会丢失部分情感信息等问题，提出一种基于非文本模态强化和门控融合方法的多模态情感分析模型。该模型通过设计一个音频-视觉强化模块来实现音频和视觉模态的信息增强，从而减小与文本模态的信息差距。之后，通过跨模态注意力和门控融合方法，使得模型充分学习到多模态情感信息和原始情感信息，从而增强模型的表达能力。在对齐和非对齐的CMU-MOSEI数据集上的实验结果表明，所提模型是有效的，相比现有的一些模型取得了更好的性能。

Multimodal sentiment analysis based on non-text modality reinforcement and gating fusion method

To address the problems of information density gaps between modalities and the possibility of losing some sentiment information in the fusion process, this paper proposed a multimodal sentiment analysis model based on non-text modality reinforcement and gating fusion method. The model reduced the gap with text modality by designing an audio-visual reinforcement module to achieve information enhancement of audio and visual modalities. Then, the cross-modal attention and gating fusion method allowed the model to fully learn the multimodal sentiment information and the original sentiment information to enhance the representation of the model. Experimental results on the aligned and unaligned CMU-MOSEI datasets show that the proposed model is effective and achieves better performances than some existing models.