基于感知重采样和多模态融合的连续情感识别

情感识别在人机交互中发挥着重要的作用，连续情感识别因其能检测到更广泛更细微的情感而备受关注。在多模态连续情感识别中，针对现有方法获取的时序信息包含较多冗余以及多模态交互信息捕捉不全面的问题，提出基于感知重采样和多模态融合的连续情感识别方法。首先感知重采样模块通过非对称交叉注意力机制去除模态冗余信息，将包含时序关系的关键特征压缩到隐藏向量中，降低后期融合的计算复杂度。其次多模态融合模块通过交叉注意力机制捕捉模态间的互补信息，并利用自注意力机制获取模态内的隐藏信息，使特征信息更丰富全面。在Ulm-TSST和Aff-Wild2数据集上唤醒度和愉悦度的CCC均值分别为63.62%和50.09%，证明了该模型的有效性。

Continuous emotion recognition based on perceiver resampling and multimodal fusion

Emotion recognition plays a crucial role in human-computer interaction, and continuous emotion recognition has gained significant attention due to its ability to capture a broader range of emotions, including more subtle ones. In the field of multimodal continuous emotion recognition, this paper proposed a continuous emotion recognition method based on perceiver resampling and multimodal fusion for the problems that the temporal series information obtained by the existing methods contains more redundancy and the obtained multimodal interactive information is not comprehensive. Firstly, the perceiver resampling module removed redundant information, focused on key information, compressed the key features with temporal relationships into hidden vectors, and reduced the computational complexity of the later fusion. Secondly, the multimodal fusion module captured the interactive information between modalities through cross-attention mechanism, and used the self-attention mechanism to obtain the hidden information within each modality, so as to make the feature information richer and more comprehensive. The mean CCC values of arousal and valence on the Ulm-TSST and Aff-Wild2 datasets are 63.62% and 50.09%, respectively, which prove the effectiveness of the model.