基于双层解码的多轮情感对话生成模型

情感对话系统的成功取决于语言理解、情感感知和表达能力，同时面部表情和个性等也能提供帮助。然而，尽管这些信息对于多轮情感对话至关重要，但是现有系统既未能够充分利用多模态信息的优势，又忽略了上下文相关性的重要性。为了解决这个问题，提出了一种基于双层解码的多轮情感对话生成模型（MEDG-DD）。该模型利用异构的图神经网络编码器将历史对话、面部表情、情感流和说话者信息进行融合，以获得更加全面的对话上下文。然后，使用基于注意力机制的双层解码器，以生成与对话上下文相关的富含情感的言辞。实验结果表明，该模型能够有效地整合多模态信息，实现更为准确、自然且连贯的情感话语。与传统的ReCoSa模型相比，该模型在各项评估指标上均有显著的提升。

Multi-turn emotion dialogue generation model based on dual-decoder

The success of emotional dialogue systems relies on the ability to comprehend, perceive, and express emotions, while facial expressions and personality can also help. However, despite the crucial importance of this multi-modal information in multi-turn emotional dialogues, existing systems still need to be improved to leverage multi-modal information''s advantages and overlook the significance of contextual relevance. To address this issue, this paper proposed a multi-turn emotional dialogue generation model based on a dua-decoding method(MEDG-DD). The model utilized a heterogeneous graph neural network encoder to integrate historical dialogue, facial expressions, emotion flow, and speaker information, obtaining a more comprehensive dialogue context. Subsequently, it employed a dual-decoding mechanism based on attention to generate emotionally rich expressions relevant to the dialogue context. Experimental results demonstrate that the proposed model effectively integrates multi-modal information, achieving more accurate, natural, and coherent emotional expressions. Compared to the traditional ReCoSa model, this model exhibits significant improvements across various evaluation metrics.