面向聊天机器人的多注意力记忆网络

如何对多轮的对话历史进行建模和推理是构建一个智能聊天机器人的主要挑战之一.基于循环或门控的记忆网络已经被证明是进行对话建模的有效方式.然而，这种方式有两个缺点，一是使用复杂的循环结构，导致计算效率较低；二是使用代价较大的强监督信息或先验信息，不利于扩展和迁移应用到新的领域.针对上述问题，本文提出了一种端到端的多注意力记忆网络.首先，该网络采取结合词向量和位置编码的方式对文本输入进行表示；其次，使用并行的多层注意力在不同子空间捕获对话交互中的关键信息来更好地建模对话历史；最后，通过捷径连接的方式叠加多注意力层管理信息流，实现对建模结果的多次推理.在bAbI-dialog数据集上的实验表明，该网络可以有效地对多轮对话进行建模和推理，而且具有较好的时间性能.

Memory Network with Multi-Head Attention for Chatbot

Modeling and reasoning about the multi-turn dialogue history is a main challenge for building an intelligent chatbot. Memory Networks with recurrent or gated architectures have been demonstrated promising for conversation modeling. However, it still suffers from two drawbacks, one is relatively low computational efficiency for its complex architectures, the other is costly strong supervision information or fixed priori knowledge, which hinders its extension and application to new domains. This paper proposes an end-to-end memory network with multi-head attention. Firstly, the model adopts a method using word embedding combined with position encoding to represent text input; Secondly, it uses multi-head attention to capture important information in different subspaces of conversational interactions. Finally, multi-layered attention is stacked via shortcut connections to achieve repeatedly reasoning over the modeling result. Experiments on the bAbI-dialog datasets show that the network can effectively model and reason for multi-turn dialogue and has a better time performance.