N-Reader:基于双层Self-attention的机器阅读理解模型

机器阅读理解是自然语言处理和人工智能领域的重要前沿课题,近年来受到学术界和工业界的广泛关注。为了提升机器阅读理解系统在多文档中文数据集上的处理能力,我们提出了N-Reader,一个基于神经网络的端到端机器阅读理解模型。该模型的主要特点是采用双层self-attention机制对单个文档以及所有输入文档进行编码。通过这样的编码,不仅可以获取单篇文档中的关键信息,还可以利用多篇文档中的相似性信息。另外,我们还提出了一种多相关段落补全算法来对输入文档进行数据预处理。这种补全算法可以帮助模型识别出具有相关语义的段落,进而预测出更好的答案序列。使用N-Reader模型,我们参加了由中国中文信息学会、中国计算机学会和百度公司联手举办的“2018机器阅读理解技术竞赛”,取得了第3名的成绩。

N-Reader: Machine Reading Comprehension Model Based on Double Layers of Self-attention

Machine reading comprehension (MRC) is an important task in natural language processing and artificial intelligence. To improve the Chinese multi-document MRC, this paper proposes N-Reader, an end-to-end MRC model based on neural network. It applies a two-layer self-attention mechanism to encode the input documents to ultilize both the information from a single document and the similarity information from multiple documents. Besides, this paper also proposes a multi-paragraph completion algorithm to preprocess the input documents. This preprocessing method can further recognize the semantics-related paragraphs among input documents, and contribute to a better answer sequence. In the “2018 NLP Challenge on Machine Reading Comprehension” jointly organized by Chinese Information Processing Society of China (CIPS), Chinese Computer Federation (CCF), and Baidu Inc., our model ranks No.3 among the hugely competitive models.