基于数据增广和复制的中文语法错误纠正方法

中文作为一种使用很广泛的文字,因其同印欧语系文字的天然差别,使得汉语初学者往往会出现各种各样的语法错误。本文针对初学者在汉语书写中可能出现的错别字、语序错误等,提出一种自动化的语法纠正方法。首先,本文在自注意力模型中引入复制机制,构建新的C-Transformer模型。构建从错误文本序列到正确文本序列的文本语法错误纠正模型,其次,在公开数据集的基础上,本文利用序列到序列学习的方式从正确文本学习对应的不同形式的错误文本,并设计基于通顺度、语义和句法度量的错误文本筛选方法;最后,还结合中文象形文字的特点,构造同形、同音词表,按词表映射的方式人工构造错误样本扩充训练数据。实验结果表明,本文的方法能够很好地纠正错别字、语序不当、缺失、冗余等错误,并在中文文本语法错误纠正标准测试集上取得了目前最好的结果。

Chinese grammatical error correction method based on data augmentation and copy mechanism

Chinese is a widely used language. However, due to its natural difference between Indo-European languages, Chinese learners tend to make various grammatical errors. This article proposes an automatic grammar correction method for those who will make errors like typos or improper words order. First, we built the C-Transformer model that adopts copy mechanism in the self-attention model to translate wrong text sequence to the correct one. Second, based on the public data set, a pure sequence to sequence method is utilized to generate wrong text corresponding to the correct one, and an error text filter is designed based on fluency, semantic, and syntactic measurements. Finally, since Chinese words are pictographic, based on the collected homographs and homophones dictionaries, some error samples are artificially constructed to expand training data. The experimental results show that our method can well correct typos, improper word order, missing, redundancy and other errors, and achieved the state-of-the-art performance on the standard test set of Chinese text grammatical error correction.