基于自注意力机制的中文标点符号预测模型

中文标点符号预测是自然语言处理的一项重要任务,能够帮助人们消除歧义,更准确地理解文本。为解决传统自注意力机制模型不能处理序列位置信息的问题,提出一种基于自注意力机制的中文标点符号预测模型。在自注意力机制的基础上堆叠多层Bi-LSTM网络,并结合词性与语法信息进行联合学习,完成标点符号预测。自注意力机制可以捕获任意两个词的关系而不依赖距离,同时词性和语法信息能够提升预测标点符号的正确率。在真实新闻数据集上的实验结果表明,该模型F1值达到85.63%,明显高于传统CRF、LSTM预测方法,可实现对中文标点符号的准确预测。

Prediction Model of Chinese Punctuation Based on Self-Attention Mechanism

Chinese Punctuation Prediction(PP)is an important task of Natural Language Pprocessing(NLP),which can help people eliminate ambiguity and understand the text more accurately.In order to solve the problem that the self-attention mechanism cannot process sequence position information,this paper proposes a Chinese punctuation prediction model based on the self-attention mechanism.This model stacks multiple layers of Bi-directional Long Short-Term Memory(Bi-LSTM)network on the basis of self-attention mechanism,and combines the part of speech and grammar information for joint learning to complete the punctuation prediction.The self-attention mechanism can capture the relationship between any two words without relying on their distance,and the accuracy of predicted punctuation can be improved by part of speech and grammatical information.Experimental results on real news datasets show that the F1 value of the proposed model reaches 85.63%,which is significantly higher than traditional CRF and LSTM prediction methods,and achieves accurate prediction of Chinese punctuation.