基于线性链条件随机场的用户生成文本标点标注

标点符号的正确性对于用户生成文本的词性标注,命名实体识别,依存句法分析等有着重要的作用,正确的标点标注可以使用户生成文本的语法结构准确完整。线性链条件随机场模型可以容纳任意的非独立的特征信息,本文通过选取标点符号所在位置左右词性对作为模型的观测序列,使用条件随机场进行标点符号的标注。实验的测试语料采用京东在线产品评论,结果表明基于线性链条件随机场的用户生成文本标点标注效率较高。

User-generated Text Punctuation Labelling Based on Linear Chain Conditional Random Field

The correctness of punctuation marks plays an important role in the part-of-speech tagging of user-generated texts, named entity recognition, and dependency parsing,proper punctuation labelling can make the grammatical structure of user-generated text accurate and complete. The linear chain condition random field model can accommodate any non-independent feature information,in this paper, we use the left and right part-of-speech of the position of the punctuation as the observation sequence of the model, and the conditional random field is used to label the punctuation .The test corpus of the experiment uses Jingdong online product reviews, and the results show that the user-generated text punctuation labelling based on the linear chain conditional random field is more efficient.