基于组块及记忆的词性自动标注

基于组块及记忆的模型(BMM)采用与传统方法明显不同的标注思路,以汉语中的整句为处理单元,从组块出发,立足于单个词汇,分析更为丰富的上下文语境知识,并借助知网词典记忆词性集合,同时采用渐增式的机械学习方式获取参数值。对于棘手的稀疏数据问题只简单地设置平伏常数加以平滑,最后利用少量人工规则修正标注结果。实验表明,该模型的封闭式测试准确率将近99%,开放式测试准确率为95%以上。

Block and memory based part of speech tagging

Automatic part-of-speech tagging is often applied to natural language processing.The approach of Block and Memory based Model(BMM) is other than that of the traditional models.BMM takes a whole Chinese sentence as a processing unit.Each word is considered respectively in a more abundant and informative context environment.The lexicon of WordNet is employed to store the tag sets,and,to improve the efficiency,the incremental learning method is applied to obtain parameters.A constant is given to smooth the sparse data and some handcrafting rules are used to amend the results.Experiments show that the accuracy of close test is about 99% and the accuracy of open test is higher than 95%.