汉语词性标注的特征工程

上下文特征对汉语词性标注性能有重要影响。为了提高标注性能,采用最大熵模型探讨了汉语词性标注的特征工程,对其中的两个关键问题:特征窗口大小和特征模板集的设定,本文作者进行了深入研究。在Bakeoff2007的PKU、NCC、CTB 3种语料上进行了封闭测试,通过对“5词语”和“3词语”不同大小的特征窗口,以及单词语、双词语和两者混合的不同特征模板集进行汉语词性标注的训练过程和标注精度的对比实验,实验结果表明:3词特征窗口训练情况和标注性能均优于5词窗口;单词语特征模板集比双词语特征模板集标注性能高出10％。这说明汉语词性标注中特征窗口开设的大小以3词窗口为宜,单词语特征模板集标注性能更好。

Feature engineering for Chinese part-of-speech tagging

Context features have a major impact on the performance of Chinese part-of-speech tagging. In order to improve the performance, the feature engineering for Chinese part-of-speech tagging was explored by the using maximum entropy model. Two key issues of feature engineering, the size of the feature window and the feature templates, were studied. Closed evaluations were performed on PKU, NCC and CTB corpus from the Bakeoff-2007. Then, comparative experiments about the training process and tagging accuracy for Chinese part-of-speech tagging were performed on different feature windows, the “5 words” and “3 words” feature windows, and different feature templates: single-word, double word and mixing feature templates. Experimental results showed that the feature window including 3 words was better than that of 5 words, and the performance increased 10％ using single-word feature templates than double-word feature templates. All the results showed that the feature window including 3 words and single-word feature templates were appropriate for Chinese part-of-speech tagging.