基于众包标注的中文微博命名实体识别

中文微博命名实体的有效识别对使用微博进行社会舆论监测具有重要意义。鉴于微博更新速度快、语言不规范、噪声多,使得命名实体识别成本高、识别效率低。针对这些问题,提出基于众包标注的中文微博命名实体识别的方法。对众包工作者的能力进行评估,使用最大期望算法(EM算法)对评估后的能力值进行分析学习,过滤掉每个标注者的噪声并对众包标注的结果进行优化,从而确定最后的命名实体。实验结果表明,该方法能够有效地提高中文微博中命名实体识别的准确率。

NAMED ENTITY RECOGNITION IN CHINESE MICROBLOG BASED ON CROWDSOURCED LABELING

Effective identification of named entities in Chinese microblog is of great significance to the monitoring of public opinion by using microblog. However, microblog has the characteristics of fast updating speed, nonstandard language and lots of noise, which leads to the high cost and low efficiency of named entity recognition. This paper proposed a Chinese microblog named entity recognition based on crowdsourced labeling. We evaluated the ability of the crowdsourcing workers, then we used the maximum expected algorithm to analyze and learn the ability value after evaluation, and to filter out the noise of each tagger and optimize the results of crowdsourcing, so as to determine the last named entity. The experimental results show that this method can effectively improve the accuracy of named entity recognition in Chinese microblog.