基于语言模型验证的词义消歧语料获取

作为一种稀缺资源,人工标注语料的匮乏限制了有指导词义消歧系统的大规模应用。有人提出了利用目标词的单义同义词在生语料中自动获取词义消歧语料的方法,然而,在某些上下文当中,用目标词替换这些单义的同义词并不合适,从而带来噪声。为此,笔者使用语言模型过滤这些噪声,达到净化训练数据,提高系统性能的目的。笔者在Senseval-3国际评测中文采样词词义消歧数据集上进行了实验,结果表明经过语言模型过滤的词义消歧系统性能明显高于未经过滤的系统。

Word Sense Disambiguation Corpus Acquisition by Language Model Validation

The lack of hand-crafted training data is a critical issue for supervised word sense disambiguation(WSD) systems.The monosemous lexical relatives substitution of target words have been proposed to acquire WSD corpus from the Web automatically.However,in some cases,the monosemous lexical relatives cannot be substituted by the target word suitably and then noises will be brought in.We propose a language models validation method to filter these noises,which can purify the training data,and improve the performance accordingly.Our experiments on Senseval-3 Chinese lexical sample task show that the system based on the training data acquired from the Web with language model validation achieves better accuracy than the one without language models validation.