基于介词向量的英语真词错误检查算法

在基于Winnow算法的基础上引入混淆词和介词搭配的方法.首先通过混淆集获得训练集,对训练集进行预处理后利用文本特征提取方法获得特征词集,然后对特征词集进行Winnow训练得到带有权重的特征词集并把出现在混淆词后的介词提取出来生成介词向量,最后从测试集提取特征并进行结合Winnow算法和混淆词与介词搭配方法的测试得到真词错误检查的结果.混淆词与介词搭配方法的加入使得某些混淆词的正确率、召回率以及F1测度提高了10%~20%,有的甚至提高到了100%.

English Real-Word Errors Checking Algorithm Based on Preposition Vector

This paper introduces the method of collocation of confusion words and prepositions based on Winnow algorithm. Firstly, we obtain training sets by confusion sets. After preprocessing the training sets, we use the text feature extracted method to obtain feature sets. Secondly, we get the feature sets with weights by training on the feature sets based on Winnow and extract the prepositions which appear after the confusion words to generate the preposition vectors. Finally, we extract features from the test sets and the test sets and get the real-word errors checking results by the test which combines Winnow algorithm and the method of collocation of confusion words and prepositions. The correct rate, recall rate and F1 measure of some confusion words are improved by 10% ~ 20% when we join the method of collocation of confusion words and prepositions, some even up to 100%.