基于概念的文本过滤模型

当前,文本过滤技术基本上停留在关键词阶段,无法处理同义和概念之间的上下位关系,因此,准确率和召回率达到一定值后,无论如何改进算法也无法再取得突破进展。文章试图从语义的角度突破这个困境,对常识知识库HowNet在文本过滤中的应用,以及文本过滤中所涉及的关键技术,包括基于概念的文本表示方法、用户模板表示方法、文本过滤算法进行了研究和探讨。实验结果表明,基于概念的方法的确可以提高文本过滤的性能。

A Text Filtering Model Based on Concept

Nowadays,text filtering techniques basically suspend at the indexing stage,which is typical of ,even merely words,failing to deal with the up-down relationship between synonymies and concepts.Consequently,when precision and recall reach certain value,breakthrough can't be made no matter how to improve retrieve algorithm.This paper attempts to break through this predicament in terms of semanteme,to study the application of HowNet,which is a common sense knowledge database,in text filtering,to discuss key techniques.Experimental results show that the methods can improve the text filtering performance.