一种中文文档的非受限无词典抽词方法

本文提出了一种非受限无词典抽词模型,该模型通过自增长算法获取中文文档中的汉字结合模式,并引入支持度、置信度等概念来筛选词条。实验表明:在无需词典支持和利用语料库学习的前提下,该算法能够快速、准确地抽取中文文档中的中、高频词条。适于对词条频度敏感,而又对计算速度要求很高的中文信息处理应用,例如实时文档自动分类系统。

A Domain-independent Dictionary-free Lexical Acquisition Model For Chinese Document

A domain independent dictionary free lexical acquisition model is presented in this paper,which introduces a self increasing algorithm to acquire the co occurrence patterns of Chinese characters,and introduces some criteria such as support and confidence to filter these co occurrence patterns to get lexical items.Experiments show that it can acquire lexical items with high frequency effectively and efficiently without the support of the dictionary and the supervised learning in term of corpus.The model proposed in this paper particularly suits for lexical frequency sensitive but time critical Chinese information processing applications,such as real time automatic Chinese text classification systems.