运用有向图进行中文分词研究

首先说明了分词在中文信息处理中的作用,然后介绍了分词系统中的关键技术。提出了一种基于有向图的中文分词算法,该算法首先构造中文分词有向图,然后计算中文分词有向图中所有可能的切分路径,最后利用了最少分词原则、汉字之间的互信息和词语的频率等信息给中文分词有向图中的每条切分路径打分,分数最高的路径就对应正确的切分结果。开放测试结果表明分词精确率可达90%以上。

Method of Chinese word segmentation based on directed graph

Chinese word segmentation is the first step for any Chinese information processing and hinders seriously its development.This paper introduces the critical technologies in the segmentation systems.It proposes a refinement of the segmentation algorithm based on the directed graph,this algorithm first constructs the Chinese segmentation directed graph,and calculates the weight of every segmentation path,last evaluates every segmentation path based on the principle of least segmentation,the mutual info of char...