汉语词语边界自动划分的模型与算法

在引入词形和汉字结合点等概念基础上，文中分别描述了一个基于字串构词能力的词形模型和一个基于词语内部、外部汉字结合度的汉字结合点模型，并采用线性插值方法 两种模型融合于一体进行汉词语边界划分。在分析汉语切分候选择空间的基础上，文中还给出了相应的优化搜索算法。与一般的统计方法相比，文中方法的参数可直接从未经加工粗语料中得到，具有较强的适应能力，初步试验表明该方法是有效和可靠的。

MODELS AND ALGORITHM FOR ASSIGNING WORD BREAKS TO CHINESE TEXT

In this paper, the word form model (WFM) based on word formation power of Chinese character string and the character juncture model (CJM) based on the affinity of the Chinese character pairs inside or outside words are described respectively. Then a linear interpolation method is applied to combine these two models together to assign word breaks to Chinese text. The relative searching algorithm is also given after the searching space is analyzed. Compared with general statistic models, the parameters of the models proposed can be directly trained from raw corpus, which results in a strong adaptability. The approach has proven both reliable and efficient by experiments.