基于条件随机场的蒙古语词切分研究

词干和构形附加成分是蒙古语词的组成成分,在构形附加成分中包含着数、格、体、时等大量语法信息。利用这些语法信息有助于使用计算机对蒙古语进行有效处理。蒙古语词在结构上表现为一个整体,为了利用其中的语法信息需要识别出词干和各构形附加成分。通过分析蒙古语词的构形特点,提出一种有效的蒙古语词标注方法,并基于条件随机场模型构建了一个实用的蒙古语词切分系统。实验表明该系统的词切分准确率比现有蒙古语词切分系统的准确率有较大提高,达到了0.992。

Research on Conditional Random Fields Based Mongolian Word Segmentation

Etyma and morphological affix are the components of Mongolian words, which include lots of grammar information. Using this grammar information is helpful for effective processing Mongolian language. Mongolian words exhibit as a whole in the structure, and therefore, the detection of etyma and each morphological affix is necessary to capture this grammar information. By analyzing the characteristics of morphological construction of Mongolian words, this paper proposes an effective Mongolian word labeling method, and constructs a practical Mongolian word segmentation system based on conditional random fields model. Experiments show that the accuracy of segmentation has a significant improvement than current system, reaching an accuracy rate of 0.992.
Key wordsMongolian; word segmentation; etyma; morphological affix; conditional random fields; statistical language model