融合上下文字符信息的泰语神经网络分词方法

自动分词是自然语言处理的关键基础技术。针对传统泰语统计分词方法特征模板复杂、搜索空间大的问题，提出融合上下文字符信息的泰语神经网络分词模型。该模型借助词分布表示方法，训练泰语字符表示向量，利用多层神经网络分类器实现泰语分词。基于InterBEST 2009泰语分词评测语料的实验结果表明，所提方法相较于条件随机场分词模型、Character-Cluster Hybrid 分词模型以及 GLR and N-gram 分词模型取得了更好的分词效果，分词准确率、召回率和F值分别达到了97.27%、99.26 %及98.26 %，相比条件随机场分词速度提高了112.78%。

A context character feature based neural network model for Thai word segmentation

Automatic word segmentation is a fundamental technology of natural language processing. Aiming at the problem of complex feature template and large search space in the traditional Thai word segmentation method, this paper proposes a context character feature based neural network model for Thai word segmentation. The proposed model uses the word distribution table to train the word representation vector, and utilizes a multi-layer neural network classifier for Thai word segmentation. Experimental results on InterBEST 2009 Thai word evaluation corpus show that, compared with the conditional random field model, the Character-Cluster Hybrid segmentation model, and the GLR and N-gram segmentation model, our proposal achieves better performance. Word segmentation accuracy, recall ratio and F value reach 97.27%, 99.26% and 98.26%, respectively. Our model improves the segmentation speed by 112.78% in comparison to the conditional random field model.