一种实用的资源稀缺条件下的分词方法

在一些使用人数较少的语言中，缺少人工标注语料，研究在资源稀缺条件下的分词方法成了亟待解决的问题。本文研究了无监督的VE算法，以及最大匹配间隔标注算法，在此基础之上提出一种无监督分词方法与最大匹配方法相结合的分词方法，并在汉语语料上进行实验。该分词方法显著提高了最大匹配分词方法的分词效果，同时也优于当前最好的无监督分词结果。实验表明，该方法快速、有效，利用较小的资源获得了较好的分词效果。

A Practical Approach of Word Segmentation in the Poor Resource Situation

There is lack of human-annotated corpus in some language, which not a lot of people used, it is a burning question for word segmentation in the situation of poor resource. This article studied the unsupervised VE algorithm, and maximum match gap tagging, and proposed a method, which combined the unsupervised word segmentation method, and the maximum matching method, and implemented in the Chinese datasets. The method significantly increased the performance of maximum matching word segmentation method, and gives a better performance than currently state of the art unsupervised word segmentation method. The experimental results show that our method is fast and effective and have a better performance with a smaller datasets resource.