藏语音节标注研究

藏语的“音节”在词汇语法研究和文本信息处理研究中都十分重要,尤其在解决未登录词切分问题和标注中能够发挥积极的作用。然而在现有的研究中,对音节的重视还不够。该文提出在文本标注时,可以先进行音节的性质标注,然后通过音节构词的规律预测复合词的词性,尤其是未登录词的词性。该文作者对藏语音节的定义进行了界定,提出音节的性质分类及标注原则,利用统计模型,在约24万音节的中小学语文教材语料库上进行实验,音节性质标注的正确率为93.520 8%。在此基础上,把音节性质标注信息用到词性标注中。实验结果表明: 即使在音节性质标注存在一定错误的情况下,词性标注的正确率也提高到94.196 7%;如果在保证音节性质标注完全正确的情况下,词性标注的正确率可以提高到97.775 4%,这说明音节性质标注信息对词性标注有帮助。

Research on Tagging of Tibetan Syllables

“Syllables” of Tibetan language are very important in vocabulary construction and text information processing, especially for solving the segmentation and annotation of OOVs. This paper proposes to tag the syllables, which can be applied to predict POS of compound words (especially OOVs) according to the rules of words-construction. This paper presents the definition of the Tibetan syllable, outlines and the principles of classification and labeling. The train and test texts are selected from teaching material of Tibetan language of primary and secondary schools, total 240K syllables. Experiments reveals a precision of 93.5208% for syllable tagging, upon which an improved 94.1967% accuracy for POS tagging can be reached. And given the gold-standard of syllable tagging, the accuracy of POS tagging will be improved to 97.775 4%.