语义词特征提取及其在维吾尔文文本分类中的应用

基于机器学习的文本分类中,维吾尔文传统分词方法表现出非常明显的不足和局限性。该文使用另外一种维吾尔文自动分词方法dme-TS。dme-TS中,不再以词间空格作为切分标记提取词特征,而是用一种组合统计量(dme)来度量文本中相邻单词之间的关联程度,并以dme度量的弱关联的词间位置作为切分点,提取对学习算法真正有意义的语义词特征。实验结果表明,用dme-TS提取文本特征可以降低特征空间的维度,同时也能有效的提高传统以单词为特征的分类算法的性能。

Semantics-based Feature Extraction and Its Application in Uyghur Text Classification

In the text classification based on machine learning, the Uyghur traditional segmentation shows its deficiencies and limitations obviously. This paper uses another Uyghur automatic word segmentation method named as dme-TS. This segmentation method, no longer uses inter-word space as natural delimiter, but uses a kind of combination statistics (dme) to estimate the agglutinative strength between two adjacent Uyghur words, with the weak dme position as a segmentation point, The experimental result shows that, dme-TS can reduce the dimension of the feature space, at the same time also can effectively improve the classification performance of the tradition algorithm with the word for the features.