基于WWW的未登录词识别研究

一、引言当前,随着国民经济信息化的不断发展以及Internet的普及应用,全世界丰富的信息资源展现在我们每个人面前。如何从大量的信息中迅速有效地提取出所需信息极大地影响着我国计算机技术和信息技术的发展和应用推广。据统计,在信息领域中,80％以上的信息是以语言文字为载体的,因此,中文信息处理技术成为我国重要的计算机应用技术。未登录词的识别是中文信息处理技术中的难点之一。它在Internet数据挖掘、信息检索、图书馆图书文献管理、语音识别等应用中

WWW-based Recognition of Non-login Words

Currently, very little reference material can be found on the research of non-login word recognition. Solu-tions based on rules and syntaxes can't satisfactorily solve all kinds of problems of non-login word recognition. Thispaper will study and compare several existing solutions. The proposed solution is to extract N-grams after words sep-aration, from which non-login words can be extracted by means of probability statistics. Experiments have demon-strated that this method has favorable efficiency, recall ratio, and accuracy.