| 基于词内部模式的新词识别 |
| |
| 引用本文： | 林自芳,蒋秀凤. 基于词内部模式的新词识别[J]. 计算机与现代化, 2010, 0(11): 162-164,167. DOI: 10.3969/j.issn.1006-2475.2010.11.046 |
| |
| 作者姓名： | 林自芳 蒋秀凤 |
| |
| 作者单位： | 福州大学数学与计算机科学学院,福建福州350108 |
| |
| 基金项目： | 福建省科技创新平台计划项目，福建省教育厅科研基金资助项目，福建省发展改革委员会基金资助项目 |
| |
| 摘 要： | 提出一种基于词内部模式的新词识别算法,该算法在重复串查找的基础上,结合词内部模式的特征提出改进位置成词概率和首尾单字成词概率的加权,依次判断互信息、邻接类别等统计量,对新词进行识别。通过不同的实验对比发现,该算法在一定程度上能有效提取新词。
|
| 关 键 词： | 词内部模式 新词语识别 改进位置成词概率 首尾单字成词概率 |
A New Method for Chinese New Word Identification Based on Inner Pattern of Word |
| |
| LIN Zi-fang,JIANG Xiu-feng. A New Method for Chinese New Word Identification Based on Inner Pattern of Word[J]. Computer and Modernization, 2010, 0(11): 162-164,167. DOI: 10.3969/j.issn.1006-2475.2010.11.046 |
| |
| Authors: | LIN Zi-fang JIANG Xiu-feng |
| |
| Affiliation: | (College of Mathematics and Computer Science,Fuzhou University,Fuzhou 350108,China) |
| |
| Abstract: | As to new word identification problem,this paper proposes a new method for Chinese new word identification based on the inner pattern of word.After repeat finding based on suffix arrays and longest common preffix,it propses the weighting of the improved PWP and inside word probabilities in view of the inner pattern of word.At the meanwhile,the paper uses AV and MI statistics to identify Chinese new words.By comparison,find that this method is effective in recognition of Chinese new words. |
| |
| Keywords: | inner pattern of word new word identification improved PWP inside word probabilities |
| 本文献已被 维普 万方数据 等数据库收录！ |
