| 基于CRF的先秦汉语分词标注一体化研究 |
| |
| 引用本文： | 石民,李斌,陈小荷.基于CRF的先秦汉语分词标注一体化研究[J].中文信息学报,2010,24(2):39-46. |
| |
| 作者姓名： | 石民 李斌 陈小荷 |
| |
| 作者单位： | 南京师范大学 文学院,江苏 南京,210097 |
| |
| 基金项目： | 国家“211工程”三期重点学科建设项目“语言科技创新及工作平台建设”子课题“先秦文献词汇统计与知识检索系统” |
| |
| 摘 要： | 该文探索了古代汉语,特别是先秦文献的词切分及词性标注。首先对《左传》文本进行了词汇处理(分词和词性标注)和考察分析,然后采用条件随机场模型(CRF),进行自动分词、词性标注、分词标注一体化的对比实验。结果表明,一体化分词比单独分词的准确率和召回率均有明显提高,开放测试的F值达到了94.60%;一体化词性标注的F值达到了89.65%,比传统的先分词后标注的“两步走”方法有明显提高。该项研究可以服务于古代汉语词汇研究和语料库建设,以弥补人工标注的不足。
|
| 关 键 词： | 计算机应用 中文信息处理 先秦汉语 分词 词性标注 左传 条件随机场模型 |
CRF Based Research on a Unified Approach to Word Segmentation and POS Tagging for Pre-Qin Chinese |
| |
| SHI Min,LI Bin,CHEN Xiaohe.CRF Based Research on a Unified Approach to Word Segmentation and POS Tagging for Pre-Qin Chinese[J].Journal of Chinese Information Processing,2010,24(2):39-46. |
| |
| Authors: | SHI Min LI Bin CHEN Xiaohe |
| |
| Affiliation: | School of Chinese language and literature, Nanjing Normal University, Nanjing, Jiangsu 210097, China |
| |
| Abstract: | This paper explores the cross field between NLP and ancient Chinese,particularly the pre-Qin documents.The text of Zuo Zhuan is firstly analyzed after manual segmentationand POS tagging.Then the Conditional Random Fields model(CRF) is adopted for the word segmentation(WS),POS tagging(PT) and a unified process of WS and PT,respectively.The precision and recall of the unified approach are much higher than the independent WS and PT in the open test,with a F-score of 94.60% in WS and 89.65% in PT.This method is... |
| |
| Keywords: | computer application Chinese information processing Pre-Qin Chinese word segmentation POS tagging Zuo Zhuan conditional random fields model |
| 本文献已被 CNKI 万方数据 等数据库收录！ |
| 点击此处可从《中文信息学报》浏览原始摘要信息 |
|
点击此处可从《中文信息学报》下载全文 |
|
