古汉语双字词自动获取方法的比较与分析

词汇的自动获取在自然语言生成、计算词典编纂、句法分析以及语料库语言学等领域均有着重要的研究价值。该文针对古汉语双字词的自动获取问题,以《史记》全文语料为例,分别应用基于频率、互信息、假设检验的统计方法获取古汉语双字词,并结合人工标注结果进行了详细的比较和分析,评价了各方法的优缺点及可靠性,为不同应用背景下的古汉语双字词自动获取提供了相应的解决方案。

A Comparative Study on the Automatic Extraction of Two-character Word from Ancient Chinese

Word extraction is of great importance in the research fields of natural language generation, computational lexicography, parsing, corpus linguistic, etc. To address the issue of automatic extraction of two-character word from ancient Chinese, this paper takes the “Records of the Grand Historian” corpus as an example, and uses the statistical methods that based on frequency, mutual information and hypothesis testing to extract two-character word, respectively. Then it compares and analyzes the results according to the manual marked result in detail. It paves the way for the scheme design for the two-character word extraction from ancient Chinese in different applications.
Key wordsChinese information processing; Ancient Chinese; Records of the Grand Historian; two-character word; statistical model