面向机器辅助翻译的汉语语块自动抽取研究

本文提出了一种统计和规则相结合的语块抽取方法。本文使用Nagao串频统计算法进行基于词语的串频统计,进一步分别利用统计方法、语块边界过滤规则对2-gram到10-gram语块进行过滤,得到候选语块,取得了令人满意的结果。通过实验发现,在统计方法中互信息和信息熵相结合的方法较单一的互信息方法好;在语块边界规则过滤方法中语块左右边界规则和停用词对语块抽取的结果有较大影响。实验结果表明统计和过滤规则相结合的方法要优于纯粹的统计方法。应用本文方法,再辅以人工校对,可以方便地获取重复出现的多词语块。在机器辅助翻译系统中,使用现有的语块抽取方法抽取重复的语言单位,就可以方便地建设翻译记忆库,提高翻译的工作效率。

Chinese Multi-word Chunks Extraction for Computer Aided Translation

This paper suggests a methodology which is aimed to extract multi word chunks for translation purposes.Our basic idea is to use a hybrid method which combines the statistical method and linguistic rules.The extraction system used in our work operated at four steps:(1) Tokenization of Chinese corpus;(2) Extraction of multi-word chunks(2-gram to 10-gram) using Nagao's Algorithm and Substring Reduction Algorithm;(3)Statistical Filtering which combines Mutual Information(or Log-likelihood Ratio) and Left/Right Entropy;(4) Linguistic filtering by chunk formation rules and stop-word list.As a result,hybrid method proved to be a suitable method for selecting multi-word chunks,it has considerably improved the precision of the extraction which is much higher than that of purely statistical method.We believe that multi-word chunks extracted in this way could be used effectively to supplement existing translation memory database.