自然语言处理中词语上下文有效范围的定量描述

词语的上下文（context）是语料库语言学中自然语言知识获取和解决自然语言处理中多种实际应用问题必须依靠的资源和基础，但上下文“窗口”开多大为宜呢？为克服当前仅凭主观经验或通过某一特定应用问题中最终结果正确率界定上下文有效范围的不足，我们在引入信息增益方法确定上下文各位置的信息量后，构造上下文位置信息量函数，最终通过多项式积分确定85％信息量的上下文边界，即汉语核心词语最近距离－8，＋9]和英文－16，＋13]位置之间的上下文范围，该文的结果对上下文在自然语言处理中的价值和作用提出了一个具有统计意义的量化解释。

Quantitative Analysis of Context Field in Natural Language Processing

Context is the necessary resource not only for corpus based linguistic but also for solving the problem in computational linguistics. But what is the size of context window? Because there is no method to define the effective field of context window by quantitative analysis, this paper put forward one computational method to do it with quantitative analysis. First, context position is weighted by information gain; second, the position weight function is constructed according to weight of context positions, and then the function is integraled to arrive to the information ratio 85% to define the size of windows. The result is -8,+9] in Chinese and -16,+13] in English for context window. The result explained quantitatively the value and function of context in natural language processing.