基于最大熵的隐马尔可夫模型文本信息抽取

文本信息抽取是处理海量文本的重要手段之一.最大熵模型提供了一种自然语言处理的方法.提出了一种基于最大熵的隐马尔可夫模型文本信息抽取算法.该算法结合最大熵模型在处理规则知识上的优势,以及隐马尔可夫模型在序列处理和统计学习上的技术基础,将每个观察文本单元所有特征的加权之和用来调整隐马尔可夫模型中的转移概率参数,实现文本信息抽取.实验结果表明,新的算法在精确度和召回率指标上比简单隐马尔可夫模型具有更好的性能.

Using Hidden Markov Model for Text Information Extraction Based on Maximum Entropy

Text information extraction is an important approach to processing large quantity of text.Maximum entropy provides a kind of framework for natural language processing.A new algorithm using hidden Markov model based on maximal entropy is proposed for text information extraction.The new algorithm combines the advantage of maximum entropy model,which can integrate and process rules and knowledge efficiently,with that of hidden Markov model,which has powerful technique foundations to solve sequence representation and statistical problem,and uses the sum of all features with weights to adjust the transition parameters in hidden Markov model for text information extraction.Experimental results show that the new algorithm improves the performance in precision and recall.