基于主题语言模型的中文信息检索系统研究

准确的文档语言模型估计对于改善语言模型检索系统的性能是非常重要的。在本文中我们提出了基于主题语言模型的信息检索系统,首先设计了“改进的两阶段K2Means 聚类算法”来对文档集合进行聚类,通过引入Aspect Model 结合聚类结果可以得到基于主题的语言模型。这个新的语言模型较深入地刻画了词汇在不同主题下的分布规律以及文档所蕴含不同主题的分布规律。将主题语言模型和文档本身的语言模型通过线性插值可以更准确地估计文档语言模型。实验结果表明我们提出的这个方法显著改善了检索系统的性能,与Jelinek2Mercer 模型方法相比较,主题语言模型检索系统的平均精度提高大约16117 % ,召回率提高大约 9164 %。

A Topic Language Model-based Chinese IR System

Exact estimation of the document language model is important to the performance of the language model based IR system. In this paper we proposed a topic-based approach to language modeling for ad-hoc Information Retrieval. An improved two-stage k-means clustering method is designed to deal with the document collection and the clustered results are regarded as the topic information contained in the collection. Through combing the aspect model and text clustering technology, we can derive a more accurate document language model for ad-hoc Information Retrieval. Experiments have shown that the performance of IR system has been improved greatly. Compared with Jelinek-Mercer language model IR system, Precision of the trigger language model based IR system increased almost 16.17% and recall of the system increased 9.64%.