INFORMATION RETRIEVAL FOR SHORT DOCUMENTS

I. Introduction Most of current Information Retrieval (IR) sys-tems try to match terms of queries with terms of documents. One major problem of these approaches lies in that users want to retrieve documents accord-ing to content, while individual words provide unre-liable evidence about the content of the texts1?3]. When some parts of text in the document collection are missing, e.g. only the abstract is available, the word-use variability problem will have substantial impact on the IR per…

Information Retrieval for short documents

The major problem of the most current approaches of information models lies in that individual words provide unreliable evidence about the content of the texts. When the document is short, e.g. only the ab-stract is available, the word-use variability problem will have substantial impact on the Information Retrieval (IR) performance. To solve the problem, a new technology to short document retrieval named Reference Document Model (RDM) is put forward in this letter. RDM gets the statistical semantic of the query/document by pseudo feedback both for the query and document from reference documents. The contributions of this model are three-fold: (1) Pseudo feedback both for the query and the document; (2) Building the query model and the document model from reference documents; (3) Flexible indexing units, which can be any linguistic elements such as documents, paragraphs, sentences, n-grams, term or character. For short document retrieval, RDM achieves significant improvements over the classical probabilistic models on the task of ad hoc retrieval on Text REtrieval Conference (TREC) test sets. Results also show that the shorter the document, the better the RDM performance.