RUBRIC: A System for Rule-Based Information Retrieval

A research prototype software system for conceptual information retrieval has been developed. The goal of the system, called RUBRIC, is to provide more automated and relevant access to unformatted textual databases. The approach is to use production rules from artificial intelligence to define a hierarchy of retrieval subtopics, with fuzzy context expressions and specific word phrases at the bottom. RUBRIC allows the definition of detailed queries starting at a conceptual level, partial matching of a query and a document, selection of only the highest ranked documents for presentation to the user, and detailed explanation of how and why a particular document was selected. Initial experiments indicate that a RUBRIC rule set better matches human retrieval judgment than a standard Boolean keyword expression, given equal amounts of effort in defining each. The techniques presented may be useful in stand-alone retrieval systems, front-ends to existing information retrieval systems, or real-time document filtering and routing.     

基于模糊语言方法的信息检索系统的研究

该文提出了一个基于模糊语言方法的信息检索系统模型。该系统分为查询界面子系统、数据库子系统和检索子系统三大部分。在查询界面子系统,用布尔表达式表示用户的查询请求,并对每个查询关键词赋予了两种不同语义的语言值权重,该权重表达了用户的模糊检索要求;在数据库子系统,用索引词一文档模糊矩阵表示待检索的文档,对每个索引词。根据其在文档中的出现频率大小。引入了数值权重;在检索子系统,运用模糊语言方法,对用户输入的布尔查询表达式与索引词一文档模糊矩阵进行自底向上的模糊匹配,最后返回满足用户要求的检索结果。相对于传统的基于查询关键词精确匹配的检索系统而言,该系统能较好地满足用户查询要求中的灵活性。     

基于查询扩展和数据融合的检索过程优化

介绍典型的检索过程优化方法——数据融合和基于相关度反馈的查询扩展,前者通过集成多个检索结果提高检索性能,后者执行多次查询,依据前次结果修改／扩展用户查询,以求更好地反映用户信息需求,并在此基础上提出一种新的检索过程优化方法——HQD方法,由相关度反馈结果生成多个替代查询,在检索这些替代查询后,采用求和余弦法生成最终检索结果。仿真实验结果表明,该方法是有效的。     

一种基于排序学习方法的查询扩展技术

查询扩展作为一门重要的信息检索技术,是以用户查询为基础,通过一定策略在原始查询中加入一些相关的扩展词,从而使得查询能够更加准确地描述用户信息需求。排序学习方法利用机器学习的知识构造排序模型对数据进行排序,是当前机器学习与信息检索交叉领域的研究热点。该文尝试利用伪相关反馈技术,在查询扩展中引入排序学习算法,从文档集合中提取与扩展词相关的特征,训练针对于扩展词的排序模型,并利用排序模型对新查询的扩展词集合进行重新排序,将排序后的扩展词根据排序得分赋予相应的权重,加入到原始查询中进行二次检索,从而提高信息检索的准确率。在TREC数据集合上的实验结果表明,引入排序学习算法有助于提高伪相关反馈的检索性能。     

基于检索结果排序的伪相关反馈

针对传统伪相关反馈（PRF）算法扩展源质量不高使得检索效果不佳的问题,提出一种基于检索结果的排序模型（REM）。首先,该模型从初检结果中选择排名靠前的文档作为伪相关文档集;然后,以用户查询意图与伪相关文档集中各文档的相关度最大化、并且各文档之间相似性最小化作为排序原则,将伪相关文档集中各文档进行重排序;最后,将排序后排名靠前的文档作为扩展源进行二次反馈。实验结果表明,与两种传统伪反馈方法相比,该排序模型能获得与用户查询意图相关的反馈文档,可有效地提高检索效果。     

Semisupervised Query Expansion with Minimal Feedback

Query expansion is an information retrieval technique in which new query terms are selected to improve search performance. Although useful terms can be extracted from documents whose relevance is already known, it is difficult to get enough of such feedback from a user in actual use. We propose a query expansion method that performs well even if a user makes practically minimum effort, that is, chooses only a single relevant document. To improve searches in these conditions, we made two refinements to a well-known query expansion method. One uses transductive learning to obtain pseudorelevant documents, thereby increasing the total number of source documents from which expansion terms can be extracted. The other is a modified parameter estimation method that aggregates the predictions of multiple learning trials to sort candidate terms for expansion by importance. Experimental results show that our method outperforms traditional methods and is comparable to a state-of-the-art method.     

Enhancing passage retrieval in log files by query expansion based on explicit and pseudo relevance feedback

Passage retrieval is usually defined as the task of searching for passages which may contain the answer for a given query. While these approaches are very efficient when dealing with texts, applied to log files (i.e. semi-structured data containing both numerical and symbolic information) they usually provide irrelevant or useless results. Nevertheless one appealing way for improving the results could be to consider query expansions that aim at adding automatically or semi-automatically additional information in the query to improve the reliability and accuracy of the returned results. In this paper, we present a new approach for enhancing the relevancy of queries during a passage retrieval in log files. It is based on two relevance feedback steps. In the first one, we determine the explicit relevance feedback by identifying the context of the requested information within a learning process. The second step is a new kind of pseudo relevance feedback. Based on a novel term weighting measure it aims at assigning a weight to terms according to their relatedness to queries. This measure, called TRQ (Term Relatedness to Query), is used to identify the most relevant expansion terms.The main advantage of our approach is that is can be applied both on log files and documents from general domains. Experiments conducted on real data from logs and documents show that our query expansion protocol enables retrieval of relevant passages.     

Relevance feedback revisited: dealing with content and structure in XML documents

Relevance feedback (RF) is a technique that allows to enrich an initial query according to the user feedback. The goal is to express more precisely the user’s needs. Some open issues arise when considering semi-structured documents like XML documents. They are mainly related to the form of XML documents which mix content and structure information and to the new granularity of information. Indeed, the main objective of XML retrieval is to select relevant elements in XML documents instead of whole documents. Most of the RF approaches proposed in XML retrieval are simple adaptation of traditional RF to the new granularity of information. They usually enrich queries by adding terms extracted from relevant elements instead of terms extracted from whole documents. In this article, we describe a new approach of RF that takes advantage of two sources of evidence: the content and the structure. We propose to use the query term proximity to select terms to be added to the initial query and to use generic structures to express structural constraints. Both sources of evidence are used in different combined forms. Experiments were carried out within the INEX evaluation campaign and results show the effectiveness of our approaches.     

Conceptual clustering in information retrieval

Clustering is used in information retrieval systems to enhance the efficiency and effectiveness of the retrieval process. Clustering is achieved by partitioning the documents in a collection into classes such that documents that are associated with each other are assigned to the same cluster. This association is generally determined by examining the index term representation of documents or by capturing user feedback on queries on the system. In cluster-oriented systems, the retrieval process can be enhanced by employing characterization of clusters. In this paper, we present the techniques to develop clusters and cluster characterizations by employing user viewpoint. The user viewpoint is elicited through a structured interview based on a knowledge acquisition technique, namely personal construct theory. It is demonstrated that the application of personal construct theory results in a cluster representation that can be used during query as well as to assign new documents to the appropriate clusters.     

Examining topic shifts in content-oriented XML retrieval

Content-oriented XML retrieval systems support access to XML repositories by retrieving, in response to user queries, XML document components (XML elements) instead of whole documents. The retrieved XML elements should not only contain information relevant to the query, but also provide the right level of granularity. In INEX, the INitiative for the Evaluation of XML retrieval, a relevant element is defined to be at the right level of granularity if it is exhaustive and specific to the query. Specificity was specifically introduced to capture how focused an element is on the query (i.e., discusses no other irrelevant topics). To score XML elements according to how exhaustive and specific they are given a query, the content and logical structure of XML documents have been widely used. One source of evidence that has led to promising results with respect to retrieval effectiveness is element length. This work aims at examining a new source of evidence deriving from the semantic decomposition of XML documents. We consider that XML documents can be semantically decomposed through the application of a topic segmentation algorithm. Using the semantic decomposition and the logical structure of XML documents, we propose a new source of evidence, the number of topic shifts in an element, to reflect its relevance and more particularly its specificity. This paper has three research objectives. Firstly, we investigate the characteristics of XML elements reflected by their number of topic shifts. Secondly, we compare topic shifts to element length, by incorporating each of them as a feature in a retrieval setting and examining their effects in estimating the relevance of XML elements given a query. Finally, we use the number of topic shifts as evidence for capturing specificity to provide a focused access to XML repositories.     

信息检索中的相关反馈技术综述*

论述了信息检索中的向量空间模型、概率模型以及语言模型中所采用的相关反馈技术。其中主要介绍检索词的权重调整、查询扩展、文档相关反馈,以及语言模型中的查询语言模型和文档语言模型的调整。针对最近反馈方面的最新成果——基于term的反馈技术进行了探讨,指出了相关反馈在今后研究的方向,即提供个性化的如分层反馈和利用日志进行反馈,并讨论了相关反馈技术对检索性能的影响。     

A new method for fuzzy information retrieval based on fuzzy hierarchical clustering and fuzzy inference techniques

In this paper, we extend the work of Kraft et al. to present a new method for fuzzy information retrieval based on fuzzy hierarchical clustering and fuzzy inference techniques. First, we present a fuzzy agglomerative hierarchical clustering algorithm for clustering documents and to get the document cluster centers of document clusters. Then, we present a method to construct fuzzy logic rules based on the document clusters and their document cluster centers. Finally, we apply the constructed fuzzy logic rules to modify the user's query for query expansion and to guide the information retrieval system to retrieve documents relevant to the user's request. The fuzzy logic rules can represent three kinds of fuzzy relationships (i.e., fuzzy positive association relationship, fuzzy specialization relationship and fuzzy generalization relationship) between index terms. The proposed fuzzy information retrieval method is more flexible and more intelligent than the existing methods due to the fact that it can expand users' queries for fuzzy information retrieval in a more effective manner.     

基于数据融合和相关度反馈的信息检索方法

数据融合和基于相关度反馈的查询扩展是两种有效的检索过程优化技术。前者通过集成多个检索结果提高检索性能,后者执行多次查询,依据前次结果修改/扩展用户查询,以求更好地反映用户信息需求。在混合数据融合和查询扩展技术的基础上提出一种检索过程优化方法——HQD方法,由相关度反馈结果生成多个替代查询,检索这些替代查询后采用求和余弦方法生成最终检索结果。HQD方法能有效提高检索性能。     

一种图像检索中的灰色相关反馈算法

在交互式CBIR系统中，由于用户的查询需求常常是模糊的，因此检索结果从某种意义上说是不确定的。于是，可以将图像检索过程视为一个“灰色系统”，其中的查询向量以及图像特征的权重可视为“灰数”。基于此，该文提出了一种新的相关反馈技术，它采用“灰关联分析”理论来分析和描述“例子图像”与“相关图像”之间的关系，据此自动更新查询向量与图像特征的权重，从而更准确地描述用户的查询需求。实验结果表明，这种相关反馈算法能较好地描述用户的查询需求，显著地改善了图像检索的性能。     

Boolean query mapping across heterogeneous information sources

Searching over heterogeneous information sources is difficult because of the nonuniform query languages. Our approach is to allow a user to compose Boolean queries in one rich front end language. For each user query and target source, we transform the user query into a subsuming query that can be supported by the source but that may return extra documents. The results are then processed by a filter query to yield the correct final result. We introduce the architecture and associated algorithms for generating the supported subsuming queries and filters. We show that generated subsuming queries return a minimal number of documents; we also discuss how minimal cost filters can be obtained. We have implemented prototype versions of these algorithms and demonstrated them on heterogeneous Boolean systems     

Large-scale information retrieval with latent semantic indexing

As the amount of electronic information increases, traditional lexical (or Boolean) information retrieval techniques will become less useful. Large, heterogeneous collections will be difficult to search since the sheer volume of unranked documents returned in response to a query will overwhelm the user. Vector-space approaches to information retrieval, on the other hand, allow the user to search for concepts rather than specific words, and rank the results of the search according to their relative similarity to the query. One vector-space approach, Latent Semantic Indexing (LSI), has achieved up to 30% better retrieval performance than lexical searching techniques by employing a reduced-rank model of the term-document space. However, the original implementation of LSI lacked the execution efficiency required to make LSI useful for large data sets. A new implementation of LSI, LSI++, seeks to make LSI efficient, extensible, portable, and maintainable. The LSI++ Application Programming Interface (API) allows applications to immediately use LSI without knowing the implementation details of the underlying system. LSI++ supports both serial and distributed searching of large data sets, providing the same programming interface regardless of the implementation actually executing. In addition, a World Wide Web interface was created to allow simple, intuitive searching of document collections using LSI++. Timing results indicate that the serial implementation of LSI++ searches up to six times faster than the original implementation of LSI, while the parallel implementation searches nearly 180 times faster on large document collections.     

A framework for a feedback process to analyze and personalize a document vector space in a feature extraction model

In this paper, we present a framework for a feedback process to implement a highly accurate document retrieval system. In the system, a document vector space is created dynamically to implement retrieval processing. The retrieval accuracy of the system depends on the vector space. When the vector space is created based on a specific purpose and interest of a user, highly accurate retrieval results can be obtained. In this paper, we present a method for analyzing and personalizing the vector space according to the purposes and interests of users. In order to optimize the document vector space, we defined and implemented functions for the operations of adding, deleting and weighting the terms that were used to create the vector space. By exploiting effectively and dynamically the classified-document information related to the queries, our methods allow users to retrieve relevant documents for their interests and purposes. Even if the search results of the initial retrieval space are not appropriate, by applying the proposed feedback operations, our proposed method effectively improves the search results. We also implemented an experimental search system for semantic document retrieval. Several experimental results including comparisons of our method with the traditional relevance feedback method is presented to clarify how retrieval accuracy was improved by the feedback process and how accurately documents that satisfied the purpose and interests of users were extracted.