文本检索的统计语言建模方法综述

统计语言建模技术（statistical language modeling,SLM）已逐渐成为当前语言信息处理的主流技术之一.近几年的研究和实验表明,SLM技术在文本检索领域有着广阔的发展前景和拓展空间.对基于SLM的文本检索方法（SLMTR）进行了综述,重点论述SLMTR的主要方法和关键技术.首先对查询似然检索模型进行形式化的描述;然后详细论述语言模型的估计和数据平滑问题;并讨论了平滑对检索性能的影响;之后简要介绍了对查询似然模型的一些主要的扩展和改进工作;最后的总结部分讨论了SLMTR所面临的一些挑战.     

基于查询模型的专家检索方法分析

在对基于语言模型的专家检索方法进行分析后,提出查询式建模的专家检索方法。该方法通过运用查询扩展技术和术语生成概率建立一套新的专家排序体系。根据对国际文本检索会议组织的企业追踪专题进行初步实验评估,结果表明,这种方法能够有效地提高专家检索任务的检索准确度。     

基于统计模型和小波变换的文本检索方法

针对当前几种常用文本检索方法的不足,文中基于统计模型和小波变换,提出了一种新的文本检索方法。与传统方法的主要区别在于:1)利用小波变换把输入信号引入到频域进行处理,消除了交叉比较运算的巨大计算量;2)在进行相关度计算时,同时考虑了检索词的出现次数和出现位置因素,有效提高了检索精确度。理论分析和实验结果表明该方法较传统方法在查准率和查询速度上均有所提高。     

基于统计模型和小波变换的文本检索方法

针对当前几种常用文本检索方法的不足,文中基于统计模型和小波变换,提出了一种新的文本检索方法。与传统方法的主要区别在于：1）利用小波变换把输入信号引入到频域进行处理,消除了交叉比较运算的巨大计算量;2）在进行相关度计算时,同时考虑了检索词的出现次数和出现位置因素,有效提高了检索精确度。理论分析和实验结果表明该方法较传统方法在查准率和查询速度上均有所提高。     

一种融合文本重要性的文本检索算法

分析了查询似然模型,针对传统查询似然检索模型没有考虑文本间相关性的缺点,将链接模型引入到文本检索中,提出一个计算文本间相关性的DocRank算法。该算法通过计算两两文本间的相关性,构建一个文本矩阵,利用幂迭代法得到每个文本的优先度值,将其融合到查询似然检索模型中以准确定位所检索文本,实验结果验证了改进算法在文本检索中的有效性。     

基于查询语义特性的稠密文本检索模型

针对现有的稠密文本检索模型（dense passage retrieval,DPR）存在的负采样效率低、易产生过拟合等问题,提出了一种基于查询语义特性的稠密文本检索模型（Q-DPR）。首先,针对模型的负采样过程,提出了一种基于近邻查询的负采样方法。该方法通过检索近邻查询,快速地构建高质量的负相关样本,以降低模型的训练成本。其次,针对模型易产生过拟合的问题,提出了一种基于对比学习的查询自监督方法。该方法通过建立查询间的自监督对比损失,缓解模型对训练标签的过拟合,从而提升模型的检索准确性。Q-DPR在面向开放领域问答的大型数据集MSMARCO上表现优异,取得了0.348的平均倒数排名以及0.975的召回率。实验结果证明,该模型成功地降低了训练的开销,同时也提升了检索的性能。     

基于查询词扩展的文本检索算法研究

针对文本检索中所使用的查询词可能与文本词语不匹配而影响检索效果这一问题,提出了一种基于上下文的查询词扩展的方法,该方法根据查询词出现的上下文信息进行扩展词选择,同时考虑到查询扩展词与整个查询语句以及查询词的位置关系。实验结果表明,该方法大大提高了平均查准率。     

微博检索的研究进展

随着微博的快速发展,微博检索已经成为近年来研究领域的热点之一。该文首先以TREC Microblog数据为基础,从分析微博文档和微博查询两方面出发,得出微博检索与传统文本检索之间的两点不同: 一是微博文档相较于网页具有很多独有的特征;二是微博查询属于时间敏感查询,即在排序时除了考虑文本的语义相似度,还需要考虑时间因素,将这类方法统称为时间感知的检索技术。这两点差异使得已有的信息检索技术不能满足微博搜索的需求。该文主要介绍了近年来这两方面的相关研究: 首先描述了微博本身的多种特征以及基于这些特征提出的检索方法;然后以传统信息检索过程为主线,分别介绍了将时间信息用于文本表示、文档先验、查询扩展三方面的排序模型,最后总结了已有工作并且对未来研究内容进行了展望。     

基于关联规则挖掘的查询扩展检索性能研究

介绍了基于关联规则的局部反馈查询扩展基本思想,重点研究关联规则支持度、置信度和扩展词数量对查询扩展检索性能的影响.实验结果表明,这种查询扩展的检索性能对其支持度、置信度以及扩展词数量比较敏感;从关联规则获得的扩展词可以分为两类,即与原查询正相关的扩展词和与原查询负相关或者假相关的扩展词(即噪音),前者可以提高和改善查询扩展的检索性能,而后者只能降低其检索性能.     

P2P社区的信息检索算法设计

为解决P2P社区的资源定位及信息检索问题,采用混合型P2P网络模型,将社区内的检索划分为本地检索、组内搜索和组间搜索。对于本地检索设计了新的词条权重的计算方法,解决了同构文档集内的文本检索问题。对于组内搜索和组间搜索,通过设计节点选择策略,使一部分与查询相关度高的节点执行查询任务。最后提出结果融合的方法并对特定的实验数据进行测试,实验表明设计的算法在较小的查询开销下,能取得较好的检索效果。     

Query Expansion for Effective Retrieval Results of Hindi–English Cross-Lingual IR

Information retrieval (IR) is the science of identifying documents or sub-documents from a collection of information or database. The collection of information does not necessarily be available in only one language as information does not depend on languages. Monolingual IR is the process of retrieving information in query language whereas cross-lingual information retrieval (CLIR) is the process of retrieving information in a language that differs from query language. In current scenario, there is a strong demand of CLIR system because it allows the user to expand the international scope of searching a relevant document. As compared to monolingual IR, one of the biggest problems of CLIR is poor retrieval performance that occurs due to query mismatching, multiple representations of query terms and untranslated query terms. Query expansion (QE) is the process or technique of adding related terms to the original query for query reformulation. Purpose of QE is to improve the performance and quality of retrieved information in CLIR system. In this paper, QE has been explored for a Hindi–English CLIR in which Hindi queries are used to search English documents. We used Okapi BM25 for documents ranking, and then by using term selection value, translated queries have been expanded. All experiments have been performed using FIRE 2012 dataset. Our result shows that the relevancy of Hindi–English CLIR can be improved by adding the lowest frequency term.     

An efficient inverted index technique for XML documents using RDBMS

The inverted index is widely used in the existing information retrieval field. In order to support containment queries for structured documents such as XML, it needs to be extended. Previous work suggested an extension in storing the inverted index for XML documents and processing containment queries, and compared two implementation options: using an RDBMS and using an Information Retrieval (IR) engine. However, the previous work has two drawbacks in extending the inverted index. One is that the RDBMS implementation is generally much worse in the performance than the IR engine implementation. The other is that when a containment query is processed in an RDBMS, the number of join operations increases in proportion to the number of containment relationships in the query and a join operation always occurs between large relations. In order to solve these problems, we propose in this paper a novel approach to extend the inverted index for containment query processing, and show its effectiveness through experimental results. In particular, our performance study shows that (1) our RDBMS approach almost always outperforms the previous RDBMS and IR approaches, (2) our RDBMS approach is not far behind our IR approach with respect to performance, and (3) our approach is scalable to the number of containment relationships in queries. Therefore, our results suggest that, without having to make any modifications on the RDBMS engine, a native implementation using an RDBMS can support containment queries as efficiently as an IR implementation.     

Enhanced Neuro-Fuzzy-Based Crop Ontology for Effective Information Retrieval

Ontology is the progression of interpreting the conceptions of the information domain for an assembly of handlers. Familiarizing ontology as information retrieval (IR) aids in augmenting the searching effects of user-required relevant information. The crux of conventional keyword matching-related IR utilizes advanced algorithms for recovering facts from the Internet, mapping the connection between keywords and information, and categorizing the retrieval outcomes. The prevailing procedures for IR consume considerable time, and they could not recover information proficiently. In this study, through applying a modified neuro-fuzzy algorithm (MNFA), the IR time is mitigated, and the retrieval accuracy is enhanced for trouncing the above-stated downsides. The proposed method encompasses three phases: i) development of a crop ontology, ii) implementation of the IR system, and iii) processing of user query. In the initial phase, a crop ontology is developed and evaluated by gathering crop information. In the next phase, a hash tree is constructed using closed frequent patterns (CFPs), and MNFA is used to train the database. In the last phase, for a specified user query, CFP is calculated, and similarity assessment results are retrieved using the database. The performance of the proposed system is measured and compared with that of existing techniques. Experimental results demonstrate that the proposed MNFA has an accuracy of 92.77% for simple queries and 91.45% for complex queries.     

Training query filtering for semi-supervised learning to rank with pseudo labels

Semi-supervised learning is a machine learning paradigm that can be applied to create pseudo labels from unlabeled data for learning a ranking model, when there is only limited or no training examples available. However, the effectiveness of semi-supervised learning in information retrieval (IR) can be hindered by the low quality pseudo labels, hence the need for the training query filtering that removes the low quality queries. In this paper, we assume two application scenarios with respect to the availability of human labels. First, for applications without any labeled data available, a clustering-based approach is proposed to select the high quality training queries. This approach selects the training queries following the empirical observation that the relevant documents of high quality training queries are highly coherent. Second, for applications with limited labeled data available, a classification-based approach is proposed. This approach learns a weak classifier to predict the retrieval performance gain of a given training query by making use of query features. The queries with high performance gains are selected for the following transduction process to create the pseudo labels for learning to rank algorithms. Experimental results on the standard LETOR dataset show that our proposed approaches outperform the strong baselines.     

Integrating relevance feedback techniques for image retrieval using reinforcement learning

Relevance feedback (RF) is an interactive process which refines the retrievals to a particular query by utilizing the user's feedback on previously retrieved results. Most researchers strive to develop new RF techniques and ignore the advantages of existing ones. In this paper, we propose an image relevance reinforcement learning (IRRL) model for integrating existing RF techniques in a content-based image retrieval system. Various integration schemes are presented and a long-term shared memory is used to exploit the retrieval experience from multiple users. Also, a concept digesting method is proposed to reduce the complexity of storage demand. The experimental results manifest that the integration of multiple RF approaches gives better retrieval performance than using one RF technique alone, and that the sharing of relevance knowledge between multiple query sessions significantly improves the performance. Further, the storage demand is significantly reduced by the concept digesting technique. This shows the scalability of the proposed model with the increasing-size of database.     

基于位置语言模型的中文信息检索系统的研究

在大多数现有的检索模型中常常忽略了如下事实:一个文档中匹配到的查询词项的近邻性和打分时所基于的段落检索也可以被用来促进文档的打分。受此启发,提出了基于位置语言模型的中文信息检索系统,首先通过定义位置传播数的概念,为每个位置单独地建立语言模型;然后通过引入KL-divergence检索模型,并结合位置语言模型给每个位置单独打分;最后由多参数打分策略得到文档的最终得分。实验中还重点比较了基于词表和基于二元两种中文索引方法在位置语言模型中的检索效果。在标准NTCIR5、NTCIR6测试集上的实验结果表明,该检索方法在两种索引方式上都显著改善了中文检索系统的性能,并且优于向量空间模型、BM25概率模型、统计语言模型。     

Integrating document and data retrieval based on XML

For querying structured and semistructured data, data retrieval and document retrieval are two valuable and complementary techniques that have not yet been fully integrated. In this paper, we introduce integrated information retrieval (IIR), an XML-based retrieval approach that closes this gap. We introduce the syntax and semantics of an extension of the XQuery language called XQuery/IR. The extended language realizes IIR and thereby allows users to formulate new kinds of queries by nesting ranked document retrieval and precise data retrieval queries. Furthermore, we detail index structures and efficient query processing approaches for implementing XQuery/IR. Based on a new identification scheme for nodes in node-labeled tree structures, the extended index structures require only a fraction of the space of comparable index structures that only support data retrieval.     

Query performance prediction

The prediction of query performance is an interesting and important issue in Information Retrieval (IR). Current predictors involve the use of relevance scores, which are time-consuming to compute. Therefore, current predictors are not very suitable for practical applications. In this paper, we study six predictors of query performance, which can be generated prior to the retrieval process without the use of relevance scores. As a consequence, the cost of computing these predictors is marginal. The linear and non-parametric correlations of the proposed predictors with query performance are thoroughly assessed on the Text REtrieval Conference (TREC) disk4 and disk5 (minus CR) collection with the 249 TREC topics that were used in the recent TREC2004 Robust Track. According to the results, some of the proposed predictors have significant correlation with query performance, showing that these predictors can be useful to infer query performance in practical applications.     

基于查询术语同义词的扩展信念网络检索模型

针对信念网络模型没有考虑术语之间关系的缺陷,引入了查询同义词的概念,提出了一个基于查询术语同义词的扩展信念网络检索模型。给出了模型的拓扑结构,讨论了利用新模型进行信息检索的方法,并给出了一个实用案例。新模型同时考虑了用户查询术语及其同义词的作用,提高了检索性能。