基于局部共现分析和概念语义的查询扩展研究

针对信息检索中文档与查询之间的词不匹配问题,提出了一种基于共现分析和概念语义的查询扩展方法.该方法结合概念语义空间和局部共现分析来实现扩展,并改进了扩展词筛选函数.实验结果表明,该方法对于传统的查询扩展技术的信息查询效果有了很大提高,具有较好的查询性能.     

基于文档团的Markov网络检索模型

查询扩展是提高检索效率的有效方法.但是许多查询扩展方法中扩展词的选择没有充分考虑词项之间以及词项与文档之间的相关性,这样可能在查询扩展时加入太多不相关信息降低检索的性能.通过对文档间相关性和词间相关性的计算,把文档和词关联起来构建Markov网络检索模型,然后根据词项子空间和文档子空间的映射关系提取词团,将提取的词团信息用于查询扩展,使得查询扩展的内容更为相关.实验表明:基于文档团依赖的Markov检索模型能有效地提高检索效果.     

基于语义向量表示的查询扩展方法

针对传统查询扩展方法在专业领域中扩展词与原始查询之间缺乏语义关联的问题,提出一种基于语义向量表示的查询扩展方法。首先,构建了一个语义向量表示模型,通过对语料库中词的上下文语义进行学习,得到词的语义向量表示;其次,根据词语义向量表示,计算词之间的语义相似度;然后,选取与查询中词汇的语义最相似的词作为查询的扩展词,扩展原始查询语句;最后,基于提出的查询扩展方法构建了生物医学文档检索系统,针对基于维基百科或WordNet的传统查询扩展方法和BioASQ 2014—2015参加竞赛的系统进行对比实验和显著性差异指标分析。实验结果表明,基于语义向量表示查询扩展的检索方法所得到结果优于传统查询扩展方法的结果,平均准确率至少提高了1个百分点,在与竞赛系统的对比中,系统的效果均有显著性提高。     

基于语义分析的查询扩展方法

查询扩展是优化信息检索的有效途径。为此,提出一种基于语义分析的查询扩展方法,利用基于互信息的共现模型分析初检文档,并将其作为部分扩展源,用模型的统计结果剪枝由语义词典WordNet生成的语义树,限制扩展范围。从初检文档和语义词典两方面选取扩展词对原查询进行扩展形成新的查询集。对返回结果进行重排序,调整前n篇文档的查准率。实验证明该方法是切实可行的。     

基于迭代方法的多层Markov网络信息检索模型

查询扩展是提高检索效果的有效方法,传统的查询扩展方法大都以单个查询词的相关性来扩展查询词,没有充分考虑词项之间、文档之间以及查询之间的相关性,使得扩展效果不佳。针对此问题,该文首先通过分别构造词项子空间和文档子空间的Markov网络,用于提取出最大词团和最大文档团,然后根据词团与文档团的映射关系将词团分为文档依赖和非文档依赖词团,并构建基于文档团依赖的Markov网络检索模型做初次检索,从返回的检索结果集合中构造出查询子空间的Markov网络,用于提取出最大查询团,最后,采用迭代的方法计算文档与查询的相关概率,并构建出最终的基于迭代方法的多层Markov网络信息检索模型。实验结果表明 该文的模型能较好地提高检索效果。     

基于文档平滑和查询扩展的文档敏感信息检测方法

由于办公终端可能出现敏感信息泄露的风险,对终端上的文档进行敏感信息检测就显得十分重要,但现有敏感信息检测方法中存在上下文信息无关的索引导致文档建模不准确、查询语义扩展不充分的问题。为此,首先提出基于上下文的文档索引平滑算法,构建尽可能保留文档信息的索引;然后改进查询语义扩展算法,结合领域本体中概念敏感度适当扩大敏感信息检测范围;最后将文档平滑和查询扩展融合于语言模型,在其基础上提出了文档敏感信息检测方法。将采用不同索引机制、查询关键字扩展算法及检测模型的四种方法进行比较,所提出的算法在文档敏感信息检测中的查全率、准确率和F值分别为0.798,0.786和0.792,各项性能指标均明显优于对比算法。结果表明该算法是一种能更有效检测敏感信息的方法。     

一种基于局部共现的查询扩展方法

针对信息检索中文档与查询之间的词不匹配问题,本文提出了一种基于局部共现的查询扩展方法LOCOOC。LOCOOC利用词项与所有查询词在局部文档集合中的共现程度来评估扩展词的质量,并整合了词项在语料集中的全局统计信息,使得选取的扩展词与初始查询所表征的主题或概念具有更好的相关性。实验结果表明:与未进行查询扩展时相比,采用LOCOOC方法进行扩展后,平均准确率提高40%以上;与传统的局部反馈方法以及局部上下文分析方法(LCA,Local Context Analysis)相比,LOCOOC不仅具有更优的检索性能,而且有着更好的鲁棒性。     

基于深度语义信息的查询扩展

随着互联网时代的到来，搜索引擎开始被普遍使用。在针对冷门数据时，由于用户的搜索词范围过小，搜索引擎无法检索出需要的数据，此时查询扩展系统可以有效辅助搜索引擎来提供可靠服务。基于全局文档分析的查询扩展方法，提出结合神经网络模型与包含语义信息的语料的语义相关模型，来更深层地提取词语间的语义信息。这些深层语义信息可以为查询扩展系统提供更加全面有效的特征支持，从而分析词语间的可扩展关系。在近义词林、语言知识库“HowNet”义原标注信息等语义数据中抽取局部可扩展词分布，利用神经网络模型的深度挖掘能力将语料空间中每一个词语的局部可扩展词分布拟合成全局可扩展词分布。在与分别基于语言模型和近义词林的查询扩展方法对比实验中，使用基于语义相关模型的查询扩展方法拥有较高的查询扩展效率；尤其针对冷门搜索数据时，语义相关模型的查全率比对比方法分别提高了11.1个百分点与5.29个百分点。     

一种面向元数据描述文档的概念检索方法

元数据描述文档在检索过程中仍然存在着检索词和描述词不匹配的问题。文章在准确描述领域概念之间关系的概念网的支持下,给出检索词和描述词的概念相关度计算公式,提出了用概念扩展来提高检索质量的新方法。并在领域概念网和元数据描述的科技文档组成的实验系统上,进行了多种实验和分析,证明了检索方法的有效性。     

知识管理中的联想检索

提供高效便捷的知识检索途径是知识管理系统走向实用化的关键，但传统的检索方法会遗漏大量有用信息而不适用于知识管理系统。文中根据语义网络中概念之间的语义关系把概念分解成核心网络和同义网络，并构造了一个检索扩展模板，以实现对查询语句的语义扩展，最后构造了一个权值计算函数对检索结果进行排序。     

A Fuzzy-Related Thesaurus for Query Refinement

Query refinement is essential for information retrieval. In this study, a fuzzy-related thesaurus based query refinement mechanism is proposed. This thesaurus can be dynamically generated during the retrieval process for a document collection that is classified by an unsupervised neural network, the self-organising map. In contrast with general relational thesaurus, the fuzzy-related thesaurus is more effective and efficient. The relationships between the terms are based on the classification of a document collection, and thus, the generated thesaurus naturally has more power to enhance retrieval quality. The recognition of the relationships can be done automatically without human involvement, which significantly reduces the cost associated with the construction of the thesaurus. An evaluation on the query refinement mechanism based on the fuzzy-related thesaurus has conducted and the preliminary result is promising. A significant improvement on retrieval performance was observed when a fuzzy-related thesaurus was used for query refinement for a software document collection.     

Efficient query expansion with auxiliary data structures

Query expansion is a well-known method for improving average effectiveness in information retrieval. The most effective query expansion methods rely on retrieving documents which are used as a source of expansion terms. Retrieving those documents is costly. We examine the bottlenecks of a conventional approach and investigate alternative methods aimed at reducing query evaluation time. We propose a new method that draws candidate terms from brief document summaries that are held in memory for each document. While approximately maintaining the effectiveness of the conventional approach, this method significantly reduces the time required for query expansion by a factor of 5–10.     

Deep Learning and Semantic Concept Spaceare Used in Query Expansion

In the practice of information retrieval, there are some problems such as the lack of accurate expression of user query requests, the mismatch between document and query and query optimization. Focusing on these problems, we propose the query expansion method based on conceptual semantic space with deep learning, this hybrid query expansion technique include deep learning and pseudocorrelation feedback, use the deep learning and semantic network WordNet to construct query concept tree in the level of concept semantic space, the pseudo-correlation feedback documents are processed by observation window, compute the co-occurrence weight of the words by using the average mutual information and get the final extended words set. The results of experiment show that the expansion algorithm based on conceptual semantic space with deep learning has better performance than the traditional pseudo-correlation feedback algorithm on query expansion.     

Chinese document re-ranking based on automatically acquired term resource

In this paper, we address the problem of document re-ranking in information retrieval, which is usually conducted after initial retrieval to improve rankings of relevant documents. To deal with this problem, we propose a method which automatically constructs a term resource specific to the document collection and then applies the resource to document re-ranking. The term resource includes a list of terms extracted from the documents as well as their weighting and correlations computed after initial retrieval. The term weighting based on local and global distribution ensures the re-ranking not sensitive to different choices of pseudo relevance, while the term correlation helps avoid any bias to certain specific concept embedded in queries. Experiments with NTCIR3 data show that the approach can not only improve performance of initial retrieval, but also make significant contribution to standard query expansion.     

局部上下文分析剪枝概念树的查询扩展

介绍一种局部上下文分析（LCA）剪枝概念树的方法。利用LCA方法初次检索的与原查询最相关的文章作为备选扩展词的来源,用扩展词剪枝语义词典构造的概念树,补充概念树上不存在的新词,并重新计算扩展词权重。实验表明,在相同的实验条件下该扩展方法查询性能有较大的提高。     

一种基于相关反馈的信息检索模型

针对现有信息检索系统难以按查询需求处理检索文档的问题,提出了一种基于相关反馈的信息检索模型,分析了查询词分解,推导了相关反馈机制和正规化过程,并进一步阐述了文档提取方法。提出的模型通过相关反馈和查询词扩展,克服了传统方法无法计算文档与查询词之间的相似度问题,并能有效地处理检索文档。仿真结果证明了该模型的有效性和可行性。     

Towards an effective automatic query expansion process using an association rule mining approach

The steady growth in the size of textual document collections is a key progress-driver for modern information retrieval techniques whose effectiveness and efficiency are constantly challenged. Given a user query, the number of retrieved documents can be overwhelmingly large, hampering their efficient exploitation by the user. In addition, retaining only relevant documents in a query answer is of paramount importance for an effective meeting of the user needs. In this situation, the query expansion technique offers an interesting solution for obtaining a complete answer while preserving the quality of retained documents. This mainly relies on an accurate choice of the added terms to an initial query. Interestingly enough, query expansion takes advantage of large text volumes by extracting statistical information about index terms co-occurrences and using it to make user queries better fit the real information needs. In this respect, a promising track consists in the application of data mining methods to the extraction of dependencies between terms. In this paper, we present a novel approach for mining knowledge supporting query expansion that is based on association rules. The key feature of our approach is a better trade-off between the size of the mining result and the conveyed knowledge. Thus, our association rules mining method implements results from Galois connection theory and compact representations of rules sets in order to reduce the huge number of potentially useful associations. An experimental study has examined the application of our approach to some real collections, whereby automatic query expansion has been performed. The results of the study show a significant improvement in the performances of the information retrieval system, both in terms of recall and precision, as highlighted by the carried out significance testing using the Wilcoxon?test.     

Multilayer SOM With Tree-Structured Data for Efficient Document Retrieval and Plagiarism Detection

This paper proposes a new document retrieval (DR) and plagiarism detection (PD) system using multilayer self-organizing map (MLSOM). A document is modeled by a rich tree-structured representation, and a SOM-based system is used as a computationally effective solution. Instead of relying on keywords/lines, the proposed scheme compares a full document as a query for performing retrieval and PD. The tree-structured representation hierarchically includes document features as document, pages, and paragraphs. Thus, it can reflect underlying context that is difficult to acquire from the currently used word-frequency information. We show that the tree-structured data is effective for DR and PD. To handle tree-structured representation in an efficient way, we use an MLSOM algorithm, which was previously developed by the authors for the application of image retrieval. In this study, it serves as an effective clustering algorithm. Using the MLSOM, local matching techniques are developed for comparing text documents. Two novel MLSOM-based PD methods are proposed. Detailed simulations are conducted and the experimental results corroborate that the proposed approach is computationally efficient and accurate for DR and PD.