一种实时更新索引结构的设计与实现

在搜索引擎的信息检索中,索引性能的优劣是影响检索质量的一个重要因素.本文针对面向主题搜索引擎内容覆盖范围窄、更新速度快的特点,设计了一种实时更新的索引结构,该方案的倒排索引结构打破了传统索引单一结构的形式,由主倒排索引、附加倒排索引和删除文件列表组成,很好的解决了索引的更新问题,实验结果显示该索引结构具有良好的性能.     

搜索引擎倒排索引技术的改进

倒排文件是搜索引擎检索系统普遍采用的索引技术。在实验基础上，针对中文搜索引擎中索引的时效性和传统倒排索引在更新时的缺点，提出分组索引技术和一种追加索引的更新算法，可以有效提高搜索引擎的检索效率，同时不影响系统检索效果。     

基于复合结构的高效索引在线更新策略

倒排索引结构已被广泛地应用在信息检索系统中，倒排索引离线的生成和更新方法已不适合在线更新。文中研究了在线索引更新方法，分析了合并更新、插入更新、复合更新等方法，提出一种结合“插入更新”和“合并更新”优点，并采用多级结构的改进复合更新策略。使用磁盘操作复杂度来衡量更新策略的性能，对几种常用的更新策略和复合更新策略在大量记录下的性能进行理论和实验分析。结果显示，改进复合更新策略具有较好的效率。     

个性化综合倒排索引在Lucene中的应用

该文针对目前通用搜索引擎存在的不足,提出在建立普通倒排索引的基础上,再建立一个记录用户手动标注信息的综合倒排索引,并结合渠道奖励词频算法和文档关注度算法动态更新综合倒排索引,最后在Lucene环境下实现了一个体现用户个性的搜索引擎。     

中文搜索引擎分块倒排索引存储模式

针对开源搜索引擎ASPSeek中的倒排索引的内部结构和访问模式,抽象出了倒排索引的形式化定义。为解决ASPSeek倒排索引更新困难和直接采用操作系统文件缓冲访问机制带来的效率问题,以125万张中文农业网页为样本,结合其特点,提出了一种块大小可变的分块倒排索引存储模式和基于CLOCK替代策略的专用缓冲管理机制。实验结果表明在禁用和启用缓冲的情况下,分块访问模式检索效率都优于ASPSeek。在启用缓冲并使用16万中文词条或5万高频中文词条作为测试集合的条件下,分块存储模式在访问倒排索引100万次后的检索时间趋于常量,即使以全部的827309个词条作为测试集合的条件下,分块存储模式在访问倒排索引200万次后的检索时间趋于收敛。     

基于DHT的分布式索引技术研究与实现

针对索引创建和维护效率不高的问题,设计了一种基于DHT(Distributed Hash Table)的分布式倒排索引构建算法。该算法利用基于改进的Chord网络的分布式哈希表技术,将分词后的结果分散到多个索引服务器上并行构建索引,同时采用前驱列表定位和减少服务器定位延迟的技术,大大缩短了索引构建时间。通过采用统一调度的基于分块的增量式倒排索引更新策略,索引更新时不再需要移动已有的索引文件,提高了索引更新效率。利用周期性稳定算法和前驱列表定位提高了系统的稳定性、容错性和索引的一致性。     

一种支持高效检索的即时更新倒排索引方法

随着万维网的快速发展,产生了一种全新概念的高效文档索引技术,文章实现了一种支持高效检索及即时更新的倒排索引,它是WebME(WebMiningEnvironment)原型系统的一部分,这部分用来对特定的查询进行高效的检索,并支持即时增量索引,即对新加入的文档可以立即加入索引,且不用重新对原内容进行重索引,并且在更新索引时不会影响查询的进行。     

一种高效的倒排索引存储结构

倒排索引是信息检索系统的核心部分,其存储结构对检索的效率和效果起着至关重要的作用,根据汉语词汇的频率分布情况和当前的软硬件环境,提出一种高效的倒排索引结构,在一定程度上能够节省磁盘空间,提高检索效率,并且支持增量更新和删除。     

基于XML文档的藏文网页倒排索引的研究与实现

如今互联网上藏文信息也不断的扩充,藏文搜索引擎作为常用的信息检索的工具和渠道,倒排索引又是搜索引擎的核心技术之一,倒排索引直接影响搜索引擎检索的结果和响应的速度。之所以文章详细介绍了一个自主开发的藏文网页倒排索引系统,它以XML文档的标签内容作为索引对象,定义了文档和文档属性等概念,采用C#语言对文藏文网页正文构建倒排索引的关键技术和实现方法进一步的阐述,实现了基于XML文档的藏文网页倒排索引数据库的底层实现,提供了技术参考。利用这种方法藏文搜索引擎中信息检索的速度和准确率有所提高。     

基于XML的索引动态更新研究与实现

提出了一种基于倒排表的索引,能很好地支持文档结构和内容的动态更新。该索引结构建有基于词条的水平索引和基于元素标志GID的垂直索引,这种双重索引结构能高效地支持文档的局部更新。另外给出了基于上下文共现分析技术的 语义检索和利用关系数据库实现该索引的方法。     

降阶模糊控制算法及其应用研究

提出降价模糊控制算法,利用多级控制策略,采用降阶函数融合信号,可以简化多输入情况下模糊控制器设计。使用线性降价模糊方法成功实现二级倒立摆的控制,证明了算法的有交效性和鲁棒性。     

A performance study of four index structures for set-valued attributes of low cardinality

The efficient retrieval of data items on set-valued attributes is an important research topic that has attracted little attention so far. We studied and modified four index structures (sequential signature files, signature trees, extendible signature hashing, and inverted files) for a fast retrieval of sets with low cardinality. We compared the index structures by implementing them and subjecting them to extensive experiments, investigating the influence of query set size, database size, domain size, and data distribution (synthetic and real). The results of the experiments clearly indicate that inverted files exhibit the best overall behavior of all tested index structures.Received: 0 May 2000, Accepted: 18 October 2000, Published online: 17 September 2003Edited by E. Bertino     

Efficient Update of Indexes for Dynamically Changing Web Documents

Recent work on incremental crawling has enabled the indexed document collection of a search engine to be more synchronized with the changing World Wide Web. However, this synchronized collection is not immediately searchable, because the keyword index is rebuilt from scratch less frequently than the collection can be refreshed. An inverted index is usually used to index documents crawled from the web. Complete index rebuild at high frequency is expensive. Previous work on incremental inverted index updates have been restricted to adding and removing documents. Updating the inverted index for previously indexed documents that have changed has not been addressed. In this paper, we propose an efficient method to update the inverted index for previously indexed documents whose contents have changed. Our method uses the idea of landmarks together with the diff algorithm to significantly reduce the number of postings in the inverted index that need to be updated. Our experiments verify that our landmark-diff method results in significant savings in the number of update operations on the inverted index.     

Inverted index maintenance strategy for flashSSDs: Revitalization of in-place index update strategy

An inverted index is a core data structure of Information Retrieval systems, especially in search engines. Since the search environments have become more dynamic, many on-line index maintenance strategies have been proposed. Previous strategies were designed for HDDs. Consequently, in order to avoid expensive random access cost, Merge-based strategies have been preferred to In-place index update strategies on HDDs. However, flashSSDs have become solid alternatives to HDDs. FlashSSDs currently are adopted in a wide range of areas due to their superior features such as the short access latency, energy efficiency, and high bandwidth. In this article, we first reexamined potentials of In-place index update strategies on flashSSDs. Thanks to the insignificant access latency of flashSSDs, we discovered that In-place index update strategies outperform Merge-based strategies, since In-place index update strategies generate much less amount of I/O than Merge-based strategies despite inducing frequent random accesses. Based on this discovery, we suggest a new inverted index maintenance strategy based on an In-place index update strategy for flashSSDs, called Multipath Flash In-place Strategy (MFIS). To enhance the index maintenance performance, MFIS stores the posting list of each term non-contiguously and exploits the internal parallelism of flashSSDs. Thus, MFIS not only induces the minimum amount of I/O but also utilizes the maximum bandwidth of flashSSDs. Furthermore, MFIS is designed to show high query processing performance by utilizing the internal parallelism of flashSSDs even though the posting list of each term is stored non-contiguously. In our experiments, the index maintenance performance of MFIS was considerably better than other previous maintenance strategies. The index maintenance performance was up to 14.93, 4.04, 5.12, and 2.33 times higher than Merge-based strategies such as Immediate Merge, Geometric Partitioning, Hybrid, and SSD-aware Hybrid, respectively. The query processing performance of MFIS was up to 1.62 times higher than non-contiguous In-place. In addition, MFIS showed almost the best query processing performance as Merge-based strategies did. In conclusion, MFIS is the best on-line inverted index maintenance strategy on flashSSDs in terms of both index maintenance and query processing performance.     

一种多变量过程辨识方法及其实现

分析讨论了在Ｍａｒｋｏｖ参数估计的基础上，通过ＦＩＲ—ＭＰＳＳＭ—ＳＳ变换，实现多变量过程的参数模型辨识的问题，并给出研究实例。该方法回避了多变量过程结构辨识的困难，是工业过程建模的良好途径。     

分布式环境下多关键词并行密文检索方案

对于可搜索加密需要均衡数据的安全性和检索效率。针对SSE-1密文检索方案中检索性能低、单关键词检索模式不足和传统单服务器架构中的单机资源局限性等问题，设计并实现了一种多关键词并行密文检索系统。该系统采用不同的索引加密方式提高密文检索性能；通过对密文倒排索引的切分实现倒排索引的分块检索，克服了单机资源的局限性并提高了检索效率；通过结合分布式特点扩展了传统单机检索架构并实现了多关键词的并行检索。实验结果表明，与SSE-1方案相比，在保证密文数据安全性的前提下所提方案能够提高检索、更新等操作的效率，实现多关键词的检索，同时动态扩展系统分布式架构以提高系统负载能力。     

一种基于倒排索引树的增量更新关联挖掘算法

增量更新关联规则挖掘主要解决事务数据库中交易记录不断更新和最小支持度发生变化时关联规则的维护问题。针对目前诸多增量更新关联规则挖掘算法存在效率低、计算成本高、规则难以维护等问题,提出一种基于倒排索引树的增量更新关联挖掘算法。该算法有效地将倒排索引技术与树型结构相结合,使得交易数据库中的数据不断更新和最小支持度随应用环境不同而不断改变时,以实现无需扫描原始交易数据库和不产生候选项集的情况下生成频繁项集。实验结果表明,该算法只需占用较小的存储空间、且检索项集的效率较高,能高效地解决增量更新关联规则难以维护的问题。