基于倒排索引的密文数据库检索方法研究

针对实际密文数据库的应用,在全文检索倒排索引技术的基础上,设计了一种通过密文倒排索引文件对其进行快速检索的方法。密文索引文件中主要包含有索引项、相对应的记录主键等信息。检索时,通过用检索词匹配索引文件中的索引项,找到对应的记录主键集合,再根据记录主键集合查询密文数据库,获取相应的密文数据,进行解密即可获取明文数据信息。整个检索过程中不对数据库进行解密,从而实现了在不解密的情况下对密文数据库的快速检索。     

定向查询引擎在Web化学数据库集成检索中的应用

Internet上的化学数据库是重要的专业资源,基于超链接分析的搜索引擎还不能索引这类资源。本论文以充分利用Internet上的化学数据库数据为目标,将“一个查询发动多个同级检索引擎,并以结构化的方式组织信息”的方案应用于以化合物标识信息为检索入口的Web化学数据库,建立了一个基于多站点集成检索的Web数据库定向查询引擎。该引擎是一个包括用户交互层、中间检索层、数据提供层的三层Web模型。各层在系统内部分别对应于响应用户检索请求的客户端代理模块、集成远程Web信息的服务器端代理模块,以及提供缓存和检索的关系数据库模块。模型采用JSP＋Java组件的开发方式,在HTTP协议标准发送方法的基础上,采用XML技术对检索返回文档进行结构化数据的提取和表示,利用XML—DBMS实现XML数据的存储和检索,建立了一套针对深层Web数据交换的解决方案。依此方案所建立的ChemDB Portal Search实现了四个分布式Web化学数据库的有效加入、同时检索和统一显示。该系统是针对深层Web信息的挖掘和集成检索的一次尝试,它可为其它领域建立类似的系统提供借鉴。     

具有扩充检索功能的石油化工专业信息搜索引擎的设计与开发

网络信息的日益增加迫切需要适宜的检索工具,特别是进行专业信息的检索,需要体现专业词汇特点的搜索引擎。本文在对搜索引擎核心技术进行研究的基础上,提出了石油化工信息搜索引擎的设计方案,开发了网络机器人模块,实现了海量网页的自动获取;采用最短路径分词和正向最大匹配相结合的算法,实现了中文自动分词;开发了信息索引模块,实现了网页的批量索引和增量索引;开发了信息检索模块,提供布尔逻辑查询,实现摘要自动生成。通过系统集成,初步建立了体现石油化工专业特点的搜索引擎。     

搜索引擎的混合索引技术

倒排文件是搜索引擎检索系统普遍采用的索引技术。针对中文搜索引擎中采用自动分词的全文检索因分词词典规模小导致的检索效率下降与词典规模扩大导致检索效果下降的矛盾,论文在天网搜索引擎的实践基础上,提出了一种基于倒排文件实现的混合索引的方法,它可以有效提高搜索引擎下短语查询的检索效率,同时不影响系统检索效果。     

基于VB与Access的MSDS查询软件的开发与设计

化学品生产或销售企业通过MSDS向客户提供有关化学品详细的安全数据说明,为危险化学品的安全使用、毒性评估、职业健康、生态环境等问题提供技术参考。为使危险化学品企业用户方便、灵活的掌握危险化学品MSDS信息,以Visual Basic 6.0为编程平台,Access2003为后台数据库,开发了MSDS查询软件,实现对化学品的MSDS信息的检索。该软件使用方便、检索快捷,具有很强的实用性和推广价值。     

K-FTP搜索引擎的核心技术

传统FTP搜索引擎对检索结果优化程度不够,会降低检索质量。在FTP用户查询日志的统计分析基础上,采用双字节倒排索引、检索结果自动分类以及查询自动纠错等技术设计了一种高性能的智能化FTP搜索引擎。试验表明该方案能够有效地提高FTP文件检索效率与质量,平均检索响应时间低于500 ms,检索准确率为92.5%。     

面向Web电子产品信息分布式检索系统的设计与实现

为了从这些海量信息中获取“有用的、满足用户需求的信息”,提出一个基于Hadoop和Lucene技术的分布式检索系统架构处理Web电子产品信息检索。利用Hadoop的Map和Reduce实现分布式索引文件的存储,通过Lucene检索技术实现索引文件的访问,从而提高信息检索的效率。并且针对Lucene_Hadoop架构存在粗粒度检索问题,提出了一种细粒度检索方法,减少了系统建立索引的时间。实验表明基于Hadoop和Lucene的分布式检索系统在Web电子产品信息中具有较高的检索性能。     

搜索引擎的文档预处理技术研究

搜索引擎的Robots(自动采集器)对WWW站点进行搜索,并将搜索到WWW页面信息存入搜索引擎的临时数据库,然后再对页面信息进行整理,形成规范的页面索引,存入索引数据库,供用户查询。设计搜索引擎的第一步就是对输入的各项进行规格化处理,使之标准化。在建立索引文件之前,文档里的信息要进行处理,例如文档标准、索引词抽取,这个过程称为项目规格化(Item normalization)。项目规格化分三步进行:建立可搜索的数据库结构,禁用词表使用和抽取词干。     

基于语义的企业搜索引擎的研究与实现

企业中有大量的数据如:文件、邮件、图片等非结构化数据存放在企业内计算机系统中,并且数据增长很快。因此,如何方便,快捷,安全地获取企业内部的信息内容,造就了一个新的但实际上非常传统的应用——企业搜索引擎。本文通过对现有的各种搜索引擎进行分析,提出一种基于语义的企业搜索引擎的框架,这种框架关键是结合本体对各种资源进行索引,提高了检索的性能。并以Lucene为基础,结合Ajax技术实现系统原型,在实验中取得了很好的结果。     

基于Lucene的烟草个性化搜索引擎的研究与实现

研究了Lucene搜索引擎框架在烟草行业的信息获取和搜索方面的系统应用。如何在异构的海量数据中获取与烟草行业相关的数据是实现烟草行业信息化管理技术的一个重要分支。通过对不同文件格式的数据处理、烟草行业相关站点的数据抓取、搜索引擎创建和数据检索四个模块框架来实现系统。Web页面基于Heritrix烟草信息数据抓取,其中的非结构化数据或者半结构化数据和不同的文件格式通过Tika工具统一转为为文本,通过Lucene对结构化和非结构化数据创建索引以实现数据检索,由此实现烟草信息搜索引擎系统的实现。     

A computational framework for retrieval of document fragments based on decomposition schemes in engineering information management

Retrieval of document fragments has a great potential for application in engineering information management. Frequently engineers have neither the time nor inclination to sift through long documents for small pieces of useful information. Yet it is frequently in the form of one or more long documents that the information that they seek is presented. Supporting the delivery of the right information, in the right format and in the right quantity motivates the search for better ways of handling document sub-components or fragments. Document fragment retrieval can be facilitated using modern computational technologies. This paper proposes a novel framework for information access utilising state-of-the-art computational technologies and introducing the use of multiple document structure views through decomposition schemes. The framework integrates document structure study, mark-up technologies, automated fragment extraction, faceted classification and a document navigation mechanism to achieve the target of retrieval of specific document fragments using precise, complex queries. These disparate elements have been brought together in an exploratory Engineering Document Content Management System (EDCMS). Using this, investigations using representative engineering documents have shown that information users can access and retrieve document content – at fragment level rather than at document level – both through data in a document and document metadata, through different perspectives and at different granularities, and simultaneously across multiple documents as well as within a single document.     

Querying Documents using Content, Structure and Properties

Much information is nowadays stored electronically in document bases. Users retrieve information from these document bases by browsing and querying. While a large number of tools are available nowadays, not much work has been done on tools that support queries involving all the characteristics of documents as well as the use of domain knowledge during the search for information. In this paper we propose a query language that allows for querying documents using content information, information about the logical structure of the documents as well as information about properties of the documents. Domain knowledge is taken into account during the search as well. We also present an architecture for a system supporting such a language and we describe a prototype implementation together with test results.     

On the identification of electronic documents and collections

In this work, a series of principal questions of the existence of documents in the information space is considered. Based on natural concepts of a document as an information object, the questions of document identification, concepts of the instance, copy, and original are studied, along with accompanying concepts of the registry, catalogue, and repository. Requirements for the identification and rules of the existence and use of documents are considered. An approach is suggested to make possible an adequate treatment of copies of documents for the case when copies are more available than originals. Features of an XML-based prototype are described. The work can serve as a basis for constructing document collections of an arbitrary nature, especially for documents naturally distributed in the information space.     

Logical structure analysis: From HTML to XML

This paper presents an efficient method for extracting a logical structure from a Web document. The proposed method consists of three phases: visual grouping, element identification, and logical grouping. To produce a logical structure more accurately, the proposed method defines a document model that is able to describe logical structure information of a specific document class. Since the proposed method is based on a visual structure from the visual grouping phase as well as a document model that describes logical structure information of a document type, it supports sophisticated structure analysis. Experimental results with HTML documents from the Web show that the method has performed logical structure analysis successfully, compared with previous work. Particularly, the method generates XML documents as the result of structure analysis, so that it enhances the reusability of documents.     

Table form document analysis based on the document structure grammar

Structure analysis of table form documents is an important issue because a printed document and even an electronic document do not provide logical structural information but merely geometrical layout and lexical information. To handle these documents automatically, logical structure information is necessary. In this paper, we first analyze the elements of the form documents from a communication point of view and retrieve the grammatical elements that appear in them. Then, we present a document structure grammar which governs the logical structure of the form documents. Finally, we propose a structure analysis system of the table form documents based on the grammar. By using grammar notation, we can easily modify and keep it consistent, as the rules are relatively simple. Another advantage of using grammar notation is that it can be used for generating documents only from logical structure. In our system, documents are assumed to be composed of a set of boxes and they are classified as seven box types. Then the box relations between the indication box and its associated entry box are analyzed based on the semantic and geometric knowledge defined in the document structure grammar. Experimental results have shown that the system successfully analyzed several kinds of table forms.     

Machine Learning for Intelligent Processing of Printed Documents

A paper document processing system is an information system component which transforms information on printed or handwritten documents into a computer-revisable form. In intelligent systems for paper document processing this information capture process is based on knowledge of the specific layout and logical structures of the documents. This article proposes the application of machine learning techniques to acquire the specific knowledge required by an intelligent document processing system, named WISDOM++, that manages printed documents, such as letters and journals. Knowledge is represented by means of decision trees and first-order rules automatically generated from a set of training documents. In particular, an incremental decision tree learning system is applied for the acquisition of decision trees used for the classification of segmented blocks, while a first-order learning system is applied for the induction of rules used for the layout-based classification and understanding of documents. Issues concerning the incremental induction of decision trees and the handling of both numeric and symbolic data in first-order rule learning are discussed, and the validity of the proposed solutions is empirically evaluated by processing a set of real printed documents.     

Personal Document Management and Retrieval: A Knowledge-Based Approach

This paper presents a knowledge-based approach to managing and retrieving personal documents. The dual document models consist of a document type hierarchy and a folder organization. The document type hierarchy is used to capture the layout, logical and conceptual structures of documents. The folder organization mimics the user's real-world document filing system for organizing and storing documents in an office environment. Predicate-based representation of documents is formalized for specifying knowledge about documents. Document filing and retrieval are predicate-driven. The filing criteria for the folders, which are specified in terms of predicates, govern the grouping of frame instances, regardless of their document types. We incorporated the notions of document type hierarchy and folder organization into the multilevel architecture of document storage. This architecture supports various text-based information retrieval techniques and content-based multimedia information retrieval techniques. The paper also proposes a knowledge-based query-preprocessing algorithm, which reduces the search space. For automating the document filing and retrieval, a predicate evaluation engine with a knowledge base is proposed. The learning agent is responsible for acquiring the knowledge needed by the evaluation engine.     

Don't Search,Just Show Me What I Did: Visualizing Provenance of Documents and Applications

Computer documents have evolved over time. As a result, this article presents the results from a survey that redefines what a “document” is. The study discovered that people think a document is something that holds information; is manipulated by people (not the system); and is anything that can be seen, heard, or touched. Second, based on the results of the survey, we present a novel, real-time visualization tool that shows the results of nonintrusive tracking of documents and applications for a 6-month period. The tool focuses on document provenance—the history or genealogy of a document. It shows every document and application used as well as what happened to those documents (e.g., if the documents were moved, renamed, and/or deleted). These evaluations of the visualization tool are promising in that it helped with refinding documents, finding behavior workflow patterns, finding insight into general document usage, and performing forensic-type activities.