Query Expansion for Effective Retrieval Results of Hindi–English Cross-Lingual IR

Information retrieval (IR) is the science of identifying documents or sub-documents from a collection of information or database. The collection of information does not necessarily be available in only one language as information does not depend on languages. Monolingual IR is the process of retrieving information in query language whereas cross-lingual information retrieval (CLIR) is the process of retrieving information in a language that differs from query language. In current scenario, there is a strong demand of CLIR system because it allows the user to expand the international scope of searching a relevant document. As compared to monolingual IR, one of the biggest problems of CLIR is poor retrieval performance that occurs due to query mismatching, multiple representations of query terms and untranslated query terms. Query expansion (QE) is the process or technique of adding related terms to the original query for query reformulation. Purpose of QE is to improve the performance and quality of retrieved information in CLIR system. In this paper, QE has been explored for a Hindi–English CLIR in which Hindi queries are used to search English documents. We used Okapi BM25 for documents ranking, and then by using term selection value, translated queries have been expanded. All experiments have been performed using FIRE 2012 dataset. Our result shows that the relevancy of Hindi–English CLIR can be improved by adding the lowest frequency term.     

一种基于Lucene的高效索引建立方法

随着计算机技术和互联网技术的发展,人们也越来越多的使用电子文档记录数据,电子文档具有容量大,易存储和易转移等特点,但当数据量较大时查找电子文档的内容就需要借助索引技术加快搜索速度,索引技术的优劣直接影响用户的使用体验。主要探讨单机存储大规模文档时高效索引的建立问题,论文分析检索系统的组成和原理,最后使用Lucene工具包通过多线程建立多个索引目录的方式,加速索引的建立以及索引查询的速度。实验结果表明,该文的方法能有效提升索引的创建和搜索速度。     

WebDB: A System for Querying Semi-structured Data on the Web

The World-Wide Web can be viewed as a collection of semi-structured multimedia documents in the form of Web pages connected through hyperlinks. Unlike most web search engines, which primarily focus on information retrieval functionality, WebDB aims at supporting a comprehensive database-like query functionality, including selection, aggregation, sorting, summary, grouping, and projection. WebDB allows users to access (1) document level information, such as title, URL, length, keywords types and last modified date; (2) intra-document structures, such as tables, forms and images and (3) inter-document linkage information, such as destination URLs and anchors. With these three types of information, comprehensive queries for complex Web-based applications, such as Web mining and Web site management, can be answered. WebDB is based on object-relational concepts: Object-oriented modeling and relational query language. In this paper, we present the data model, language and implementation of WebDB. We also present the novel visual query/browsing interface for semi-structured Web and Web documents. Our system provides high usability compared with other existing systems.     

基于全文文档检索技术的化学教学模式

为了利用网络资源进行化学教学,提出了使用全文文档检索技术整合网络资源进行教学的模式。该技术由3部分组成：一是文档系统,各种格式的文档以文件的形式在服务器硬盘上使用文件系统进行组织。二是全文检索系统,使用Index Server对文档进行过滤和索引。三是检索系统,以IIS（Internet Information Server）为Web服务器,利用ADO访问Index Server数据库,使用ASP编程,实现检索和排序。实践证明该模式实现容易,操作简单,性能优秀,适合于大学化学教学。     

Effectiveness of keyword-based display and selection of retrieval results for interactive searches

We present an approach to increasing the effectiveness of ranked-output retrieval systems that relies on graphical display and user manipulation of “views” of retrieval results, where a view is the subset of retrieved documents that contain a specified subset of query terms. This approach has been implemented in a system named VIEWER (VIEwing WEb Results), acting as an interface to available search engines. An experimental evaluation of the performance of VIEWER in contrast to AltaVista is the major focus of the paper. We first report the results of an experiment on single, short query searches where VIEWER, used as an interactive ranking system, markedly outperformed AltaVista. We then concentrate on a more realistic searching scenario, involving free query formulation, unconstrained selection of retrieval results, and possibility of query reformulation. We report the results of an experiment where the use of VIEWER, compared to AltaVista, seemed to shift the user effort from inspection to evaluation of results, increasing retrieval effectiveness, and user satisfaction. In particular, we found that the VIEWER users retrieved half as many nonrelevant documents as the AltaVista users while retrieving a comparable number of relevant documents. Published online: 22 September 2000     

A model of cluster searching based on classification

The use of document clusters has been suggested as an efficient file organization for a document retrieval system. It is possible that by using this information about the relationships between documents that the effectiveness of the system (i.e. its ability to distinguish relevant from non-relevant documents) may also be improved. In this paper a probabilistic model of cluster searching based on query classification is described. This model is tested with retrieval experiments which indicate that it can be more effective than heuristic cluster searches and cluster searches based on other models. It can also be more effective than a full search in which every document is compared to the query. The efficiency aspects of the implementation of the model are discussed.     

Image retrieval using efficient local-area matching

We present an efficient and accurate method for retrieving images based on color similarity with a given query image or histogram. The method matches the query against parts of the image using histogram intersection. Efficient searching for the best matching subimage is done by pruning the set of subimages using upper bound estimates. The method is fast, has high precision and recall and also allows queries based on the positions of one or more objects in the database image. Experimental results showing the efficiency of the proposed search method, and high precision and recall of retrieval are presented. Received: 20 January 1997 / Accepted: 5 January 1998     

一种通过内容和结构查询文档数据库的方法

文档是有一定逻辑结构的,标题、章节、段落等这些概念是文档的内在逻辑.不同的用户对文档的检索,有不同的需求,检索系统如何提供有意义的信息,一直是研究的中心任务.结合文档的结构和内容,对结构化文件的检索,提出了一种新的计算相似度的方法.这种方法可以提供多粒度的文档内容的检索,包括从单词、短语到段落或者章节.基于这种方法实现了一个问题回答系统,测试集是微软的百科全书Encarta,通过与传统方法实验比较,证明通过这种方法检索的文章片断更合理、更有效.     

An architectural design for effective information retrieval in semantic web

The current web IR system retrieves relevant information only based on the keywords which is inadequate for that vast amount of data. It provides limited capabilities to capture the concepts of the user needs and the relation between the keywords. These limitations lead to the idea of the user conceptual search which includes concepts and meanings. This study deals with the Semantic Based Information Retrieval System for a semantic web search and presented with an improved algorithm to retrieve the information in a more efficient way.This architecture takes as input a list of plain keywords provided by the user and the query is converted into semantic query. This conversion is carried out with the help of the domain concepts of the pre-existing domain ontologies and a third party thesaurus and discover semantic relationship between them in runtime. The relevant information for the semantic query is retrieved and ranked according to the relevancy with the help of an improved algorithm. The performance analysis shows that the proposed system can improve the accuracy and effectiveness for retrieving relevant web documents compared to the existing systems.     

Techniques and data structures for efficient multimedia retrieval based on similarity

As more and more information is captured and stored in digital form, many techniques and systems have been developed for indexing and retrieval of text documents, audio, images, and video. The retrieval is normally based on similarities between extracted feature vectors of the query and stored items. Feature vectors are usually multidimensional. When the number of stored objects and/or the number of dimensions of the feature vectors are large, it will be too slow to linearly search all stored feature vectors to find those that satisfy the query criteria. Techniques and data structures are thus required to organize feature vectors and manage the search process so that objects relevant to the query can be located quickly. This paper provides a survey of these techniques and data structures.     

A personalized query expansion approach for engineering document retrieval

Engineers create engineering documents with their own terminologies, and want to search existing engineering documents quickly and accurately during a product development process. Keyword-based search methods have been widely used due to their ease of use, but their search accuracy has been often problematic because of the semantic ambiguity of terminologies in engineering documents and queries. The semantic ambiguity can be alleviated by using a domain ontology. Also, if queries are expanded to incorporate the engineer’s personalized information needs, the accuracy of the search result would be improved. Therefore, we propose a framework to search engineering documents with less semantic ambiguity and more focus on each engineer’s personalized information needs. The framework includes four processes: (1) developing a domain ontology, (2) indexing engineering documents, (3) learning user profiles, and (4) performing personalized query expansion and retrieval. A domain ontology is developed based on product structure information and engineering documents. Using the domain ontology, terminologies in documents are disambiguated and indexed. Also, a user profile is generated from the domain ontology. By user profile learning, user’s interests are captured from the relevant documents. During a personalized query expansion process, the learned user profile is used to reflect user’s interests. Simultaneously, user’s searching intent, which is implicitly inferred from the user’s task context, is also considered. To retrieve relevant documents, an expanded query in which both user’s interests and intents are reflected is then matched against the document collection. The experimental results show that the proposed approach can substantially outperform both the keyword-based approach and the existing query expansion method in retrieving engineering documents. Reflecting a user’s information needs precisely has been identified to be the most important factor underlying this notable improvement.     

Web Page Recommendation Using Distributional Recurrent Neural Network

In the data retrieval process of the Data recommendation system, the matching prediction and similarity identification take place a major role in the ontology. In that, there are several methods to improve the retrieving process with improved accuracy and to reduce the searching time. Since, in the data recommendation system, this type of data searching becomes complex to search for the best matching for given query data and fails in the accuracy of the query recommendation process. To improve the performance of data validation, this paper proposed a novel model of data similarity estimation and clustering method to retrieve the relevant data with the best matching in the big data processing. In this paper advanced model of the Logarithmic Directionality Texture Pattern (LDTP) method with a Metaheuristic Pattern Searching (MPS) system was used to estimate the similarity between the query data in the entire database. The overall work was implemented for the application of the data recommendation process. These are all indexed and grouped as a cluster to form a paged format of database structure which can reduce the computation time while at the searching period. Also, with the help of a neural network, the relevancies of feature attributes in the database are predicted, and the matching index was sorted to provide the recommended data for given query data. This was achieved by using the Distributional Recurrent Neural Network (DRNN). This is an enhanced model of Neural Network technology to find the relevancy based on the correlation factor of the feature set. The training process of the DRNN classifier was carried out by estimating the correlation factor of the attributes of the dataset. These are formed as clusters and paged with proper indexing based on the MPS parameter of similarity metric. The overall performance of the proposed work can be evaluated by varying the size of the training database by 60%, 70%, and 80%. The parameters that are considered for performance analysis are Precision, Recall, F1-score and the accuracy of data retrieval, the query recommendation output, and comparison with other state-of-art methods.     

A new evolutionary algorithm combining simulated annealing and genetic programming for relevance feedback in fuzzy information retrieval systems

 Relevance feedback techniques have demonstrated to be a powerful means to improve the results obtained when a user submits a query to an information retrieval system as the world wide web search engines. These kinds of techniques modify the user original query taking into account the relevance judgements provided by him on the retrieved documents, making it more similar to those he judged as relevant. This way, the new generated query permits to get new relevant documents thus improving the retrieval process by increasing recall. However, although powerful relevance feedback techniques have been developed for the vector space information retrieval model and some of them have been translated to the classical Boolean model, there is a lack of these tools in more advanced and powerful information retrieval models such as the fuzzy one. In this contribution we introduce a relevance feedback process for extended Boolean (fuzzy) information retrieval systems based on a hybrid evolutionary algorithm combining simulated annealing and genetic programming components. The performance of the proposed technique will be compared with the only previous existing approach to perform this task, Kraft et al.'s method, showing how our proposal outperforms the latter in terms of accuracy and sometimes also in time consumption. Moreover, it will be showed how the adaptation of the retrieval threshold by the relevance feedback mechanism allows the system effectiveness to be increased.     

基于ElasticSearch的车型搜索引擎在保险系统中的设计和实现

随着保险行业信息化规模的不断扩大,各垂直领域的业务数据越来越多,不可避免地给传统结构化数据库在存储和查询效率方面带来了巨大挑战。如何实现数据的冗余备份和快速、高效查询已成为企业信息技术的一大难题。本文提出一种基于ElasticSearch的车型分布式搜索引擎,同时结合Logstash进行数据收集,实现车型数据的过滤存储和索引,并为保险出单系统提供统一的查询入口和高效的检索服务。实践表明,该系统可实现数据的冗余备份并提高检索车型数据的效率,目前已经在保险行业核心系统得到实际应用,取得了较好效果。     

A New Retrieval Model Based on TextTiling for Document Similarity Search

Document similarity search is to find documents similar to a given query document and return a ranked list of similar documents to users, which is widely used in many text and web systems, such as digital library, search engine, etc. Traditional retrieval models, including the Okapi's BM25 model and the Smart's vector space model with length normalization, could handle this problem to some extent by taking the query document as a long query. In practice, the Cosine measure is considered as the best model for document similarity search because of its good ability to measure similarity between two documents. In this paper, the quantitative performances of the above models are compared using experiments. Because the Cosine measure is not able to reflect the structural similarity between documents, a new retrieval model based on TextTiling is proposed in the paper. The proposed model takes into account the subtopic structures of documents. It first splits the documents into text segments with TextTiling and calculates the similarities for different pairs of text segments in the documents. Lastly the overall similarity between the documents is returned by combining the similarities of different pairs of text segments with optimal matching method. Experiments are performed and results show: 1) the popular retrieval models (the Okapi's BM25 model and the Smart's vector space model with length normalization) do not perform well for document similarity search; 2) the proposed model based on TextTiling is effective and outperforms other models, including the Cosine measure; 3) the methods for the three components in the proposed model are validated to be appropriately employed.     

一种改进的页面相似性度量方法

Web 信息检索是指从大量Web 文档集合中找到与给定的查询请求相关的、恰当数目的文档子集。为了更准确地找到相似文档,借助于两个页面的单词覆盖程度,提出一种改进的Web 页面检索度量方法,并在KNN分类实验中得到验证。     

Efficient indexing for semantic search

The increasing performance and wider spread use of automated semantic annotation and entity linking platforms has empowered the possibility of using semantic information in information retrieval. While keyword-based information retrieval techniques have shown impressive performance, the addition of semantic information can increase retrieval performance by allowing for more accurate sense disambiguation, intent determination, and instance identification, just to name a few. Researchers have already delved into the possibility of integrating semantic information into practical search engines using a combination of techniques such as using graph databases, hybrid indices and adapted inverted indices, among others. One of the challenges with the efficient design of a search engine capable of considering semantic information is that it would need to be able to index information beyond the traditional information stored in inverted indices, including entity mentions and type relationships. The objective of our work in this paper is to investigate various ways in which different data structure types can be adopted to integrate three types of information including keywords, entities and types. We will systematically compare the performance of the different data structures for scenarios where (i) the same data structure types are adopted for the three types of information, and (ii) different data structure types are integrated for storing and retrieving the three different information types. We report our findings in terms of the performance of various query processing tasks such as Boolean and ranked intersection for the different indices and discuss which index type would be appropriate under different conditions for semantic search.     

混合P2P环境下有效的查询扩展及其搜索算法

查询扩展是解决信息获取领域中用词歧义性问题的关键技术,并被广泛应用于搜索引擎中,获得了巨大的成功.然而,由于P2P(peer-to-peer)系统是一个分散的、动态的系统,在P2P环境下进行有效的查询扩展具有一定的挑战性.首先,利用查询与文档的关联关系构建了LEM(local expansion method)查询扩展方法;然后,基于查询与文档用词的直接关联,提出了HEM(history_based expansion method)查询扩展方法.在此基础上,提出了一种基于查询扩展的混合P2P环境下的搜索算法.实验及分析结果表明,查询扩展及其搜索算法能够极大地提高搜索的效果.     

基于LDA主题模型的分布式信息检索集合选择方法

该文针对分布式信息检索时不同集合对最终检索结果贡献度有差异的现象,提出一种基于LDA主题模型的集合选择方法。该方法首先使用基于查询的采样方法获取各集合描述信息;其次,通过建立LDA主题模型计算查询与文档的主题相关度;再次,用基于关键词相关度与主题相关度相结合的方法估计查询与样本集中文档的综合相关度,进而估计查询与各集合的相关度;最后,选择相关度最高的M个集合进行检索。实验部分采用R_m、P@n和MAP作为评价指标,对集合选择方法的性能进行了验证。实验结果表明该方法能更准确的定位到包含相关文档多的集合,提高了检索结果的召回率和准确率。