A new document representation using term frequency and vectorized graph connectionists with application to document retrieval

This paper presents a new document representation with vectorized multiple features including term frequency and term-connection-frequency. A document is represented by undirected and directed graph, respectively. Then terms and vectorized graph connectionists are extracted from the graphs by employing several feature extraction methods. This hybrid document feature representation more accurately reflects the underlying semantics that are difficult to achieve from the currently used term histograms, and it facilitates the matching of complex graph. In application level, we develop a document retrieval system based on self-organizing map (SOM) to speed up the retrieval process. We perform extensive experimental verification, and the results suggest that the proposed method is computationally efficient and accurate for document retrieval.     

P2P网络中基于增量学习的节点聚类

本文研究了p2p网络中基于内容的节点聚类。基于文件名关键词精确匹配的查询没有考虑文本语义及内容相似性。如果能够根据节点发布内容的相似性,建立节点聚类,信息查询在类内进行,必将提高查询效率。本文提出了一种基于增量学习的节点聚类方法,通过兴趣爬虫代理计算节点得分,据此判断一个节点是否可以加入节点簇。实验表明,节点簇的建立可以有效地提高 p2p 网络的查询效率。     

基于多层次属性建模的云制造服务优选方法

随着云计算和信息技术的发展,制造企业逐渐由生产型向服务型转化.为了满足用户需求、解决云制造服务优选问题,提出一种基于多层次属性建模的云制造服务匹配和优选方法.对云制造服务资源和属性进行详细描述和划分,构建多层次属性描述模型.从基本属性、功能属性、非功能属性、综合匹配四个层次对候选服务和请求服务进行匹配计算.对不同类型的...     

A New Retrieval Model Based on TextTiling for Document Similarity Search

Document similarity search is to find documents similar to a given query document and return a ranked list of similar documents to users, which is widely used in many text and web systems, such as digital library, search engine, etc. Traditional retrieval models, including the Okapi's BM25 model and the Smart's vector space model with length normalization, could handle this problem to some extent by taking the query document as a long query. In practice, the Cosine measure is considered as the best model for document similarity search because of its good ability to measure similarity between two documents. In this paper, the quantitative performances of the above models are compared using experiments. Because the Cosine measure is not able to reflect the structural similarity between documents, a new retrieval model based on TextTiling is proposed in the paper. The proposed model takes into account the subtopic structures of documents. It first splits the documents into text segments with TextTiling and calculates the similarities for different pairs of text segments in the documents. Lastly the overall similarity between the documents is returned by combining the similarities of different pairs of text segments with optimal matching method. Experiments are performed and results show: 1) the popular retrieval models (the Okapi's BM25 model and the Smart's vector space model with length normalization) do not perform well for document similarity search; 2) the proposed model based on TextTiling is effective and outperforms other models, including the Cosine measure; 3) the methods for the three components in the proposed model are validated to be appropriately employed.     

Beyond topical similarity: a structural similarity measure for retrieving highly similar documents

Accurately measuring document similarity is important for many text applications, e.g. document similarity search, document recommendation, etc. Most traditional similarity measures are based only on “bag of words” of documents and can well evaluate document topical similarity. In this paper, we propose the notion of document structural similarity, which is expected to further evaluate document similarity by comparing document subtopic structures. Three related factors (i.e. the optimal matching factor, the text order factor and the disturbing factor) are proposed and combined to evaluate document structural similarity, among which the optimal matching factor plays the key role and the other two factors rely on its results. The experimental results demonstrate the high performance of the optimal matching factor for evaluating document topical similarity, which is as well as or better than most popular measures. The user study shows the good ability of the proposed overall measure with all three factors to further find highly similar documents from those topically similar documents, which is much better than that of the popular measures and other baseline structural similarity measures. Xiaojun Wan received a B.Sc. degree in information science, a M.Sc. degree in computer science and a Ph.D. degree in computer science from Peking University, Beijing, China, in 2000, 2003 and 2006, respectively. He is currently a lecturer at Institute of Computer Science and Technology of Peking University. His research interests include information retrieval and natural language processing.     

基于语义和结构的XML文档相似度的计算方法

个性化信息服务通过了解用户的兴趣爱好,为不同的用户提供不同的信息服务。XML是一种标示语言,是Web文档表示和交换的常用相关标准,因此XML文档之间相似度计算问题对于个性化推荐与信息检索非常重要,为此提出了一个计算XML文档之间的语义和结构相似度的方法XMLSim。首先,基于节点标记对之间的语义相似度和编辑距离计算节点标记对之间的相似度;在分析了路径上节点具有的偏序关系之后,将路径之间相似度问题抽象为最大相似子序列(MSS,Maximal Similar Subsequence)问题,并利用动态规划对MSS问题求解得到路径相似度NpathSim。最后,XML文档之间的相似度XMLSim通过路径集合之间的最大NPathSim的平均值得到。     

Information retrieval in document image databases

With the rising popularity and importance of document images as an information source, information retrieval in document image databases has become a growing and challenging problem. In this paper, we propose an approach with the capability of matching partial word images to address two issues in document image retrieval: word spotting and similarity measurement between documents. First, each word image is represented by a primitive string. Then, an inexact string matching technique is utilized to measure the similarity between the two primitive strings generated from two word images. Based on the similarity, we can estimate how a word image is relevant to the other and, thereby, decide whether one is a portion of the other. To deal with various character fonts, we use a primitive string which is tolerant to serif and font differences to represent a word image. Using this technique of inexact string matching, our method is able to successfully handle the problem of heavily touching characters. Experimental results on a variety of document image databases confirm the feasibility, validity, and efficiency of our proposed approach in document image retrieval.     

XML文档语义检索方法研究

由于半结构文档如XML越来越广泛的应用,在数据库和信息检索领域,对半结构XML数据相似度的研究也变得尤为重要。给定XML文档集D和用户查询q,XML检索即是从D中查找出符合q的XML文档。为了有效地进行XML信息检索,提出了一种新的计算用户查询与XML文档之间相似度的算法。该算法分为三步：基于WordNet对用户查询q进行同义词扩展得到q';将q'和D中的每一篇XML文档都进行数字签名,并通过签名之间的匹配对D进行有效过滤,除去大量不符合用户查询的文档,得到一个文档子集D',[D'?D];对q'与D'中的文档进行精确匹配得到检索结果。     

基于层级匹配的维吾尔文关键词文档图像检索

为实现基于关键词的维吾尔文文档图像检索,提出一种基于由粗到细层级匹配的关键词文档图像检索方法。使用改进的投影切分法将经过预处理的文档图像切分成单词图像库,使用模板匹配对关键词进行粗匹配;在粗匹配的基础上,提取单词图像的方向梯度直方图(HOG)特征向量;通过支持向量机(SVM)分类器学习特征向量,实现关键词图像检索。在包含108张文档图像的数据库中进行实验,实验结果表明,检索准确率平均值为91.14%,召回率平均值为79.31%,该方法能有效实现基于关键词的维吾尔文文档图像检索。     

一种通过内容和结构查询文档数据库的方法

文档是有一定逻辑结构的,标题、章节、段落等这些概念是文档的内在逻辑.不同的用户对文档的检索,有不同的需求,检索系统如何提供有意义的信息,一直是研究的中心任务.结合文档的结构和内容,对结构化文件的检索,提出了一种新的计算相似度的方法.这种方法可以提供多粒度的文档内容的检索,包括从单词、短语到段落或者章节.基于这种方法实现了一个问题回答系统,测试集是微软的百科全书Encarta,通过与传统方法实验比较,证明通过这种方法检索的文章片断更合理、更有效.     

基于相似度的网页标题抽取方法

目前网页标题的抽取方法大多结合HTML结构和标签特征进行抽取,但是这些方法并没有考虑标题与正文信息之间内容上的联系。该文提出一种基于相似度的网页标题抽取方法,该方法利用网页标题与正文信息之间的关系,通过计算语言“单位”之间的相似度和对应的权值,并引入HITS算法模型对权值进行调整,根据特定的选取方法抽取出真实标题。实验结果表明,该方法不仅对“非标准网页”的抽取达到满意的效果,而且对“标准网页”具有较高的泛化能力。     

一种基于贝叶斯分类的XML检索文档相似度算法

目前对于查询相似度的计算通常是从比对检索结果与查询式的相似度来考虑。本文提出一种基于贝叶斯分类的算法来计算XML查询结果相似度。在计算出每个检索结果文档与查询式相似度的基础上,使用贝叶斯分类器将XML检索文档分类成相关与不相关两个集合,再由计算相关文档与不相关文档的相似度来决定最终的相似度值。最后,通过实验分析表明,在不影响查全率的前提下,这样得到的相似度计算精度比传统方法高15%左右,有效地提高了检索性能。     

基于XML的检索结果聚类方法

针对XML文档的半结构化特点,提出一种建模XML检索结果片段的新思路,设计综合内容和结构语义信息度量相应文档相似性的方法,给出一种适应检索结果聚类应用需求的动态均值软聚类算法。实验表明,面向XML的检索结果聚类方法聚类效果优于传统方法。     

云计算下支持语义的可搜索加密方法研究

传统的云计算下的可搜索加密算法没有对查询关键词进行语义扩展,导致了用户查询意图与返回结果存在语义偏差,并且对检索结果的相关度排序不够合理,无法满足用户对智能搜索的需求。对此,提出了一种支持语义的可搜索加密方法。该方法利用本体知识库实现了用户查询的语义拓展,并通过语义相似度来控制扩展词的个数,防止因拓展词过多影响检索的精确度。同时,该方法利用文档向量、查询向量分块技术构造出对应的标记向量,以过滤无关文档,并在查询-文档的相似度得分中引入了语义相似度、关键词位置加权评分及关键词-文档相关度等影响因子,实现了检索结果的有效排序。实验结果表明,该方法在提高检索效率的基础上显著改善了检索结果的排序效果,提高了用户满意度。     

语义分析与词频统计相结合的中文文本相似度量方法研究*

基于统计的文本相似度量方法大多先采用TF-IDF方法将文本表示为词频向量,然后利用余弦计算文本之间的相似度。此类方法由于忽略文本中词项的语义信息,不能很好地反映文本之间的相似度。基于语义的方法虽然能够较好地弥补这一缺陷,但需要知识库来构建词语之间的语义关系。研究了以上两类文本相似度计算方法的优缺点,提出了一种新颖的文本相似度量方法,该方法首先对文本进行预处理,然后挑选TF-IDF值较高的词项作为特征项,再借助HowNet语义词典和TF-IDF方法对特征项进行语义分析和词频统计相结合的文本相似度计算,最后利用文本相似度在基准文本数据集合上进行聚类实验。实验结果表明,采用提出的方法得到的F-度量值明显优于只采用TF-IDF方法或词语语义的方法,从而证明了提出的文本相似度计算方法的有效性。     

改进SURF特征的维吾尔文复杂文档图像匹配检索

针对图像局部特征的词袋模型（Bag-of-Word,BOW）检索研究中聚类中心的不确定性和计算复杂性问题,提出一种由不同种类的距离进行相似程度测量的检索和由匹配点数来检索的方法。这种方法首先需要改进文档图像的SURF特征,有效降低特征提取复杂度;其次,对FAST+SURF特征实现FLANN双向匹配与KD-Tree+BBF匹配,在不同变换条件下验证特征鲁棒性;最后,基于这两种检索方法对已收集整理好的各类维吾尔文文档图像数据库进行检索。实验结果表明:基于距离的相似性度量复杂度次于基于匹配数目的检索,而且两种检索策略都能满足快速、精确查找需求。     

一种中文文档图像中检索方法研究

如何对急速增长的文档图像进行有效检索是文档图像管理系统的关键技术之一。提出了一种不需要识别文字的检索中文文档图像的方法,该方法在字符分割基础上采用基于粗外围特征粗匹配和基于改进Hausdorff距离相似度测量的两级匹配方法,以适应于时间、准确性的不同要求。同时用对200幅文档图像样本进行了实验,其结果表明,使用该方法对检索印刷体汉字的文档图像具有较高的检索效果,对于数字图书馆中文档图像检索系统的设计,有一定的参考价值。     

Tuning the matching function for a threshold weighting semantics in a linguistic information retrieval system

Information retrieval is an activity that attempts to produce documents that better fulfill user information needs. To achieve this activity an information retrieval system uses matching functions that specify the degree of relevance of a document with respect to a user query. Assuming linguistic‐weighted queries we present a new linguistic matching function for a threshold weighting semantics that is defined using a 2‐tuple fuzzy linguistic approach (Herrera F, Martínez L. IEEE Trans Fuzzy Syst 2000;8:746–752). This new 2‐tuple linguistic matching function can be interpreted as a tuning of that defined in “Modelling the Retrieval Process for an Information Retrieval System Using an Ordinal Fuzzy Linguistic Approach” (Herrera‐Viedma E. J Am Soc Inform Sci Technol 2001;52:460–475). We show that it simplifies the processes of computing in the retrieval activity, avoids the loss of precision in final results, and, consequently, can help to improve the users' satisfaction. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 921–937, 2005.