基于路径特征的XML文档结构相似性度量

针对现有XML文档结构相似性度量方法大多不能完全反映XML文档的结构特征,相似度计算结果精确度不高,导致聚类效果不够理想的问题,提出一种改进的相似度计算方法 SSPF(Similarity based on Sequence,Position and Frequency)。该方法充分利用提取的DOM树路径信息,对树路径间序列和位置的相似度计算进行优化,并考虑了路径频率对相似度的影响,使得文档间的相似性更为合理。实验结果表明,SSPF方法具有更加准确的相似度计算结果,聚类的各项评价指标均有所提高。     

新相似性度量在文档模糊聚类中的应用研究

相似文档检索在文档管理中是很重要的,提出一种在大文档集中基于模糊聚类的快速高效的聚类方法,传统方法大都通过词与词之间的比较来检索文档,该方法让文档通过两层结构得出相似度。系统用预定义模糊簇来描述相似文档的特征向量,用这些向量估计相似度,由此得出文档之间的距离,系统应用了新的相似性度量方法,并通过实验证实了其可行性和高效性。     

基于句子相似度的论文抄袭检测模型研究

提出一种基于句子相似度的论文抄袭检测模型。利用局部词频指纹算法对大规模文档进行快速检测,找出疑似抄袭文档。根据最长有序公共子序列算法计算句子间的相似度,并标注抄袭细节,给出抄袭依据。在标准中文数据集SOGOU-T上进行的实验表明,该模型具有较强的局部信息挖掘能力,在一定程度上克服了现有的论文抄袭检测算法精度不高的缺点。     

基于模糊路径匹配的XML文档分类研究

XML是互联网上信息表示和数据交互的重要标准,文档分类是解决从海量信息中获取有效信息的重要方法,提出一种基于模糊路径匹配的XML文档分类方法。首先去除对分类没有影响的信息;然后采用一种混合的XML文档相似性计算方法,将XML文档表示为路径的集合。为了提高效率,删除了文档中重复出现的路径后进行模糊匹配,用匈牙利算法计算出文档间的相似度;最后使用改进的K近邻算法进行文档的分类。使用自动生成及真实的文档集进行实验,结果表明:两组文档分类的正确率均可以达到100%。     

概念与文档的语义相似度计算

将本体作为背景知识引入到概念之间相似度和文档之间相似度的计算中。通过图模型表示本体中概念以及概念之间的语义关系,用来将一个概念和一个文档扩展为一个语义模糊集,并计算模糊集合之间的相似度。文档相似度的计算是在概念相似度计算的基础之上。在概念相似度的计算过程中引入了语义相似度矩阵以及基于共信息理论的模糊相似度方法。     

基于语言无关性语义Kernel学习的短文本分类

针对传统的短文本分类方法大量使用语法标签和词库导致产生语言依赖的问题,提出一种基于语言无关性语义核学习的短文本分类方法。首先,利用短文本的语义信息从文档中提取模式;然后,以三个标注层(词、文档和类别)标注提取出的每个模式;最后,根据三个标注层次计算文档之间的相似度,并根据相似度完成分类。在英语和汉语数据集上的实验验证了该方法的有效性。实验结果表明,相比其他几种核方法,该方法取得了更好的分类性能。     

一种融合本体与粗糙集的文档相似度计算方法

针对向量空间模型VSM中,在计算文档相似度时要求文档标引词必须相互独立这一缺陷,提出融合本体与粗糙集的文档相似度计算方法。在该方法中,不仅可以利用本体对概念关系的准确揭示,计算文档之间的概念相似度,还可以结合粗糙集对相关概念实例的属性重要性进行度量,从而计算属性相似度,克服了传统方法需要依赖人的先验知识这一缺陷,最后综合形成文档语义向量相似度,并通过实验分析证明该方法可以在很大程度上提高文档相似度计算的准确性。     

XML文档相似性的仿真研究

XML文档相似性的计算是XML文档分类中的一个难题。文中描述了一种基于结构的方法，通过序列化模式挖掘方法，挖掘出两个文档之间的最大相似路径，从而可以通过计算最大相似的路径的节点数目和所有路径的节点数目的比值，得到两个文档之间的相似度。文章提出了一种新的最小化XML文档的方法，并且综合考虑了文档节点的语义相似度和结构相似度，从而进一步地提高了计算文档相似度的精度。实验表明，该方法有着良好的应用前景。     

一种基于贝叶斯分类的XML检索文档相似度算法

目前对于查询相似度的计算通常是从比对检索结果与查询式的相似度来考虑。本文提出一种基于贝叶斯分类的算法来计算XML查询结果相似度。在计算出每个检索结果文档与查询式相似度的基础上,使用贝叶斯分类器将XML检索文档分类成相关与不相关两个集合,再由计算相关文档与不相关文档的相似度来决定最终的相似度值。最后,通过实验分析表明,在不影响查全率的前提下,这样得到的相似度计算精度比传统方法高15%左右,有效地提高了检索性能。     

一种基于最大权匹配计算的信息检索方法

基于《知网》的词汇语义计算方法,提出了一种基于向量空间模型的文本信息检索新方法。方法的基本技术思想是通过计算关键词的语义相似度,并采用最大权匹配方法来计算查询向量和文本向量的相似度,作为相关文本的检索依据。该方法基于全局最优,使文本和查询向量中各词条的相似度总和最大,从而可以从整体上提高文本信息检索的准确率。论文还通过原型实验对该方法的有效性进行了验证。     

基于双语LDA的跨语言文本相似度计算方法研究

基于双语主题模型思想分析双语文本相似性,提出基于双语LDA跨语言文本相似度计算方法。先利用双语平行语料集训练双语LDA模型,再利用该模型预测新语料集主题分布,将新语料集的双语文档映射到同一个主题向量空间,结合主题分布使用余弦相似度方法计算新语料集双语文档的相似度,使用从类别间和类别内的主题分布离散度的角度改进的主题频率-逆文档频率方法计算特征主题权重。实验表明,改进后的权重计算对于基于双语LDA相似度算法的召回率有较大提高,算法对类别不受限且有较好的可靠性。     

A New Retrieval Model Based on TextTiling for Document Similarity Search

Document similarity search is to find documents similar to a given query document and return a ranked list of similar documents to users, which is widely used in many text and web systems, such as digital library, search engine, etc. Traditional retrieval models, including the Okapi's BM25 model and the Smart's vector space model with length normalization, could handle this problem to some extent by taking the query document as a long query. In practice, the Cosine measure is considered as the best model for document similarity search because of its good ability to measure similarity between two documents. In this paper, the quantitative performances of the above models are compared using experiments. Because the Cosine measure is not able to reflect the structural similarity between documents, a new retrieval model based on TextTiling is proposed in the paper. The proposed model takes into account the subtopic structures of documents. It first splits the documents into text segments with TextTiling and calculates the similarities for different pairs of text segments in the documents. Lastly the overall similarity between the documents is returned by combining the similarities of different pairs of text segments with optimal matching method. Experiments are performed and results show: 1) the popular retrieval models (the Okapi's BM25 model and the Smart's vector space model with length normalization) do not perform well for document similarity search; 2) the proposed model based on TextTiling is effective and outperforms other models, including the Cosine measure; 3) the methods for the three components in the proposed model are validated to be appropriately employed.     

一种基于文档拓扑的相似性搜索算法

从海量文档中快速有效地搜索到相似文档是一个重要且耗时的问题。现有的文档相似性搜索算法是先找出候选文档集,再对候选文档进行相关性排序,找出最相关的文档。提出了一种基于文档拓扑的相似性搜索算法——Hub-N,将文档相似性搜索问题转化为图搜索问题,应用相应的剪枝技术,缩小了扫描文档的范围,提高了搜索效率。通过实验验证了算法的有效性和可行性。     

Document Similarity Using a Phrase Indexing Graph Model

Document clustering techniques mostly rely on single term analysis of text, such as the vector space model. To better capture the structure of documents, the underlying data model should be able to represent the phrases in the document as well as single terms. We present a novel data model, the Document Index Graph, which indexes Web documents based on phrases rather than on single terms only. The semistructured Web documents help in identifying potential phrases that when matched with other documents indicate strong similarity between the documents. The Document Index Graph captures this information, and finding significant matching phrases between documents becomes easy and efficient with such model. The model is flexible in that it could revert to a compact representation of the vector space model if we choose not to index phrases. However, using phrase indexing yields more accurate document similarity calculations. The similarity between documents is based on both single term weights and matching phrase weights. The combined similarities are used with standard document clustering techniques to test their effect on the clustering quality. Experimental results show that our phrase-based similarity, combined with single-term similarity measures, gives a more accurate measure of document similarity and thus significantly enhances Web document clustering quality.     

用带权重的pq-gram算法计算XML文档相似度

XML文档聚类是高效管理XML文档的重要手段,XML文档相似度计算正是其中的关键步骤。pq-gram算法是解决XML文档相似度计算问题的有效手段,但忽略了XML文档结点的有序性。带权重的pq-gram算法是在此基础上,依据XML文档的结构性,首先为结点赋予相应权重,然后基于结点的权重对pq-gram赋予权重,最后将设定的权重应用到XML文档相似度计算中。实验结果表明,带权重的pq-gram算法更好地描述结点在XML文档相似度计算中的贡献度,提高了XML文档相似度计算的精度。     

裁判文书类案推送中的案情相似度计算模型研究

裁判文书的类案推送策略有助于解决司法过程中的裁判尺度不统一、类案不同判、量刑不规范等问题。针对裁判文书类案推送任务,基于裁判文书在篇章结构和语言表述方面的特征,从裁判文书案情内容的抽取、案情内容中不同词性类别词项的权重分析、案情内容中未登录词的识别、案情内容中数量表述的相似度计算等角度展开研究,并设计相应的案情相似度计算模型。通过在真实裁判文书数据集合上的实验,表明了该模型的有效性。     

基于语义分析的OWL文档聚类研究

借助目前丰富的网络资源，将同一主题的现存Ontology知识聚类，提供给领域专家或用户进行二次精化和集成是Ontology研究领域的一个重要课题．OWL是目前用于表示和交换Ontology信息的基本标准．本文从OWL的语义本质出发，考虑了知识之间的继承性及复杂类比较和模糊集运算的相似性，提出一种计算OWL文档语义相似性的方式，并和层次聚类算法集成完成了对OWL文档集的聚类实验．实验结果说明本文提出的算法对自动生成和手工建立的OWL文档集都有很好的效果。     

On the use of hierarchical information in sequential mining-based XML document similarity computation

Measuring the structural similarity among XML documents is the task of finding their semantic correspondence and is fundamental to many web-based applications. While there exist several methods to address the problem, the data mining approach seems to be a novel, interesting and promising one. It explores the idea of extracting paths from XML documents, encoding them as sequences and finding the maximal frequent sequences using the sequential pattern mining algorithms. In view of the deficiencies encountered by ignoring the hierarchical information in encoding the paths for mining, a new sequential pattern mining scheme for XML document similarity computation is proposed in this paper. It makes use of a preorder tree representation (PTR) to encode the XML trees paths so that both the semantics of the elements and the hierarchical structure of the document can be taken into account when computing the structural similarity among documents. In addition, it proposes a postprocessing step to reuse the mined patterns to estimate the similarity of unmatched elements so that another metric to qualify the similarity between XML documents can be introduced. Encouraging experimental results were obtained and reported.     

Text Retrieval from Document Images Based on Word Shape Analysis

In this paper, we propose a method of text retrieval from document images using a similarity measure based on word shape analysis. We directly extract image features instead of using optical character recognition. Document images are segmented into word units and then features called vertical bar patterns are extracted from these word units through local extrema points detection. All vertical bar patterns are used to build document vectors. Lastly, we obtain the pair-wise similarity of document images by means of the scalar product of the document vectors. Four corpora of news articles were used to test the validity of our method. During the test, the similarity of document images using this method was compared with the result of ASCII version of those documents based on the N-gram algorithm for text documents.     

A multi-level matching method with hybrid similarity for document retrieval

This paper presents a multi-level matching method for document retrieval (DR) using a hybrid document similarity. Documents are represented by multi-level structure including document level and paragraph level. This multi-level-structured representation is designed to model underlying semantics in a more flexible and accurate way that the conventional flat term histograms find it hard to cope with. The matching between documents is then transformed into an optimization problem with Earth Mover’s Distance (EMD). A hybrid similarity is used to synthesize the global and local semantics in documents to improve the retrieval accuracy. In this paper, we have performed extensive experimental study and verification. The results suggest that the proposed method works well for lengthy documents with evident spatial distributions of terms.