基于文档标引图模型的文本相似度策略

文档标引图是一种基于短语的图结构文本特征表示模型,能更加全面、准确地表达文本特征信息,实现渐增的文本聚类和信息处理。该文基于文档标引图特征模型,提出文档相似度计算加法策略和乘法策略,采用变换函数对文档相似度值进行调整,增强文档之间的可区分性,改进文本聚类和分类等处理的性能,实例证明了策略的有效性。     

A New Retrieval Model Based on TextTiling for Document Similarity Search

Document similarity search is to find documents similar to a given query document and return a ranked list of similar documents to users, which is widely used in many text and web systems, such as digital library, search engine, etc. Traditional retrieval models, including the Okapi's BM25 model and the Smart's vector space model with length normalization, could handle this problem to some extent by taking the query document as a long query. In practice, the Cosine measure is considered as the best model for document similarity search because of its good ability to measure similarity between two documents. In this paper, the quantitative performances of the above models are compared using experiments. Because the Cosine measure is not able to reflect the structural similarity between documents, a new retrieval model based on TextTiling is proposed in the paper. The proposed model takes into account the subtopic structures of documents. It first splits the documents into text segments with TextTiling and calculates the similarities for different pairs of text segments in the documents. Lastly the overall similarity between the documents is returned by combining the similarities of different pairs of text segments with optimal matching method. Experiments are performed and results show: 1) the popular retrieval models (the Okapi's BM25 model and the Smart's vector space model with length normalization) do not perform well for document similarity search; 2) the proposed model based on TextTiling is effective and outperforms other models, including the Cosine measure; 3) the methods for the three components in the proposed model are validated to be appropriately employed.     

基于参考集索引的高效序列相似性查找算法

序列数据在文本、Web访问日志文件、生物数据库中普遍存在,对其进行相似性查找是一种重要的获取和分析知识的手段.基于参考集索引技术是一类解决序列相似性查找的有效方法,主要思想是找到序列数据库中的少数序列作为参考集,通过参考集过滤掉数据库中与查询序列不相关的数据,从而高效地回答查询.在现有基于参考集索引技术的基础上,提出一种过滤能力更强的序列相似性查询算法IRI(improved reference indexing).首先,充分利用了先前的查询结果集来加速当前的查询,其次考虑了基于序列特征的上界和下界,使得应用参考集进行过滤的上下界更紧,过滤能力进一步加强.最后,为了避免候选集中费时的编辑距离计算,则只计算前缀序列间的编辑距离,从而进一步加速算法运行.实验采用真实的DNA序列和蛋白质序列数据,结果表明,算法IRI在查询性能上明显优于现有的基于参考集索引方法RI(reference indexing).     

逐维聚类的相似度索引算法

随着多媒体信息技术的迅速发展,多维度索引技术在图像、视频等可视信息的存储、检索方面成为一个重要的研究领域,针对“维数危机”难题,提出逐维聚类相似度索引算法,该算法根据数据集的分布特性,对特征矢量的每一维进行聚类,算法在实现检索时可以逐步滤除与查询矢量不相似的数据集,缩小检索范围,进而提高了检索速度,实验结果表明,逐维聚类算法适用于基于相似度的高维数据矢量检索和查询,是一种简单、灵活的索引结构。     

一种潜在文档相似模型

提出一种潜在文档相似模型（LDSM）,把每对文档看作一个二分图,把文档的潜在主题看作二分图的顶点,用主题问的加权相似度为相应边赋权值,并用二分图的最佳匹配表示文档的相似度。实验结果表明,LDSM的平均查准率和平均查全率都优于用TextTiling和二分图最佳匹配方法构建的文档相似模型。     

Document retrieval from compressed images

With the emergence of digital libraries, more and more documents are stored and transmitted through the Internet in the format of compressed images. It is of significant meaning to develop a system which is capable of retrieving documents from these compressed document images. Aiming at the popular compression standard-CCITT Group 4 which is widely used for compressing document images, we present an approach to retrieve the documents from CCITT Group 4 compressed document images in this paper. The black and white changing elements are extracted directly from the compressed document images to act as the feature pixels, and the connected components are detected simultaneously. Then the word boxes are bounded based on the merging of the connected components. Weighted Hausdorff distance is proposed to assign all of the word objects from both the query document and the document from database to corresponding classes by an unsupervised classifier, whereas the possible stop words are excluded. Document vectors are built by the occurrence frequency of the word object classes, and the pair-wise similarity of two document images is represented by the scalar product of the document vectors. Nine groups of articles pertaining to different domains are used to test the validity of the presented approach. Preliminary experimental results with the document images captured from students’ theses show that the proposed approach has achieved a promising performance.     

基于二分图的装配体检索研究

基于事例推理是解决DFA和装配工艺规划问题的有效方法,装配体检索是应用此方法的关键步骤．对装配体中的联结关系进行表达和分类,提出联结关系定量的比较方法,基于二分图理论建立了装配体之间的相似计算模型,给出了装配体检索过程和计算实例．     

Near-duplicate document image matching: A graphical perspective

A near-duplicate document image matching approach characterized by a graphical perspective is proposed in this paper. Document images are represented by graphs whose nodes correspond to the objects in the images. Consequently, the image matching problem is then converted to graph matching. To deal with the instability of object segmentation, a multi-granularity object tree is constructed for a document image. Each level in the tree corresponds to one possible object segmentation, while different levels are characterized by various object granularities. Some graphs can be generated from the tree and the objects associated with each graph may be of different granularities. Two graphs with the maximum similarity are found from the multi-granularity object trees of the two near-duplicate document images which are to be matched. The encouraging experimental results have demonstrated the effectiveness of the proposed approach.     

一种融合音位属性的语音文档索引方法

为提高索引覆盖率并获得更多的候选路径,提出一种在词格上融合音位属性的语音文档索引方法.通过基于音位属性检测的语音识别系统建立词格,利用其信息互补性,与传统的词格进行起止节点合并.针对合并后Lattice规模增大的问题,采用基于位置的分段对齐方法对其结构进行压缩.实验结果表明,该方法在提高索引覆盖率和降低最小错误率方面均优于传统的语音文档索引方法,能够有效提高语音检索性能.     

一种基于主题的概率文档相关模型

现有文档关系分析模型难以从主题层次上判别文档相关性.为此,提出了一个基于主题的概率文档相关模型(TPDC).TPDC借助Latent Dirichlet Allocation模型学习文档的主题结构;在计算出主题后验概率和主题相似度的基础上推导出文档后验概率;基于文档后验概率构建文档相关性分析模型.实验结果证明,TPDC模型在文档检索精度和文档压缩程度两方面优于向量空间模型,因而更能胜任实际应用中的文档检索任务.     

分布式环境下基于图的语义相似性方法研究

论文为不同格式的数据源提供了一个统一的概念模型,并定义了各种信息源到概念模型的转换规则。基于该模型,提出了计算模式间精确语义相似性的方法。     

用基于移动均值的索引实现时间序列相似查询

提出了基于移动均值的索引来解决子序列匹配中的"ε-查询"问题:提出并证明了基于移动均值的缩距定理和缩距比关系定理,后者具有很好的"裁减"能力,可以在相似查询时淘汰大部分不符合条件的候选时间序列,从而达到快速相似查找的目的;引入了由Jagadish等人提出的BATON~*-树,并在此基础上适当修改,建立了MABI索引,极大地加快了相似查询过程;最后,在一个股票交易数据集上进行了实验,证明了MABI索引的良好性能.     

基于词共现的文档表示模型

文档表示模型是文本自动处理的基础,是将非结构化的文本数据转化为结构化数据的有效手段。然而,目前通用的空间向量模型(Vector Space Model,VSM)是以单个的词汇为基础的文档表示模型,因其忽略了词间的关联关系,导致文本挖掘的准确率难以得到很大的提升。该文以词共现分析为基础,讨论了文档主题与词的二阶关系之间的潜在联系,进而定义了词共现度及与文档主题相关度的量化计算方法,利用关联规则算法抽取出文档集上的词共现组合,提出了基于词共现组合的文档向量主题表示模型(Co-occurrence Term based Vector Space Model, CTVSM),定义了基于CTVSM的文档相似度。实验表明,CTVSM能够准确反映文档之间的相关关系,比经典的文档向量空间模型(Vector Space Model,VSM)具有更强的主题区分能力。     

结合稀疏表示与拓扑相似性的图像匹配算法

为了解决图像匹配算法中存在的匹配效率低、时间复杂度与计算量高等问题,通过结合稀疏表示和拓扑相似性,提出了一种图像匹配算法.该算法先对图像进行特征检测,计算轮廓相似度,找到待匹配图像中相似的最大轮廓区域,用稀疏编码对轮廓内特征进行稀疏表示,建立稀疏模型,将复杂特征变得单一化,但又不影响特征的分类方式,将相同类别或者相同属...     

基于整句相似性计算的韵律短语预测模型

语音合成系统中，韵律短语的预测对合成语音的自然度有重要影响．为了突破主流的基于决策树预测方法的若干缺陷，提出了基于整句相似性计算的韵律短语预测模型．通过对1000个句子的测试，该方法在可接受的语料手工标注工作量的范围内，超过了传统决策树的方法．     

基于混合语言模型的文档相似性计算模型

为了克服现有文档相似性模型对文档特性拟合的不完全性和缺乏理论根据的弱点,本文在统计语言模型的基础上,提出了一种基于混合语言模型(Mixture Language Model,MLM)文档相似性计算模型。MLM利用统计语言模型描述文档特征,将相关影响因素作为模型的潜在子模型,文档语言模型由各子模型混合构成,从而准确和全面地反映文档特征。由于MLM根据具体应用确定相关影响因素,并以此构建相应文档描述模型,因此具有很强的灵活性和扩展性。在MLM的基础上,本文给出了一个基于文档主题内容相似性的实例,在TREC9数据集上的实验表明MLM优于向量空间模型(VSM)。     

基于跨语言广义向量空间模型的跨语言文档聚类方法

跨语言文档聚类主要是将跨语言文档按照内容或者话题组织为不同的类簇。该文通过采用跨语言词相似度计算将单语广义向量空间模型(Generalized Vector Space Model, GVSM)拓展到跨语言文档表示中,即跨语言广义空间向量模型(Cross-Lingual Generalized Vector Space Model,CLGVSM),并且比较了不同相似度在文档聚类下的性能。同时提出了适用于GVSM的特征选择算法。实验证明,采用SOCPMI词汇相似度度量算法构造GVSM时,跨语言文档聚类的性能优于LSA。     

基于分级匹配的维吾尔语文档相似性计算及剽窃检测方法

针对以维吾尔语书写的文档间的相似性计算及剽窃检测问题,提出了一种基于内容的维吾尔语剽窃检测(U-PD)方法。首先,通过预处理阶段对维吾尔语文本进行分词、删除停止词、提取词干和同义词替换,其中提取词干是基于N-gram 统计模型实现。然后,通过BKDRhash算法计算每个文本块的hash值并构建整个文档的hash指纹信息。最后,根据hash指纹信息,基于RKR-GST匹配算法在文档级、段落级和句子级将文档与文档库进行匹配,获得文档相似度,以此实现剽窃检测。通过在维吾尔语文档中的实验评估表明,提出的方法能够准确检测出剽窃文档,具有可行性和有效性。     

使用倒排索引优化面向组合的语义服务发现

提出为服务库中所有注册服务的输出建立倒排索引,以快速、准确、高效地发现目标服务.即为每个输出维护一个服务列表,用于记录在该服务库中所有能够产生该输出的服务.基于倒排索引机制,提出面向组合的服务发现算法.该方法利用倒排索引的优势,极大地减少了搜索空间,并通过挖掘服务组合提高服务发现的查全率.仿真实验表明,该方法能够在大规模服务库中快速、全面地响应用户请求.     

LSISOM — A Latent Semantic Indexing Approach to Self-Organizing Maps of Document Collections

The Self Organizing Map (SOM) algorithm has been utilized, with much success, in a variety of applications for the automatic organization of full-text document collections. A great advantage of the SOM method is that document collections can be ordered in such a way so that documents with similar content are positioned at nearby locations of the 2-dimensional SOM lattice. The resulting ordered map thus presents a general view of the document collection which helps the exploration of information contained in the whole document space. The most notable example of such an application is the WEBSOM method where the document collection is ordered onto a map by utilizing word category histograms for representing the documents data vectors. In this paper, we introduce the LSISOM method which resembles WEBSOM in the sense that the document maps are generated from word category histograms rather than simple histograms of the words. However, a major difference between the two methods is that in WEBSOM the word category histograms are formed using statistical information of short word contexts whereas in LSISOM these histograms are obtained from the SOM clustering of the Latent Semantic Indexing representation of document terms.