位置加权文本聚类算法

文本聚类是自然语言处理研究中一项重要研究课题,文本聚类技术广泛地应用于信息检索、Web挖掘和数字图书馆等领域。本文针对特征词在文档中的不同位置对文档的贡献大小不同,提出了基于特征词的位置加权文本聚类改进算法——TCABPW。通过选取反映文档主题的前L个高权值的特征项构造新的文本特征向量,采用层次聚类和K-means文本聚类相结合的改进算法实现文本聚类。实验结果表明,提出的改进算法在不影响聚类质量的情况下大大地降低了文本聚类的维度,在稳定性和纯度上都有显著提高,获得了较好的聚类效果。     

基于主题的Web文本聚类方法

针对传统Web文本聚类算法没有考虑Web文本主题信息导致对多主题Web文本聚类结果准确率不高的问题,提出基于主题的Web文本聚类方法。该方法通过主题提取、特征抽取、文本聚类三个步骤实现对多主题Web文本的聚类。相对于传统的Web文本聚类算法,所提方法充分考虑了Web文本的主题信息。实验结果表明,对多主题Web文本聚类,所提方法的准确率比基于K-means的文本聚类方法和基于《知网》的文本聚类方法要好。     

文本聚类算法的设计与实现

为了有效地提高丈本聚类的质量和效率,在对已有的层次聚类和K-means算法分析和研究的基础上,针对互联网信息处理量大、实时性高的特点,设计并实现了一种用于高维稀疏相似矩阵的文本聚类算法.该算法结合了层次聚类和K-means聚类的思想,根据一个阈值来控制聚类算法的选取和新簇的建立,并通过文本特征提取和文档相似度矩阵计算实现文本聚类.实验结果表明,该算法的召回率和正确率更高.     

结合语义改进的K-means短文本聚类算法

针对短文本聚类存在的三个主要挑战,特征关键词的稀疏性、高维空间处理的复杂性和簇的可理解性,提出了一种结合语义改进的K-means短文本聚类算法。该算法通过词语集合表示短文本,缓解了短文本特征关键词的稀疏性问题;通过挖掘短文本集的最大频繁词集获取初始聚类中心,有效克服了K-means聚类算法对初始聚类中心敏感的缺点,解决了簇的理解性问题;通过结合TF-IDF值的语义相似度计算文档之间的相似度,避免了高维空间的运算。实验结果表明,从语义角度出发实现的短文本聚类算法优于传统的短文本聚类算法。     

一种基于语义特征的Web文档检索方法

Web文档聚类在Web信息检索中起着重要的作用。文中提出了一种新的Web文档聚类和检索算法。该算法采用有序聚类的方法,根据Web文档的物理结构概括其语义段落和提取相应的语义特征,并以此作为文档检索的基础;在此基础上,根据用户的检索要求直接在文档的语义段落层次计算其相似性,大大提高了检索的精度和效率。实验结果表明,文中提出的算法具有一定的实用性。     

基于文档特征的Web主题文本提取

Web主题文本提取是从Web页面中找出文本型主题内容,对Web信息过滤具有重大作用。针对目前Web主题文本提取算法复杂而且响应速度较慢的不足,提出一种新的Web主题文本提取方法。该方法直接从HTML文档结构本身的特征出发,提取文档中文本的多个特征．并设计了一个有效的公式综合利用提取特征来定位主题文本。实验表明该算法简单、快速、有效,能很好地运用于Web信息过滤。     

K-means聚类算法的研究和应用

介绍了K-means算法的思想,分析了在文档聚类中运用K-means算法的步骤。以开源的机器学习软件Weka为平台,详细论述在Weka上进行文档聚类的前端处理过程,利用搜狗语料库中的文档在Weka上进行了Kmeans算法的聚类测试。实验结果表明,K-means算法在Web文档聚类中表现出较好的效果。根据实验结果,分析了K-means算法存在的不足和聚类分析中特征选择的重要性。     

改进的K-means 算法在网络舆情分析中的应用

结合网络舆情分析的应用需求背景,首先介绍了文本信息的处理,然后探讨了文本聚类中的K-means算法,针对其对初始聚类中心的依赖性的特点,对算法加以改进。基于文档标题能够代表文档内容的思想,改进算法采用稀疏特征向量表示文本标题,计算标题间的稀疏相似度,确定初始聚类中心。最后实验证明改进的K-means算法提高了聚类的准确度;与基于最大最小距离原则的初始中心选择算法比较,提高了执行效率,同时保证了聚类准确度。     

一种结合GAAC和K-means的维吾尔文文本聚类算法

介绍了K-means和GAAC聚类算法思想和两种特征提取方法对维吾尔文文本表示及聚类效率的影响.在较大规模文本语料库基础上,分别用K-means和GAAC的方法进行维吾尔文文本聚类实验及性能对比分析,针对经典K-means算法对初始聚类中心的过分依赖性及不稳定性缺点以及GAAC的高计算复杂性,提出了一种结合GACC和K-means的维吾尔文聚类算法.本算法分两步完成聚类操作,首先是GAAC模块从少量文本集中获取最优的初始类中心,然后是K-means模块对大量文本集进行快速聚类.实验结果表明,新算法在聚类准确率和时间复杂度上都有了显著的提高.     

基于边界距离的多向量文本聚类方法

文本聚类是自然语言处理中的一项重要研究课题,主要应用于信息检索和Web挖掘等领域。其中的关键是文本的表示和聚类算法。在层次聚类的基础上,提出了一种新的基于边界距离的层次聚类算法,该方法通过选择两个类间边缘样本点的距离作为类间距离,有效地利用类的边界信息,提高类间距离计算的准确性。综合考虑不同词性特征对文本的贡献,采用多向量模型对文本进行表示。不同文本集上的实验表明,基于边界距离的多向量文本聚类算法取得了较好的性能。     

Visual Similarity Based Document Layout Analysis

In this paper, a visual similarity based document layout analysis （DLA） scheme is proposed, which by using clustering strategy can adaptively deal with documents in different languages, with different layout structures and skew angles. Aiming at a robust and adaptive DLA approach, the authors first manage to find a set of representative filters and statistics to characterize typical texture patterns in document images, which is through a visual similarity testing process. Texture features are then extracted from these filters and passed into a dynamic clustering procedure, which is called visual similarity clustering. Finally, text contents are located from the clustered results. Benefit from this scheme, the algorithm demonstrates strong robustness and adaptability in a wide variety of documents, which previous traditional DLA approaches do not possess.     

一种基于LDA的潜在语义区划分及Web文档聚类算法

该文应用LDA模型进行文档的潜在语义分析,将语义分布划分成低频、中频、高频语义区,以低频语义区的语义进行Web游离文档检测,以中、高频语义区的语义作为文档特征进行文档聚类,采用文档类别与语义互作用机制对聚类结果进行修正。与相关工作比较,该文不仅应用LDA模型表示文档,而且进行了深入的语义分布区域划分,并将分析结果应用于Web文档聚类。实验表明,该文提出的基于LDA的文档类别与语义互作用聚类算法获得了更好的聚类结果。     

Narrowing the semantic gap - improved text-based web document retrieval using visual features

We present the results of our work that seek to negotiate the gap between low-level features and high-level concepts in the domain of web document retrieval. This work concerns a technique, called the latent semantic indexing (LSI), which has been used for textual information retrieval for many years. In this environment, LSI determines clusters of co-occurring keywords so that a query which uses a particular keyword can then retrieve documents perhaps not containing this keyword, but containing other keywords from the same cluster. In this paper, we examine the use of this technique for content-based web document retrieval, using both keywords and image features to represent the documents. Two different approaches to image feature representation, namely, color histograms and color anglograms, are adopted and evaluated. Experimental results show that LSI, together with both textual and visual features, is able to extract the underlying semantic structure of web documents, thus helping to improve the retrieval performance significantly, even when querying is done using only keywords.     

一种基于群体智能的Web文档聚类算法

将群体智能聚类模型运用于文档聚类，提出了一种基于群体智能的Web文档聚类算法，首先运用向量空间模型表示Web文档信息，采用常规方法如消除无用词和特征词条约简法则得到文本特征集，然后将文档的向量随机分布到一个平面上，运用基于群体智能的聚类方法进行文档聚类，最后从平面上采用递归算法收集聚类结果，为了改善算法的实用性，将原算法与k均值算法结合提出一种混合聚类算法，通过实验比较，结果表明基于群体智能的Web文档聚类算法具有较好的聚类特性，它能将与一个主题相关的Web文档较完全而准确地聚成一类。     

Design and Development of Density-Based Effective Document Clustering Method Using Ontology

Text document clustering is used to separate a collection of documents into several clusters by allowing the documents in a cluster to be substantially similar. The documents in one cluster are distinct from documents in other clusters. The high-dimensional sparse document term matrix reduces the clustering process efficiency. This study proposes a new way of clustering documents using domain ontology and WordNet ontology. The main objective of this work is to increase cluster output quality. This work aims to investigate and examine the method of selecting feature dimensions to minimize the features of the document name matrix. The sports documents are clustered using conventional K-Means with the dimension reduction features selection process and density-based clustering. A novel approach named ontology-based document clustering is proposed for grouping the text documents. Three critical steps were used in order to develop this technique. The initial step for an ontology-based clustering approach starts with data pre-processing, and the characteristics of the DR method are reduced with the Info-Gain collection. The documents are clustered using two clustering methods: K-Means and Density-Based clustering with DR Feature Selection Process. These methods validate the findings of ontology-based clustering, and this study compared them using the measurement metrics. The second step of this study examines the sports field ontology development and describes the principles and relationship of the terms using sports-related documents. The semantic web rational process is used to test the ontology for validation purposes. An algorithm for the synonym retrieval of the sports domain ontology terms has been proposed and implemented. The retrieved terms from the documents and sport ontology concepts are mapped to the retrieved synonym set words from the WorldNet ontology. The suggested technique is based on synonyms of mapped concepts. The proposed ontology approach employs the reduced feature set in order to clustering the text documents. The results are compared with two traditional approaches on two datasets. The proposed ontology-based clustering approach is found to be effective in clustering the documents with high precision, recall, and accuracy. In addition, this study also compared the different RDF serialization formats for sports ontology.

     

利用IRF模型丰富文档的语义代表

文本挖掘是发现文本中所包含的内容和意义的过程。向量空间模型是文本挖掘中成熟的文本表示模型,而特征项的选择对其性能有着重要的影响。但以前的研究都把目光聚焦于文本中出现的特征项,忽略了文档之间的相关性。这种局限使这些特征项不能提供丰富的语义信息。始于2005年的Web2.0大潮席卷了整个互联网,在此背景下应运而生的社会化标注成了相关文档的语义桥梁,此文本挖掘带来了新的生机。据此本文利用IRF（Iterative Reinforcement Framwork）模型为文档产生了丰富的特征项,大大提高了文档的检索率。     

基于跨语言广义向量空间模型的跨语言文档聚类方法

跨语言文档聚类主要是将跨语言文档按照内容或者话题组织为不同的类簇。该文通过采用跨语言词相似度计算将单语广义向量空间模型(Generalized Vector Space Model, GVSM)拓展到跨语言文档表示中,即跨语言广义空间向量模型(Cross-Lingual Generalized Vector Space Model,CLGVSM),并且比较了不同相似度在文档聚类下的性能。同时提出了适用于GVSM的特征选择算法。实验证明,采用SOCPMI词汇相似度度量算法构造GVSM时,跨语言文档聚类的性能优于LSA。     

Distributed Web Log Mining Using Maximal Large Itemsets

We introduce a partitioning-based distributed document-clustering algorithm using user access patterns from multi-server web sites. Our algorithm makes it possible to exploit simultaneously adaptive document replication and persistent connections, two techniques that are most effective in decreasing the response time that is observed by web users. The algorithm first distributes the user access data evenly among the servers by using a hash function. Then, each server generates a local clustering on its fair share of the user sessions records by employing a traditional single-machine document-clustering algorithm. Finally, those local clustering results are combined together by using a novel procedure that generates maximal large itemsets of web documents. We present preliminary experimental results and discuss alternative approaches to be pursued in the future. Received 30 August 2000 / Revised 30 January 2001 / Accepted in revised form 9 May 2001