一种基于约束的中垂面相似度准则

在数据挖掘和机器学习的基于距离的各种技术中,例如基于距离的聚类和基于距离的分类,如何度量数据间的相似性已经成为一项基础任务.对于某一具体问题,采用合适的相似性度量,会使问题得到更有效的解决.越来越多的研究表明,通过对成对约束(正约束和负约束)的充分利用,从而得到与问题相匹配的相似性度量,能够大幅度地提升算法性能.目前基于约束的相似性度量研究主要是基于约束的距离度量学习,通过对约束信息的利用,学习一个距离度量矩阵,然后再进行分类或者聚类.通过对成对约束尤其是负约束的挖掘,提出一种基于成对约束的相似性度量准则,然后将此准则应用于聚类和分类任务中,分别提出聚类和分类算法,最后在大量标准数据集上将这些算法的性能与目前流行的算法进行实验比较,并据此得出了一些经验性的启示.     

基于混合距离学习的鲁棒的模糊C均值聚类算法

距离度量对模糊聚类算法FCM的聚类结果有关键性的影响。实际应用中存在这样一种场景,聚类的数据集中存在着一定量的带标签的成对约束集合的辅助信息。为了充分利用这些辅助信息,首先提出了一种基于混合距离学习方法,它能利用这样的辅助信息来学习出数据集合的距离度量公式。然后,提出了一种基于混合距离学习的鲁棒的模糊C均值聚类算法（HR-FCM算法）,它是一种半监督的聚类算法。算法HR-FCM既保留了GIFP-FCM（Generalized FCM algorithm with improved fuzzy partitions）算法的鲁棒性等性能,也因为所采用更为合适的距离度量而具有更好的聚类性能。实验结果证明了所提算法的有效性。     

不规则时间序列距离的度量

针对时间序列距离度量的算法很多,但没有适用于不规则时序距离度量算法的现状,基于寻求全局序列点构成的边集之间的距离路径最小的思想,提出一种不规则时序距离度量的算法,并给出了事件序列生成算法和不规则时序距离度量算法的实现,最后利用UCI KDD的时间序列测试数据对算法进行了测试。测试结果证明了该不规则时序距离算法能够有效度量不规则时序的相似性。     

基于谱图和成对约束的主动半监督聚类算法

针对半监督聚类学习算法中缺乏主动学习的缺陷,提出一种纠错式主动学习成对约束方法.算法通过寻找一般聚类算法自身难以发现的成对约束信息,同时避免这部分约束信息之间本身的关系,将其引入谱聚类算法,利用该监督信息调整谱聚类中点与点之间的距离矩阵对两点间距离进行排序,采用双向寻找的方法,使得学习器即使接收到没有标记的数据也能进行主动学习.实验分析表明,所提出算法能够获得较为满意的聚类效果.     

基于成对约束的非线性维数约减框架

半监督维数约简是指借助于辅助信息与大量无标记样本信息从高维数据空间找到一个最优低维判别空间,便于后续的分类或聚类操作,它被看作是理解基因序列、文本与人脸图像等高维数据的有效方法。提出一个基于成对约束的半监督维数约简一般框架（SSPC）。该方法首先通过使用成对约束和无标号样本的内在几何结构学习一个判别邻接矩阵;其次,新方法应用学到的投影将原来高维空间中的数据映射到低维空间中,以至于聚类内的样本之间距离变得更加紧凑,而不同聚类间的样本之间距离变得尽可能得远。所提出的算法不仅能找到一个最佳的线性判别子空间,还可以揭示流形数据的非线性结构。在一些真实数据集上的实验结果表明,新方法的性能优于当前主流基于成对约束的维数约简算法的性能。     

基于DLT-Kmedoids算法的用电负荷聚类分析

针对高校用电负荷中传统聚类算法直接应用于时间序列聚类效果准确性较低的问题,提出一种融合DTW距离、LB＿Keogh距离以及时间窗口的DLT-Kmedoids算法,以提高聚类算法应用于时间序列的准确性以及算法效率。DLT-Kmedoids算法使用DTW计算时序数据之间的距离取代传统的欧氏距离度量方式,提高了相似性度量算法精度,同时也提高了聚类算法的准确性和复杂度,引入LB＿Keogh距离在计算DTW距离之前过滤掉大部分不可能是最优匹配序列的序列,对于剩下的序列再使用DTW逐个比较,进一步降低算法的复杂度。最后结合高校建筑用电负荷时间序列数据进行分析,通过与主流聚类算法进行比较,表明该算法对于高校用电负荷数据的聚类任务,能够更准确地识别相似的负荷模式,并以更高的效率进行聚类分析。     

一种结合主动学习的半监督文档聚类算法

半监督文档聚类,即利用少量具有监督信息的数据来辅助无监督文档聚类,近几年来逐渐成为机器学习和数据挖掘领域研究的热点问题.由于获取大量监督信息费时费力,因此,国内外学者考虑如何获得少量但对聚类性能提高显著的监督信息.提出一种结合主动学习的半监督文档聚类算法,通过引入成对约束信息指导DBSCAN的聚类过程来提高聚类性能,得到一种半监督文档聚类算法Cons-DBSCAN.通过对约束集中所含信息量的衡量和对DBSCAN算法本身的分析,提出了一种启发式的主动学习算法,能够选取含信息量大的成对约束集,从而能够更高效地辅助半监督文档聚类.实验结果表明,所提出的算法能够高效地进行文档聚类.通过主动学习算法获得的成对约束集,能够显著地提高聚类性能.并且,算法的性能优于两个代表性的结合主动学习的半监督聚类算法.     

基于分量属性近邻传播的多元时间序列数据聚类方法

鉴于传统方法不能直接有效地对多元时间序列数据进行聚类分析,提出一种基于分量属性近邻传播的多元时间序列数据聚类方法.通过动态时间弯曲方法度量多元时间序列数据之间的总体距离,利用近邻传播聚类算法分别对数据之间的总体距离矩阵和分量近似距离矩阵进行聚类分析,综合考虑这两种视角下序列数据之间的关联关系,使用近邻传播方法对反映原始多元时间序列数据的综合关系矩阵实现较高质量的聚类.数值实验结果表明,与传统聚类方法相比,所提出方法不仅能够有效地反映总体数据特征之间的关系,而且通过重要分量属性序列之间的关联关系分析能够提高原始时间序列数据的聚类效果.     

基于成对约束的交叉熵半监督聚类算法*

极大熵聚类(MEC)目标函数中缺乏成对约束的有效信息表达,在拥有少量成对约束的情况下,可能导致有效监督信息的浪费.在MEC的基础上,文中提出基于成对约束的交叉熵半监督聚类算法.利用样本交叉熵表达成对约束信息,并作为惩罚项引入至MEC的目标函数中,通过拉格朗日最优化处理目标函数,得出聚类中心与隶属度的迭代公式.实验表明,文中算法能有效利用少量的成对约束监督信息提高聚类性能,在实际数据应用中性能较好     

面向时间序列的鲁棒性半监督模糊C均值聚类

针对时间序列模糊C均值聚类算法对噪声数据敏感,及其未能将数据中少量已标记数据所包含的监督信息进行有效利用的问题,提出了一种改进的鲁棒性半监督模糊C均值聚类算法。该算法中先使用马氏距离提出一种样本不确定性分析方法,并加入到半监督模糊C均值聚类建模中,以消除噪声点的影响。并改进半监督模糊C均值聚类的部分监督机制来加大已标记数据的监督能力。采用能够弹性度量时间序列相似性的时间扭曲编辑距离代替欧氏距离进行聚类。通过对7组公开的时间序列数据集进行实验对比,结果表明所提算法具有良好的聚类效果。     

Relaxational metric adaptation and its application to semi-supervised clustering and content-based image retrieval

The performance of many supervised and unsupervised learning algorithms is very sensitive to the choice of an appropriate distance metric. Previous work in metric learning and adaptation has mostly been focused on classification tasks by making use of class label information. In standard clustering tasks, however, class label information is not available. In order to adapt the metric to improve the clustering results, some background knowledge or side information is needed. One useful type of side information is in the form of pairwise similarity or dissimilarity information. Recently, some novel methods (e.g., the parametric method proposed by Xing et al.) for learning global metrics based on pairwise side information have been shown to demonstrate promising results. In this paper, we propose a nonparametric method, called relaxational metric adaptation (RMA), for the same metric adaptation problem. While RMA is local in the sense that it allows locally adaptive metrics, it is also global because even patterns not in the vicinity can have long-range effects on the metric adaptation process. Experimental results for semi-supervised clustering based on both simulated and real-world data sets show that RMA outperforms Xing et al.'s method under most situations. Besides applying RMA to semi-supervised learning, we have also used it to improve the performance of content-based image retrieval systems through metric adaptation. Experimental results based on two real-world image databases show that RMA significantly outperforms other methods in improving the image retrieval performance.     

Locally linear metric adaptation with application to semi-supervised clustering and image retrieval

Many computer vision and pattern recognition algorithms are very sensitive to the choice of an appropriate distance metric. Some recent research sought to address a variant of the conventional clustering problem called semi-supervised clustering, which performs clustering in the presence of some background knowledge or supervisory information expressed as pairwise similarity or dissimilarity constraints. However, existing metric learning methods for semi-supervised clustering mostly perform global metric learning through a linear transformation. In this paper, we propose a new metric learning method that performs nonlinear transformation globally but linear transformation locally. In particular, we formulate the learning problem as an optimization problem and present three methods for solving it. Through some toy data sets, we show empirically that our locally linear metric adaptation (LLMA) method can handle some difficult cases that cannot be handled satisfactorily by previous methods. We also demonstrate the effectiveness of our method on some UCI data sets. Besides applying LLMA to semi-supervised clustering, we have also used it to improve the performance of content-based image retrieval systems through metric learning. Experimental results based on two real-world image databases show that LLMA significantly outperforms other methods in boosting the image retrieval performance.     

Non-linear metric learning using pairwise similarity and dissimilarity constraints and the geometrical structure of data

The problem of clustering with side information has received much recent attention and metric learning has been considered as a powerful approach to this problem. Until now, various metric learning methods have been proposed for semi-supervised clustering. Although some of the existing methods can use both positive (must-link) and negative (cannot-link) constraints, they are usually limited to learning a linear transformation (i.e., finding a global Mahalanobis metric). In this paper, we propose a framework for learning linear and non-linear transformations efficiently. We use both positive and negative constraints and also the intrinsic topological structure of data. We formulate our metric learning method as an appropriate optimization problem and find the global optimum of this problem. The proposed non-linear method can be considered as an efficient kernel learning method that yields an explicit non-linear transformation and thus shows out-of-sample generalization ability. Experimental results on synthetic and real-world data sets show the effectiveness of our metric learning method for semi-supervised clustering tasks.     

Improving clustering with pairwise constraints: a discriminative approach

To obtain a user-desired and accurate clustering result in practical applications, one way is to utilize additional pairwise constraints that indicate the relationship between two samples, that is, whether these samples belong to the same cluster or not. In this paper, we put forward a discriminative learning approach which can incorporate pairwise constraints into the recently proposed two-class maximum margin clustering framework. In particular, a set of pairwise loss functions is proposed, which features robust detection and penalization for violating the pairwise constraints. Consequently, the proposed method is able to directly find the partitioning hyperplane, which can separate the data into two groups and satisfy the given pairwise constraints as much as possible. In this way, it makes fewer assumptions on the distance metric or similarity matrix for the data, which may be complicated in practice, than existing popular constrained clustering algorithms. Finally, an iterative updating algorithm is proposed for the resulting optimization problem. The experiments on a number of real-world data sets demonstrate that the proposed pairwise constrained two-class clustering algorithm outperforms several representative pairwise constrained clustering counterparts in the literature.     

Semi-supervised clustering with metric learning: An adaptive kernel method

Most existing representative works in semi-supervised clustering do not sufficiently solve the violation problem of pairwise constraints. On the other hand, traditional kernel methods for semi-supervised clustering not only face the problem of manually tuning the kernel parameters due to the fact that no sufficient supervision is provided, but also lack a measure that achieves better effectiveness of clustering. In this paper, we propose an adaptive Semi-supervised Clustering Kernel Method based on Metric learning (SCKMM) to mitigate the above problems. Specifically, we first construct an objective function from pairwise constraints to automatically estimate the parameter of the Gaussian kernel. Then, we use pairwise constraint-based K-means approach to solve the violation issue of constraints and to cluster the data. Furthermore, we introduce metric learning into nonlinear semi-supervised clustering to improve separability of the data for clustering. Finally, we perform clustering and metric learning simultaneously. Experimental results on a number of real-world data sets validate the effectiveness of the proposed method.     

Distance metric learning guided adaptive subspace semi-supervised clustering

Most existing semi-supervised clustering algorithms are not designed for handling high-dimensional data. On the other hand, semi-supervised dimensionality reduction methods may not necessarily improve the clustering performance, due to the fact that the inherent relationship between subspace selection and clustering is ignored. In order to mitigate the above problems, we present a semi-supervised clustering algorithm using adaptive distance metric learning (SCADM) which performs semi-supervised clustering and distance metric learning simultaneously. SCADM applies the clustering results to learn a distance metric and then projects the data onto a low-dimensional space where the separability of the data is maximized. Experimental results on real-world data sets show that the proposed method can effectively deal with high-dimensional data and provides an appealing clustering performance.     

双层随机游走半监督聚类

半监督聚类旨在根据用户给出的必连和不连约束,把所有数据点划分到不同的簇中,从而获得更准确、更加符合用户要求的聚类结果.目前的半监督聚类算法大多数通过修改已有的聚类算法或者结合度规学习,使聚类结果与点对约束尽可能地保持一致,却很少考虑点对约束对周围无约束数据的显式影响程度.提出一种由在顶点上的低层随机游走和在组件上的高层随机游走两部分构成的双层随机游走半监督聚类算法,其中,低层随机游走主要负责计算选出的约束顶点对其他顶点的影响范围和影响程度,称为组件;高层随机游走则进一步将各个点对约束以自适应的强度在组件上进行约束传播,把它们在每个顶点上的影响综合在一个簇指示矩阵中.UCI数据集和大型真实数据集上的实验结果表明,双层随机游走半监督聚类算法比其他半监督聚类算法更准确,也比较高效.     

融合矩阵分解与距离度量学习的社会化推荐算法

为解决传统推荐系统中存在的冷启动难题,基于距离反映偏好的假设提出了一种融合矩阵分解与距离度量学习的社会化推荐算法。该算法同时对样本和距离度量进行训练,在满足距离约束的前提下更新距离度量和用户与项目的坐标,并将用户与项目嵌入到统一的低维空间,利用用户与项目之间的距离生成推荐结果。基于豆瓣和Epi-nions数据集的对比实验结果验证了该方法可有效提高推荐系统的可解释性和精确度,明显优于基于矩阵分解的推荐方法。研究结果表明,所提方法缓解了传统推荐系统中存在的冷启动问题,为推荐系统的研究提供了另一种可供参考的研究思路。