基于半监督学习的多示例多标记E-MIMLSVM+算法

多示例多标记是一种新的机器学习框架,在该框架下一个对象用多个示例来表示,同时与多个类别标记相关联。MIMLSVM+算法将多示例多标记问题转化为一系列独立的二类分类问题,但是在退化过程中标记之间的联系信息会丢失,而E-MIMLSVM+算法则通过引入多任务学习技术对MIMLSVM+算法进行了改进。为了充分利用未标记样本来提高分类准确率,使用半监督支持向量机TSVM对E-MIMLSVM+算法进行了改进。通过实验将该算法与其他多示例多标记算法进行了比较,实验结果显示,改进算法取得了良好的分类效果。     

基于全局和局部标签相关性的MIMLSVM改进算法

多示例多标记学习是用多个示例来表示一个对象,同时该对象与多个类别标记相关联的新型机器学习框架.设计多示例多标记算法的一种方法是使用退化策略将其转化为多示例学习或者是多标记学习,最后退化为传统监督学习,然后使用某种算法进行训练和建模,但是在退化过程中会有信息丢失,从而影响到分类准确率.MIMLSVM算法是以多标记学习为桥梁,将多示例多标记学习问题退化为传统监督学习问题求解,但是该算法在退化过程中没有考虑标记之间的相关信息,本文利用一种既考虑到全局相关性又考虑到局部相关性的多标记算法GLOCAL来对MIMLSVM进行改进,实验结果显示,改进的算法取得了良好的分类效果.     

一种利用关联规则挖掘的多标记分类算法

多标记学习广泛存在于现实生活中,是当今机器学习领域的研究热点.在多标记学习框架中,每个对象由一个示例构成,但可能同时属于多个类别标记,并且各个标记之间相互关联,所以挖掘多标记之间的关联性对于多标记学习框架具有重要的意义.首先对经典的关联规则算法进行改进,提出了基于矩阵分治的频繁项集挖掘算法,并证明了该算法挖掘频繁项集的正确性;进而将该算法应用于多标记学习框架中,分别提出了基于全局关联规则挖掘和局部关联规则挖掘的多标记分类算法;最后对所提出的算法与现有多标记算法进行实验对比,结果表明,算法在5种不同的评价准则下能够取得更好的效果.     

基于近邻加权及多示例的多标记学习改进算法

多数多标记学习方法通过在输出空间中,单示例同时与多个类别标记相关联表示多义性,目前有研究通过在输入空间将单一示例转化为示例包,建立包中多示例与多标记的联系。算法在生成示例包时采用等权重平均法计算每个标记对应样例的均值。由于数据具有局部分布特征,在计算该均值时考虑数据局部分布,将会使生成的示例包更加准确。本论文充分考虑数据分布特性,提出新的分类算法。实验表明改进算法性能优于其他常用多标记学习算法。     

基于主题模型的多示例多标记学习方法

针对现有的大部分多示例多标记(MIML)算法都没有考虑如何更好地表示对象特征这一问题,将概率潜在语义分析(PLSA)模型和神经网络(NN)相结合,提出了基于主题模型的多示例多标记学习方法。算法通过概率潜在语义分析模型学习到所有训练样本的潜在主题分布,该过程是一个特征学习的过程,用于学习到更好的特征表达,用学习到的每个样本的潜在主题分布作为输入来训练神经网络。当给定一个测试样本时,学习测试样本的潜在主题分布,将学习到的潜在主题分布输入到训练好的神经网络中,从而得到测试样本的标记集合。与两种经典的基于分解策略的多示例多标记算法相比,实验结果表明提出的新方法在现实世界中的两种多示例多标记学习任务中具有更优越的性能。     

基于半监督学习的多示例多标签改进算法

多示例多标签学习框架是一种针对解决多义性问题而提出的新型机器学习框架,在多示例多标签学习框架中,一个对象是用一组示例集合来表示,并且和一组类别标签相关联。E-MIMLSVM~+算法是多示例多标签学习框架中利用退化思想的经典分类算法,针对其无法利用无标签样本进行学习从而造成泛化能力差等问题,使用半监督支持向量机对该算法进行改进。改进后的算法可以利用少量有标签样本和大量没有标签的样本进行学习,有助于发现样本集内部隐藏的结构信息,了解样本集的真实分布情况。通过对比实验可以看出,改进后的算法有效提高了分类器的泛化性能。     

用于多标记学习的阈值确定算法

提出一种多标记学习阈值确定算法(DTML),为每个类别标记确定一个阈值。当分类器将一个测试示例预测为某个类别标记的分值大于该类别标记的阈值时,则将该类别标记添加到该测试示例的最终分类结果中。该算法采用编程实现,并将其应用于PT5方法和TML算法。实验结果表明,利用DTML算法为多标记学习算法确定阈值,能够得到较好的分类效果。     

基于多示例多标记的抽油机故障诊断

针对抽油机工况数据可从位移、载荷、电流等多个方面进行描述,若仅仅使用一个特征向量来描述抽油机工况数据会使其信息过于简化,丢失一部分有效信息的问题,以及工况数据具有多义性的特征,提出基于多示例多标记的抽油机故障诊断.该学习方法中,用抽油机的位移、载荷、电流数据作为抽油机工况样本包的多个示例,使用k-medoids聚类算法对样本包进行聚类,将多个样本包转换为若干示例,新示例的每一维表示样本包到样本各聚类中心的距离,再利用MLSVM算法对转换后的多标记问题进行求解.实验结果表明,多示例多标记学习能够及时、准确地诊断出抽油机故障问题.     

多标记学习研究综述

多标记学习不同于传统的监督学习,它是为了解决客观世界中多义性对象的建模问题而提出的一种学习框架。在该框架下,一个示例可以同时隶属于多个标记。经过十多年的发展,机器学习界已经出现了大量关于多标记学习的研究成果,并得到了广泛的应用。本文对多标记学习问题进行系统而详细的阐述,给出多标记学习的问题定义和评价指标,重点介绍多标记学习算法,并提出多标记学习进一步的研究方向。      

基于类属属性的多标记学习算法

在多标记学习框架中,每个对象由一个示例(属性向量)描述,却同时具有多个类别标记.在已有的多标记学习算法中,一种常用的策略是将相同的属性集合应用于所有类别标记的预测中.然而,该策略并不一定是最优选择,原因在于每个标记可能具有其自身独有的特征.基于这个假设,目前已经出现了基于标记的类属属性进行建模的多标记学习算法LIFT.LIFT包含两个步骤:属属性构建与分类模型训练.LIFT首先通过在标记的正类与负类示例上进行聚类分析,构建该标记的类属属性;然后,使用每个标记的类属属性训练对应的二类分类模型.在保留LIFT分类模型训练方法的同时,考察了另外3种多标记类属属性构造机制,从而实现LIFT算法的3种变体——LIFT-MDDM,LIFT-INSDIF以及LIFT-MLF.在12个数据集上进行了两组实验,验证了类属属性对多标记学习系统性能的影响以及LIFT采用的类属属性构造方法的有效性.     

Bayesian multi-instance multi-label learning using Gaussian process prior

Multi-instance multi-label learning (MIML) is a newly proposed framework, in which the multi-label problems are investigated by representing each sample with multiple feature vectors named instances. In this framework, the multi-label learning task becomes to learn a many-to-many relationship, and it also offers a possibility for explaining why a concerned sample has the certain class labels. The connections between instances and labels as well as the correlations among labels are equally crucial information for MIML. However, the existing MIML algorithms can rarely exploit them simultaneously. In this paper, a new MIML algorithm is proposed based on Gaussian process. The basic idea is to suppose a latent function with Gaussian process prior in the instance space for each label and infer the predictive probability of labels by integrating over uncertainties in these functions using the Bayesian approach, so that the connection between instances and every label can be exploited by defining a likelihood function and the correlations among labels can be identified by the covariance matrix of the latent functions. Moreover, since different relationships between instances and labels can be captured by defining different likelihood functions, the algorithm may be used to deal with the problems with various multi-instance assumptions. Experimental results on several benchmark data sets show that the proposed algorithm is valid and can achieve superior performance to the existing ones.     

一种基于示例非独立同分布的多示例多标签分类算法

多示例多标签学习是一种新型的机器学习框架。在多示例多标签学习中,样本以包的形式存在,一个包由多个示例组成,并被标记多个标签。以往的多示例多标签学习研究中,通常认为包中的示例是独立同分布的,但这个假设在实际应用中是很难保证的。为了利用包中示例的相关性特征,提出了一种基于示例非独立同分布的多示例多标签分类算法。该算法首先通过建立相关性矩阵表示出包内示例的相关关系,每个多示例包由一个相关性矩阵表示;然后建立基于不同尺度的相关性矩阵的核函数;最后考虑到不同标签的预测对应不同的核函数,引入多核学习构造并训练针对不同标签预测的多核SVM分类器。图像和文本数据集上的实验结果表明,该算法大大提高了多标签分类的准确性。     

Multi-instance multi-label learning

In this paper, we propose the MIML (Multi-Instance Multi-Label learning) framework where an example is described by multiple instances and associated with multiple class labels. Compared to traditional learning frameworks, the MIML framework is more convenient and natural for representing complicated objects which have multiple semantic meanings. To learn from MIML examples, we propose the MimlBoost and MimlSvm algorithms based on a simple degeneration strategy, and experiments show that solving problems involving complicated objects with multiple semantic meanings in the MIML framework can lead to good performance. Considering that the degeneration process may lose information, we propose the D-MimlSvm algorithm which tackles MIML problems directly in a regularization framework. Moreover, we show that even when we do not have access to the real objects and thus cannot capture more information from real objects by using the MIML representation, MIML is still useful. We propose the InsDif and SubCod algorithms. InsDif works by transforming single-instances into the MIML representation for learning, while SubCod works by transforming single-label examples into the MIML representation for learning. Experiments show that in some tasks they are able to achieve better performance than learning the single-instances or single-label examples directly.     

Missing multi-label learning with non-equilibrium based on two-level autoencoder

For a multi-label learning framework, each instance may belong to multiple labels simultaneously. The classification accuracy can be improved significantly by exploiting various correlations, such as label correlations, feature correlations, or the correlations between features and labels. There are few studies on how to combine the feature and label correlations, and they deal more with complete data sets. However, missing labels or other phenomena often occur because of the cost or technical limitations in the data acquisition process. A few label completion algorithms currently suitable for missing multi-label learning, ignore the noise interference of the feature space. At the same time, the threshold of the discriminant function often affects the classification results, especially those of the labels near the threshold. All these factors pose considerable difficulties in dealing with missing labels using label correlations. Therefore, we propose a missing multi-label learning algorithm with non-equilibrium based on a two-level autoencoder. First, label density is introduced to enlarge the classification margin of the label space. Then, a new supplementary label matrix is augmented from the missing label matrix with the non-equilibrium label completion method. Finally, considering feature space noise, a two-level kernel extreme learning machine autoencoder is constructed to implement the information feature and label correlation. The effectiveness of the proposed algorithm is verified by many experiments on both missing and complete label data sets. A statistical analysis of hypothesis validates our approach.

     

用于多标记学习的局部顺序分类器链算法

标记间的相关性在分类问题中具有重要作用,目前有研究将标记相关性引入多标记学习,通过分类器链的形式将标记结果引入属性空间,为学习其他标记提供有用信息。分类器链中标记的预测顺序具有随机性,分类结果存在着很大的不确定性与不稳定性,且容易造成错误信息的传播。为此充分考虑标记的局部分布特性,提出了一种局部顺序分类器链算法,解决分类器链中分类器顺序问题。实验表明,该算法性能优于其他常用多标记学习算法。     

Markov-Miml: A Markov chain-based multi-instance multi-label learning algorithm

The main aim of this paper is to propose an efficient and novel Markov chain-based multi-instance multi-label (Markov-Miml) learning algorithm to evaluate the importance of a set of labels associated with objects of multiple instances. The algorithm computes ranking of labels to indicate the importance of a set of labels to an object. Our approach is to exploit the relationships between instances and labels of objects. The rank of a class label to an object depends on (i) the affinity metric between the bag of instances of this object and the bag of instances of the other objects, and (ii) the rank of a class label of similar objects. An object, which contains a bag of instances that are highly similar to bags of instances of the other objects with a high rank of a particular class label, receives a high rank of this class label. Experimental results on benchmark data have shown that the proposed algorithm is computationally efficient and effective in label ranking for MIML data. In the comparison, we find that the classification performance of the Markov-Miml algorithm is competitive with those of the three popular MIML algorithms based on boosting, support vector machine, and regularization, but the computational time required by the proposed algorithm is less than those by the other three algorithms.     

基于负相关性增强的不平衡多标签学习算法

由于标签空间过大,标签分布不平衡问题在多标签数据集中广泛存在,解决该问题在一定程度上可以提高多标签学习的分类性能。通过标签相关性提升分类性能是解决该问题的一种最常见的有效策略,众多学者进行了大量研究,然而这些研究更多地是采用基于正相关性策略提升性能。在实际问题中,除了正相关性外,标签的负相关性也可能存在,如果在考虑正相关性的同时,兼顾负相关性,无疑能够进一步改善分类器的性能。基于此,提出了一种基于负相关性增强的不平衡多标签学习算法——MLNCE,旨在解决多标签不平衡问题的同时,兼顾标签间的正负相关性,从而提高多标签分类器的分类性能。首先利用标签密度信息改造标签空间;然后在密度标签空间中探究标签真实的正反相关性信息,并添加到分类器目标函数中;最后利用加速梯度下降法求解输出权重以得到预测结果。在11个多标签标准数据集上与其他6种多标签学习算法进行对比实验,结果表明MLNCE算法可以有效提高分类精度。