基于SVM主动学习技术的PU文本分类

PU文本分类(以正例和未标识实例集训练分类器的分类方法)关键在于从U(未标识实例)集中提取尽可能多的可靠反例,然后在正例与可靠反例的基础上使用机器学习的方法构造有效分类器,而已有的方法可靠反例的数量少或不可靠,同样构造的分类器也精度不高,基于SVM主动学习技术的PU文本分类算法提出一种利用SVM与改进的Rocchio分类器进行主动学习的PU文本分类方法,并通过spy技术来提高SVM分类器的准确度,解决某些机器学习中训练样本获取代价过大,尤其是反例样本较难获取的实际问题。实验表明,该方法比目前其它的主动学习方法及面向PU的文本分类方法具有更高的准确率和召回率。     

一种基于聚类的PU主动文本分类方法

文本分类是信息检索的关键问题之一.提取更多的可信反例和构造准确高效的分类器是PU(positive andunlabeled)文本分类的两个重要问题.然而,在现有的可信反例提取方法中,很多方法提取的可信反例数量较少,构建的分类器质量有待提高.分别针对这两个重要步骤提供了一种基于聚类的半监督主动分类方法.与传统的反例提取方法不同,利用聚类技术和正例文档应与反例文档共享尽可能少的特征项这一特点,从未标识数据集中尽可能多地移除正例,从而可以获得更多的可信反例.结合SVM 主动学习和改进的Rocchio 构建分类器,并采用改进的TFIDF(term frequency inverse document frequency)进行特征提取,可以显著提高分类的准确度.分别在3 个不同的数据集中测试了分类结果(RCV1,Reuters-21578,20 Newsgoups).实验结果表明,基于聚类寻找可信反例可以在保持较低错误率的情况下获取更多的可信反例,而且主动学习方法的引入也显著提升了分类精度.     

基于两阶段学习的半监督支持向量机分类算法

提出了一种基于两阶段学习的半监督支持向量机(semi-supervised SVM)分类算法.首先使用基于图的标签传递算法给未标识样本赋予初始伪标识,并利用k近邻图将可能的噪声样本点识别出来并剔除;然后将去噪处理后的样本集视为已标识样本集输入到支持向量机(SVM)中,使得SVM在训练时能兼顾整个样本集的信息,从而提高SVM的分类准确率.实验结果证明,同其它半监督学习算法相比较,本文算法在标识的训练样本较少的情况下,分类性能有所提高且具有较高的可靠性.     

基于支持向量机与无监督聚类相结合的中文网页分类器

提出了一种将支持向量机与无监督聚类相结合的新分类算法,给出了一种新的网页表示方法并应用于网页分类问题。该算法首先利用无监督聚类分别对训练集中正例和反例聚类,然后挑选一些例子训练SVM并获得SVM分类器,任何网页可以通过比较其与聚类中心的距离决定采用无监督聚类方法或SVM分类器进行分类。该算法充分利用了SVM准确率高与无监督聚类速度快的优点。实验表明它不仅具有较高的训练效率,而且有很高的精确度。     

基于正例和未标文档的半监督分类研究

针对负例类别很难获得训练样本的情况,提出了一种基于正例和未标文档的半监督分类方法.已知仅有正例文本的情况下,引入k-means 聚类算法对未标样本集进行聚类,从未标样本集中选出最为可靠的负例样本信息,初始化分类器.基于EM的极大似然估计理论,在每步迭代的E-step中,将中间分类器最有把握对其类别进行预测的未标注样本进行分类,并应用到M-step中修正分类器的参数值,迭代选择最优分类器.实验结果表明,该方法取得了较好的分类效果.     

基于集成学习的半监督情感分类方法研究

情感分类旨在对文本所表达的情感色彩类别进行分类的任务。该文研究基于半监督学习的情感分类方法,即在很少规模的标注样本的基础上,借助非标注样本提高情感分类性能。为了提高半监督学习能力,该文提出了一种基于一致性标签的集成方法,用于融合两种主流的半监督情感分类方法:基于随机特征子空间的协同训练方法和标签传播方法。首先,使用这两种半监督学习方法训练出的分类器对未标注样本进行标注;其次,选取出标注一致的未标注样本;最后,使用这些挑选出的样本更新训练模型。实验结果表明,该方法能够有效降低对未标注样本的误标注率,从而获得比任一种半监督学习方法更好的分类效果。     

基于半监督的联合分类方法

提出一种基于半监督的联合分类方法.该方法在训练过程中,先构造一个基于类中心思想的简易分类器,通过设定有效阈值,从未标记数据中挑选区别度较大的数据加入到SVM的训练集中;在分类过程中,根据待分类点与分类面的相对位置,结合SVM和KNN算法,分两种情况来对其进行分类.实验结果表明,该方法既能在一定程度上克服监督学习算法手动标记大量训练集的困难,又能相应地提高分类准确率.     

基于RSC模型和噪声去除的半监督训练方法

“半监督学习”是利用已经标记好的训练样本和未标记的训练样本一起训练分类器.传统的半监督训练过程对噪声不作辨别,这种做法会因噪声的存在破坏分类器的训练过程,进而影响分类器的分类效果.针对该问题,提出了基于RSC模型和噪声去除的半监督训练方法,在样本训练过程中,使用RSC标签扩展的方法,并添加噪声去除环节.实验表明,该算法能有效降低半监督学习中噪声对分类器的影响,得到更加精确的分类边界,最终提高算法的性能和稳定性.     

基于Tri-Training半监督分类算法的研究

在实际应用中,容易获取大量的未标记样本数据,而样本数据是有限的,因此,半监督分类算法成为研究者关注的热点.文中在协同训练Tri-Training算法的基础上,提出了采用两个不同的训练分类器的Simple-Tri-Training方法和对标记数据进行编辑的Edit-Tri-Training方法,给出了这三种分类方法与监督分类SVM的分类实验结果的比较和分析.实验表明,无标记数据的引入,在一定程度上提高了分类的性能;初始训练集和分类器的选取以及标记过程中数据编辑技术,都是影响半监督分类稳定性和性能的关键点.     

半监督极限学习机及其在近红外光谱数据分析中的应用

当数据集中包含的训练信息不充分时,监督的极限学习机较难应用,因此将半监督学习应用到极限学习机,提出一种半监督极限学习机分类模型;但其模型是非凸、非光滑的,很难直接求其全局最优解。为此利用组合优化方法,将提出的半监督极限学习机化为线性混合整数规划,可直接得到其全局最优解。进一步,利用近红外光谱技术,将半监督极限学习机应用于药品和杂交种子的近红外光谱数据的模式分类。与传统方法相比,在不同的光谱区域的数值实验结果显示:当数据集中包含训练信息不充分时,提出的半监督极限学习机提高了模型的推广能力,验证了所提出方法的可行性和有效性。     

基于k个标记样本的弱监督学习框架

聚类是机器学习领域中的一个研究热点,弱监督学习是半监督学习中一个重要的研究方向,有广泛的应用场景.在对聚类与弱监督学习的研究中,提出了一种基于k个标记样本的弱监督学习框架.该框架首先用聚类及聚类置信度实现了标记样本的扩展.其次,对受限玻尔兹曼机的能量函数进行改进,提出了基于k个标记样本的受限玻尔兹曼机学习模型.最后,完成了对该模型的推理并设计相关算法.为了完成对该框架和模型的检验,选择公开的数据集进行对比实验,实验结果表明,基于k个标记样本的弱监督学习框架实验效果较好.     

Semi-supervised learning with density-ratio estimation

In this paper we study statistical properties of semi-supervised learning, which is considered to be an important problem in the field of machine learning. In standard supervised learning only labeled data is observed, and classification and regression problems are formalized as supervised learning. On the other hand, in semi-supervised learning, unlabeled data is also obtained in addition to labeled data. Hence, the ability to exploit unlabeled data is important to improve prediction accuracy in semi-supervised learning. This problem is regarded as a semiparametric estimation problem with missing data. Under discriminative probabilistic models, it was considered that unlabeled data is useless to improve the estimation accuracy. Recently, the weighted estimator using unlabeled data achieves a better prediction accuracy compared to the learning method using only labeled data, especially when the discriminative probabilistic model is misspecified. That is, improvement under the semiparametric model with missing data is possible when the semiparametric model is misspecified. In this paper, we apply the density-ratio estimator to obtain the weight function in semi-supervised learning. Our approach is advantageous because the proposed estimator does not require well-specified probabilistic models for the probability of the unlabeled data. Based on statistical asymptotic theory, we prove that the estimation accuracy of our method outperforms supervised learning using only labeled data. Some numerical experiments present the usefulness of our methods.     

One-class learning and concept summarization for data streams

In this paper, we formulate a new research problem of concept learning and summarization for one-class data streams. The main objectives are to (1) allow users to label instance groups, instead of single instances, as positive samples for learning, and (2) summarize concepts labeled by users over the whole stream. The employment of the batch-labeling raises serious issues for stream-oriented concept learning and summarization, because a labeled instance group may contain non-positive samples and users may change their labeling interests at any time. As a result, so the positive samples labeled by users, over the whole stream, may be inconsistent and contain multiple concepts. To resolve these issues, we propose a one-class learning and summarization (OCLS) framework with two major components. In the first component, we propose a vague one-class learning (VOCL) module for concept learning from data streams using an ensemble of classifiers with instance level and classifier level weighting strategies. In the second component, we propose a one-class concept summarization (OCCS) module that uses clustering techniques and a Markov model to summarize concepts labeled by users, with only one scanning of the stream data. Experimental results on synthetic and real-world data streams demonstrate that the proposed VOCL module outperforms its peers for learning concepts from vaguely labeled stream data. The OCCS module is also able to rebuild a high-level summary for concepts marked by users over the stream.     

最大规范化依赖性多标记半监督学习方法

针对现有多标记学习方法大多属于有监督学习方法, 而不能有效利用相对便宜且容易获得的大量未标记样本的问题, 本文提出了一种新的多标记半监督学习方法, 称为最大规范化依赖性多标记半监督学习方法(Normalized dependence maximization multi-label semi-supervised learning method). 该方法将已有标签作为约束条件,利用所有样本, 包括已标记和未标记样本,对特征集和标签集的规范化依赖性进行估计, 并以该估计值的最大化为目标, 最终通过求解带边界的迹比值问题为未标记样本打上标签. 与其他经典多标记学习方法在多个真实多标记数据集上的对比实验表明, 本文方法可以有效从已标记和未标记样本中学习, 尤其是已标记样本相对稀少时,学习效果得到了显著提高.     

基于深度贝叶斯主动学习的高光谱图像分类

针对高光谱图像分类中标记样本获取费时费力,无标记数据难以得到有效利用以及主动学习与深度学习结合难等问题,结合贝叶斯深度学习与主动学习的最新进展,提出一种基于深度贝叶斯的主动学习高光谱图像分类算法。利用少量标记样本训练一个卷积神经网络模型,根据与贝叶斯方法结合的主动学习采样策略从无标记样本中选择模型分类最不确定性的样本,选取的样本经人工标记后加入到训练集重新训练模型,减小模型不确定性,提高模型分类精度。通过PaviaU高光谱图像分类的实验结果表明,在少量的标记样本下,提出的方法比传统的方法分类效果更好。     

Ensemble learning from multiple information sources via label propagation and consensus

Many applications are facing the problem of learning from multiple information sources, where sources may be labeled or unlabeled, and information from multiple information sources may be beneficial but cannot be integrated into a single information source for learning. In this paper, we propose an ensemble learning method for different labeled and unlabeled sources. We first present two label propagation methods to infer the labels of training objects from unlabeled sources by making a full use of class label information from labeled sources and internal structure information from unlabeled sources, which are processes referred to as global consensus and local consensus, respectively. We then predict the labels of testing objects using the ensemble learning model of multiple information sources. Experimental results show that our method outperforms two baseline methods. Meanwhile, our method is more scalable for large information sources and is more robust for labeled sources with noisy data.     

基于分层高斯混合模型的半监督学习算法

提出了一种基于分层高斯混合模型的半监督学习算法，半监督学习算法的学习样本包括已标记类别样本和未标记类别学习样本。如用高斯混合模型拟合每个类别已标记学习样本的概率分布，进而用高斯数为类别数的分层高斯混合模型拟合全部(已标记和未标记)学习样本的分布，则形成为一个基于分层的高斯混合模型的半监督学习问题。基于EM算法，首先利用每个类别已标记样本学习高斯混合模型，然后以该模型参数和已标记样本的频率分布作为分层高斯混合模型参数的初值，给出了基于分层高斯混合模型的半监督学习算法，以银行票据印刷体数字识别做实验，实验结果表明，本算法能够获得较好的效果。     

Solution path for manifold regularized semisupervised classification

Traditional learning algorithms use only labeled data for training. However, labeled examples are often difficult or time consuming to obtain since they require substantial human labeling efforts. On the other hand, unlabeled data are often relatively easy to collect. Semisupervised learning addresses this problem by using large quantities of unlabeled data with labeled data to build better learning algorithms. In this paper, we use the manifold regularization approach to formulate the semisupervised learning problem where a regularization framework which balances a tradeoff between loss and penalty is established. We investigate different implementations of the loss function and identify the methods which have the least computational expense. The regularization hyperparameter, which determines the balance between loss and penalty, is crucial to model selection. Accordingly, we derive an algorithm that can fit the entire path of solutions for every value of the hyperparameter. Its computational complexity after preprocessing is quadratic only in the number of labeled examples rather than the total number of labeled and unlabeled examples.     

A discriminative learning framework with pairwise constraints for video object classification

To deal with the problem of insufficient labeled data in video object classification, one solution is to utilize additional pairwise constraints that indicate the relationship between two examples, i.e., whether these examples belong to the same class or not. In this paper, we propose a discriminative learning approach which can incorporate pairwise constraints into a conventional margin-based learning framework. Different from previous work that usually attempts to learn better distance metrics or estimate the underlying data distribution, the proposed approach can directly model the decision boundary and, thus, require fewer model assumptions. Moreover, the proposed approach can handle both labeled data and pairwise constraints in a unified framework. In this work, we investigate two families of pairwise loss functions, namely, convex and nonconvex pairwise loss functions, and then derive three pairwise learning algorithms by plugging in the hinge loss and the logistic loss functions. The proposed learning algorithms were evaluated using a people identification task on two surveillance video data sets. The experiments demonstrated that the proposed pairwise learning algorithms considerably outperform the baseline classifiers using only labeled data and two other pairwise learning algorithms with the same amount of pairwise constraints.