Learning Instance Weighted Naive Bayes from labeled and unlabeled data

In real-world data mining applications, it is often the case that unlabeled instances are abundant, while available labeled instances are very limited. Thus, semi-supervised learning, which attempts to benefit from large amount of unlabeled data together with labeled data, has attracted much attention from researchers. In this paper, we propose a very fast and yet highly effective semi-supervised learning algorithm. We call our proposed algorithm Instance Weighted Naive Bayes (simply IWNB). IWNB firstly trains a naive Bayes using the labeled instances only. And the trained naive Bayes is used to estimate the class membership probabilities of the unlabeled instances. Then, the estimated class membership probabilities are used to label and weight unlabeled instances. At last, a naive Bayes is trained again using both the originally labeled data and the (newly labeled and weighted) unlabeled data. Our experimental results based on a large number of UCI data sets show that IWNB often improves the classification accuracy of original naive Bayes when available labeled data are very limited.     

基于图的半监督关系抽取

提出利用基于图的半监督学习算法,即标注传递算法,指导计算机从非结构化的文本中自动识别出实体之间的关系.该方法首先利用图策略来建立关系抽取的模型.在这个图模型中,各个有标签和未标签的样本被表示成图上的各个节点,而样本间的距离则作为图上各边的权重.然后,关系抽取的任务就转化成在这个图上估计出一个满足全局一致性假设的标注函数通过对ACE(automatic content extraction)语料库的评测,结果显示,当只有少量的标签样本时,采用该标注传递的方法可以获得比基于SVM(support vector machine)的有监督关系抽取更好的性能,同时也明显优于基于Bootstrapping的半监督关系抽取的方法.     

基于半监督学习和支持向量机的煤与瓦斯突出预测研究

针对支持向量机要求输入向量为已标记样本,而实际应用中已标记样本很难获取的问题,提出将半监督学习和支持向量机结合的煤与瓦斯突出预测方法;介绍了采用SVM预测煤与瓦斯突出的流程及其输入向量的选择;对半监督学习中的协同训练算法进行了改进:在同一属性集上训练2个不同分类器SVM和KNN,将2个分类器标记一致的样本加入训练集,从而充分利用未标记样本不断补充信息,更新训练集标记样本,达到强化训练集的目的。测试结果表明,改进后的算法比单独的支持向量机预测方法准确率更高。     

Learning model order from labeled and unlabeled data for partially supervised classification, with application to word sense disambiguation

Previous partially supervised classification methods can partition unlabeled data into positive examples and negative examples for a given class by learning from positive labeled examples and unlabeled examples, but they cannot further group the negative examples into meaningful clusters even if there are many different classes in the negative examples. Here we proposed an automatic method to obtain a natural partitioning of mixed data (labeled data + unlabeled data) by maximizing a stability criterion defined on classification results from an extended label propagation algorithm over all the possible values of model order (or the number of classes) in mixed data. Our experimental results on benchmark corpora for word sense disambiguation task indicate that this model order identification algorithm with the extended label propagation algorithm as the base classifier outperforms SVM, a one-class partially supervised classification algorithm, and the model order identification algorithm with semi-supervised k-means clustering as the base classifier when labeled data is incomplete.     

Semi-supervised orthogonal discriminant analysis via label propagation

Trace ratio is a natural criterion in discriminant analysis as it directly connects to the Euclidean distances between training data points. This criterion is re-analyzed in this paper and a fast algorithm is developed to find the global optimum for the orthogonal constrained trace ratio problem. Based on this problem, we propose a novel semi-supervised orthogonal discriminant analysis via label propagation. Differing from the existing semi-supervised dimensionality reduction algorithms, our algorithm propagates the label information from the labeled data to the unlabeled data through a specially designed label propagation, and thus the distribution of the unlabeled data can be explored more effectively to learn a better subspace. Extensive experiments on toy examples and real-world applications verify the effectiveness of our algorithm, and demonstrate much improvement over the state-of-the-art algorithms.     

基于两阶段学习的半监督支持向量机分类算法

提出了一种基于两阶段学习的半监督支持向量机(semi-supervised SVM)分类算法.首先使用基于图的标签传递算法给未标识样本赋予初始伪标识,并利用k近邻图将可能的噪声样本点识别出来并剔除;然后将去噪处理后的样本集视为已标识样本集输入到支持向量机(SVM)中,使得SVM在训练时能兼顾整个样本集的信息,从而提高SVM的分类准确率.实验结果证明,同其它半监督学习算法相比较,本文算法在标识的训练样本较少的情况下,分类性能有所提高且具有较高的可靠性.     

半监督SVM的工作集样本预选取方法

针对传统的半监督SVM训练方法把大量时间花费在非支持向量优化上的问题,提出了在凹半监督支持向量机方法中采用遗传FCM（Genetic Fuzzy C Mean,遗传模糊C均值）进行工作集样本预选取的方法。半监督SVM优化学习过程中,在原来训练集上（标签数据）加入了工作集（无标签数据）,从而构成了新的训练集。该方法首先利用遗传FCM算法将未知数据划分成某个数量的子集,然后用凹半监督SVM对新数据进行训练得到决策边界与支持矢量,最后对无标识数据进行分类。这样通过减小工作样本集,选择那些可能成为支持向量的边界向量来加入训练集,减少参与训练的样本总数,从而减小了内存开销。并且以随机三维数据为例进行分析,实验结果表明,工作集减小至原工作集的一定范围内,按比例减少工作集后的分类准确率、支持向量数与用原工作集相比差别不大,而分类时间却大为减少,获得了较为理想的样本预选取效果。     

An algorithm for semi-supervised learning in image retrieval

We study the problem of image retrieval based on semi-supervised learning. Semi-supervised learning has attracted a lot of attention in recent years. Different from traditional supervised learning. Semi-supervised learning makes use of both labeled and unlabeled data. In image retrieval, collecting labeled examples costs human efforts, while vast amounts of unlabeled data are often readily available and offer some additional information. In this paper, based on support vector machine (SVM), we introduce a semi-supervised learning method for image retrieval. The basic consideration of the method is that, if two data points are close to each, they should share the same label. Therefore, it is reasonable to search a projection with maximal margin and locality preserving property. We compare our method to standard SVM and transductive SVM. Experimental results show efficiency and effectiveness of our method.     

面向类别比例偏移的半监督支持向量机方法*

当未标记数据与有标记数据类别比例偏移较大时,半监督支持向量机性能不佳.基于此情况,文中提出面向类别比例偏移的半监督支持向量机方法.首先估计未标记数据类中心,然后对多个类别比例下的类中心进行最坏情况集成,从而提升半监督支持向量机的性能保障.实验表明,文中方法有效提升半监督支持向量机在类别比例偏移时的性能保障.     

Soft Constraint Harmonic Energy Minimization for Transductive Learning and its Two Interpretations

Using the labeled and unlabeled data to enhance the performance of classification is the core idea of transductive learning. It has recently attracted much interest of researchers on this topic. In this paper, we extend the harmonic energy minimization algorithm and propose a novel transductive learning algorithm on graph with soft label and soft constraint. Relaxing the label to real value makes the transductive problem easy to solve, while softening the hard constraint for the labeled data makes it tolerable to the noise in labeling. We discuss two cases for our algorithm and derive exactly the same form of solution. More importantly, such form of solution can be interpreted from the view of label propagation and a special random walks on graph, which make the algorithm intuitively reasonable. We also discuss several related issues of the proposed algorithm. Experiments on toy examples and real world classification problems demonstrate the effectiveness of our algorithm.     

基于集成学习的半监督情感分类方法研究

情感分类旨在对文本所表达的情感色彩类别进行分类的任务。该文研究基于半监督学习的情感分类方法,即在很少规模的标注样本的基础上,借助非标注样本提高情感分类性能。为了提高半监督学习能力,该文提出了一种基于一致性标签的集成方法,用于融合两种主流的半监督情感分类方法:基于随机特征子空间的协同训练方法和标签传播方法。首先,使用这两种半监督学习方法训练出的分类器对未标注样本进行标注;其次,选取出标注一致的未标注样本;最后,使用这些挑选出的样本更新训练模型。实验结果表明,该方法能够有效降低对未标注样本的误标注率,从而获得比任一种半监督学习方法更好的分类效果。     

一种面向不完全标记的文本数据流自适应分类方法

现实生活中网络监控、网络评论以及微博等应用领域涌现了大量文本数据流,这些数据的不完全标记和频繁概念漂移给已有的数据流分类方法带来了挑战。为此,面向不完全标记的文本数据流提出了一种自适应的数据流分类算法。该算法以一个标记数据块作为起始数据块,对未标记数据块首先提取标记数据块与未标记数据块之间的特征集,并利用特征在两个数据块间的相似度进行概念漂移检测,最后计算未标记数据中特征的极性并对数据进行预测。实验表明了算法在分类精度上的优越性,尤其在标记信息较少和概念漂移较为频繁时。     

支持向量机的半监督网络流量分类方法

针对传统网络流量分类方法准确率低、开销大、应用范围受限等问题,提出一种支持向量机（SVM）的半监督网络流量分类方法。该方法在SVM训练中,使用增量学习技术在初始和新增样本集中动态地确定支持向量,避免不必要的重复训练,改善因出现新样本而造成原分类器分类精度降低、分类时间长的情况;改进半监督Tri-training方法对分类器进行协同训练,同时使用大量未标记和少量已标记样本对分类器进行反复修正, 减少辅助分类器的噪声数据,克服传统协同验证对分类算法及样本类型要求苛刻的不足。实验结果表明,该方法可明显提高网络流量分类的准确率和效率。     

Improving Korean verb-verb morphological disambiguation using lexical knowledge from unambiguous unlabeled data and selective web counts

This paper deals with verb-verb morphological disambiguation of two different verbs that have the same inflected form. The verb-verb morphological ambiguity (VVMA) is one of the critical Korean parts of speech (POS) tagging issues. The recognition of verb base forms related to ambiguous words highly depends on the lexical information in their surrounding contexts and the domains they occur in. However, current probabilistic morpheme-based POS tagging systems cannot handle VVMA adequately since most of them have a limitation to reflect a broad context of word level, and they are trained on too small amount of labeled training data to represent sufficient lexical information required for VVMA disambiguation.In this study, we suggest a classifier based on a large pool of raw text that contains sufficient lexical information to handle the VVMA. The underlying idea is that we automatically generate the annotated training set applicable to the ambiguity problem such as VVMA resolution via unlabeled unambiguous instances which belong to the same class. This enables to label ambiguous instances with the knowledge that can be induced from unambiguous instances. Since the unambiguous instances have only one label, the automatic generation of their annotated corpus are possible with unlabeled data.In our problem, since all conjugations of irregular verbs do not lead to the spelling changes that cause the VVMA, a training data for the VVMA disambiguation are generated via the instances of unambiguous conjugations related to each possible verb base form of ambiguous words. This approach does not require an additional annotation process for an initial training data set or a selection process for good seeds to iteratively augment a labeling set which are important issues in bootstrapping methods using unlabeled data. Thus, this can be strength against previous related works using unlabeled data. Furthermore, a plenty of confident seeds that are unambiguous and can show enough coverage for learning process are assured as well.We also suggest a strategy to extend the context information incrementally with web counts only to selected test examples that are difficult to predict using the current classifier or that are highly different from the pre-trained data set.As a result, automatic data generation and knowledge acquisition from unlabeled text for the VVMA resolution improved the overall tagging accuracy (token-level) by 0.04%. In practice, 9-10% out of verb-related tagging errors are fixed by the VVMA resolution whose accuracy was about 98% by using the Naïve Bayes classifier coupled with selective web counts.     

低分辨率人脸图像的迭代标签传播识别算法

针对监控视频下低分辨率人脸识别中存在的特征表示能力不强及判别开集人脸图像身份不够准确的问题,文中提出低分辨率人脸图像的迭代标签传播识别算法.采用视觉几何组(VGG)提取人脸图像特征,依据特征的相似度获得高、低分辨率图像的映射关系.对已标记样本和未标记样本进行迭代标签传播,在每次迭代过程中,通过统计每个类别的置信度直方图,估计识别精确率逼近100%的自适应置信度阈值.根据阈值将确认的未标记样本更新至已标记样本集,提高算法召回率.在公共数据集上的实验表明,文中算法在精确率逼近100%的基础上,召回率取得较高值.     

基于可信标签扩展传递的跨领域倾向性分析*

针对传统监督分类方法不能很好地处理不同领域中服从不同分布的数据这一问题进行了研究,提出了一种基于可信标签扩展传递的半监督分类算法。情感种子词与目标领域待标注词之间按照相似度进行标签传递,将具有可信标签的词迭代移入情感种子词集实现扩展,结合目标领域词的先验情感分计算出最终情感分,从而有效地实现跨领域倾向性分析。实验表明,本文的方法能够大幅度提高跨领域情感分析的准确率。     

集成最近邻规则的半监督顺序回归算法

监督型顺序回归算法需要足够多的有标签样本,而在实践中,标注样本的序数耗时耗力,甚至难以完成。为此,提出一种集成最近邻规则的半监督顺序回归算法。基于最近邻,针对每个有标签样本,在无标签数据集选择与其最近似的若干样本赋以相同序数;再由监督型顺序回归算法训练有标签样本和新标注样本。多个数据集的实验结果显示,该方法能显著改善顺序回归性能。另外,引入折扣因子λ评估新标注样本的可信度,并讨论了λ和有标签数据集大小对方法的影响。     

Least Square Transduction Support Vector Machine

Support vector machine (SVM) is a general and powerful learning machine, which adopts supervised manner. However, for many practical machine learning and data mining applications, unlabeled training examples are readily available but labeled ones are very expensive to be obtained. Therefore, semi-supervised learning emerges as the times require. At present, the combination of SVM and semi-supervised learning principle such as transductive learning has attracted more and more attentions. Transductive support vector machine (TSVM) learns a large margin hyperplane classifier using labeled training data, but simultaneously force this hyperplane to be far away from the unlabeled data. TSVM might seem to be the perfect semi-supervised algorithm since it combines the powerful regularization of SVMs and a direct implementation of the clustering assumption, nevertheless its objective function is non-convex and then it is difficult to be optimized. This paper aims to solve this difficult problem. We apply least square support vector machine to implement TSVM, which can ensure that the objective function is convex and the optimization solution can then be easily found by solving a set of linear equations. Simulation results demonstrate that the proposed method can exploit unlabeled data to yield good performance effectively.     

改进的图半监督支持向量机用于P2P网络流识别

对于机器学习在P2P网络流识别中需要大量标记训练数据的问题,提出一种基于改进图半监督支持向量机的P2P流识别方法。采用自动调节的高斯核函数计算少量标识数据和大量未标识训练样本之间的相似距离以构建图模型,并在标记传播过程中嵌入训练样本局部分布信息以获取未标记样本的标识;在此基础上使用所有已标记样本对SVM训练实现P2P网络流识别。实验结果表明该方法能够兼顾整个训练样本集的信息,在提高SVM识别精度的同时,极大降低了人工标记训练样本的成本。     

结合Tri-training半监督学习和凸壳向量的SVM主动学习算法*

为解决监督学习过程中难以获得大量带有类标记样本且样本数据标记代价较高的问题,结合主动学习和半监督学习方法,提出基于Tri-training半监督学习和凸壳向量的SVM主动学习算法.通过计算样本集的壳向量,选择最有可能成为支持向量的壳向量进行标记.为解决以往主动学习算法在选择最富有信息量的样本标记后,不再进一步利用未标记样本的问题,将Tri-training半监督学习方法引入SVM主动学习过程,选择类标记置信度高的未标记样本加入训练样本集,利用未标记样本集中有利于学习器的信息.在UCI数据集上的实验表明,文中算法在标记样本较少时获得分类准确率较高和泛化性能较好的SVM分类器,降低SVM训练学习的样本标记代价.