一种增量贝叶斯分类模型

分类一直是机器学习，模型识别和数据挖掘研究的核心问题，从海量数据中学习分类知识，尤其是当获得大量的带有类别标注的样本代价较高时，增量学习是解决该问题的有效途径，该文将简单贝叶期方法应用于增量分类中，提出了一种增量贝叶斯学习模型，给出了增量贝叶斯推理过程，包括增量地修正分类器参数和增量地分类测试样本，实验结果表明，该算法是可行的和有效。     

基于答案辅助的半监督问题分类方法

问题分类旨在对问题的类型进行自动分类,该任务是问答系统研究的一项基本任务。提出了一种基于答案辅助的半监督问题分类方法。首先,将答案特征结合问题特征一起实现样本表示;然后,利用标签传播方法对已标注问题训练分类器,自动标注未标注问题的类别;最后,将初始标注的问题和自动标注的问题合并作为训练样本,利用最大熵模型对问题的测试文本进行分类。实验结果表明,本文提出的基于答案辅助的半监督分类方法能够充分利用未标注样本提升性能,明显优于其他的基准方法。     

Active domain adaptation with noisy labels for multimedia analysis

Supervised learning methods require sufficient labeled examples to learn a good model for classification or regression. However, available labeled data are insufficient in many applications. Active learning (AL) and domain adaptation (DA) are two strategies to minimize the required amount of labeled data for model training. AL requires the domain expert to label a small number of highly informative examples to facilitate classification, while DA involves tuning the source domain knowledge for classification on the target domain. In this paper, we demonstrate how AL can efficiently minimize the required amount of labeled data for DA. Since the source and target domains usually have different distributions, it is possible that the domain expert may not have sufficient knowledge to answer each query correctly. We exploit our active DA framework to handle incorrect labels provided by domain experts. Experiments with multimedia data demonstrate the efficiency of our proposed framework for active DA with noisy labels.     

一种基于标签传播的半监督核学习算法

一个好的核函数能提升机器学习模型的有效性,但核函数的选择并不容易,其与问题背景密切相关,且依赖于领域知识和经验。核学习是一种通过训练数据集寻找最优核函数的机器学习方法,能通过有监督学习的方式寻找到一组基核函数的最优加权组合。考虑到训练数据集获取标签的代价,提出一种基于标签传播的半监督核学习方法,该方法能够同时利用有标签数据和无标签数据进行核学习,通过半监督学习中被广泛使用的标签传播方法结合和谐函数获得数据集统一的标签分布。在UCI数据集上对提出的算法进行性能评估,结果表明该方法是有效的。     

关联子域对齐网络的跨域高光谱图像分类

目的 近年来,深度网络成功应用于高光谱图像分类。然而,难以获取充足的标记数据大大限制了深度网络的充分训练,进而导致网络对高光谱图像的分类能力下降。为解决以上困难,提出一种关联子域对齐网络的高光谱图像迁移分类方法。方法 基于深度迁移学习方法,通过对两域分布进行多角度、全面领域适应的同时将两域分类器进行差异适配。一方面,利用关联对齐从整体上对齐了两域的二阶统计量信息,适配了两域的全局分布;另一方面,利用局部最大均值差异对齐了相关子域的一阶统计量信息,适配了两域的局部分布。另外,构造一种分类器适配模块并将其加入所提网络中,通过对两域分类器差异进行适配,进一步增强网络的领域适应效果。结果 从4组真实高光谱数据集上的实验结果可看出：在分别采集于不同区域的高光谱图像数据对上,所提方法的精度比排名第2的分类方法高出1.01%、0.42%、0.73%和0.64%。本文方法的Kappa系数也取得最优结果。结论 与现有主流算法相比较,所提网络能够在整体和局部、一阶和二阶统计量上分别对两域进行有效对齐,进而充分利用在源域上训练好的分类器完成对目标域高光谱数据的跨域分类。     

基于dropout正则化的半监督域自适应方法

针对机器学习中训练样本和测试样本概率分布不一致的问题,提出了一种基于dropout正则化的半监督域自适应方法来实现将神经网络的特征表示从标签丰富的源域转移到无标签的目标域。此方法从半监督学习的角度出发,在源域数据中添加少量带标签的目标域数据,使得神经网络在学习到源域数据特征分布的同时也能学习到目标域数据的特征分布。由于有了先验知识的指导,即使没有丰富的标签信息,神经网络依然可以很好地拟合目标域数据。实验结果表明,此算法在几种典型的数字数据集SVHN、MNIST和USPS的域自适应任务上的性能优于现有的其他算法,并且在涵盖广泛自然类别的真实数据集CIFAR-10和STL-10的域自适应任务上有较好的鲁棒性。     

Robust classifiers by mixed adaptation

The production of robust classifiers by combining supervised training with unsupervised training is discussed. A supervised training phase exploits statistically scene invariant labeled data to produce an initial classifier. This is followed by an unsupervised training phase that exploits clustering properties of unlabeled data. This two-phase process is termed mixed adaptation. A probabilistic model supporting this technique is presented along with examples illustrating mixed adaptation. These examples include the detection of unspecified dotted curves in dotted noise and the detection and classification of vehicles in cinematic sequences of infrared imagery     

Robustness through prior knowledge: using explanation-based learning to distinguish handwritten Chinese characters

Handwritten Chinese character recognition is difficult due to the unstructured and noisy nature of its training examples. There are often too few training examples for a statistical learner like SVM to overcome the noise and extract useful information reliably. Existing prior domain knowledge represents a valuable source of information for classifying handwritten characters. Explanation-based learning (EBL) provides a way to incorporating prior domain knowledge into the learner. The dynamic bias formed by the interaction of domain knowledge with training examples can yield solution knowledge of potential higher quality. Two EBL approaches, one that uses a special feature kernel function in SVM, the other uses a conventional kernel for the SVM but provides additional preference in choosing the classification hyperplane, are reported.     

基于迁移学习的细粒度实体分类方法的研究

细粒度实体分类(Fine-grained entity type classification, FETC)旨在将文本中出现的实体映射到层次化的细分实体类别中. 近年来, 采用深度神经网络实现实体分类取得了很大进展. 但是, 训练一个具备精准识别度的神经网络模型需要足够数量的标注数据, 而细粒度实体分类的标注语料非常稀少, 如何在没有标注语料的领域进行实体分类成为难题. 针对缺少标注语料的实体分类任务, 本文提出了一种基于迁移学习的细粒度实体分类方法, 首先通过构建一个映射关系模型挖掘有标注语料的实体类别与无标注语料实体类别间的语义关系, 对无标注语料的每个实体类别, 构建其对应的有标注语料的类别映射集合. 然后, 构建双向长短期记忆(Bidirectional long short term memory, BiLSTM)模型, 将代表映射类别集的句子向量组合作为模型的输入用来训练无标注实体类别. 基于映射类别集中不同类别与对应的无标注类别的语义距离构建注意力机制, 从而实现实体分类器以识别未知实体分类. 实验证明, 我们的方法取得了较好的效果, 达到了在无任何标注语料前提下识别未知命名实体分类的目的.     

加权解耦语义表达的多源领域自适应方法

近年来,深度学习受到越来越多研究者的重视并成功应用于许多领域.虽然深度学习在这些领域获得了巨大的成功,但是数据采集和标注成本高,严重限制了深度学习的推广应用.迁移学习不仅可以打破训练集数据和测试集数据独立同分布的假设,而且可以利用有标签的迁移源数据和没有标签的迁移目标数据训练得到具有良好泛化能力的模型,是扩展深度学习应用场景的重要研究方向.在众多的迁移学习方法中,多源领域自适应方法可以充分利用多个迁移源的信息,具有重要的实际价值.从数据的因果生成机制出发,假设观测数据由语义隐变量和领域隐变量这两组独立的隐变量同时生成.基于上述假设,提出了一种基于多种距离度量框架和加权解耦语义表达的多源领域自适应方法.该方法利用了双重对抗网络来提取解耦的语义信息和领域信息;另一方面,采用了3种不同的语义信息聚合策略获得领域不变的语义表达;最后使用领域不变的语义表达进行图片分类.在多个多源领域自适应数据上的对比及鲁棒性分析实验中,充分地验证了所提出方法的有效性.     

渐进式分离的开放集模糊域自适应算法

域自适应的目的是利用有标记（源）域中的信息来提高未标记（目标）域模型的分类性能,且这种方法已经取得了不错的成果。然而在具有开放性的现实场景下,目标域通常包含源域中未观察到的未知类样本,这被称为开放集域自适应问题。传统的域自适应算法对这样具有挑战性的场景设定无能为力,因此提出了渐进式分离的开放集模糊域自适应算法。首先,基于引进隶属度的开放集模糊域自适应算法,探索了逐步分离目标域中已知类和未知类样本的方法;然后,仅将从目标域中分离出的已知类与源域对齐,从而减小两个域之间的分布差异,进行模糊域自适应。所提算法很好地解决了由于未知类和已知类之间的不匹配而导致的负迁移所带来的影响。在Office数据集上的6组域自适应转化实验结果表明,与传统的域自适应算法比较,所提算法在图像分类中的精度有显著的提升,验证了该算法可以逐步增强域自适应分类模型的准确性和鲁棒性。     

一种基于EM和分类损失的半监督主动DBN学习算法

对于建立动态贝叶斯网络(DBN)分类模型时,带有类标注样本数据集获得困难的问题,提出一种基于EM和分类损失的半监督主动DBN学习算法.半监督学习中的EM算法可以有效利用未标注样本数据来学习DBN分类模型,但是由于迭代过程中易于加入错误的样本分类信息而影响模型的准确性.基于分类损失的主动学习借鉴到EM学习中,可以自主选择有用的未标注样本来请求用户标注,当把这些样本加入训练集后能够最大程度减少模型对未标注样本分类的不确定性.实验表明,该算法能够显著提高DBN学习器的效率和性能,并快速收敛于预定的分类精度.     

稀疏标签传播:一种鲁棒的领域适应学习方法

稀疏表示因其所具有的鲁棒性,在模式分类领域逐渐得到关注.研究了一种基于稀疏保留模型的新颖领域适应学习方法,并提出一种鲁棒的稀疏标签传播领域适应学习(sparse label propagation domain adaptation learning,简称SLPDAL)算法.SLPDAL通过将目标领域数据进行稀疏重构,以实现源领域数据标签向目标领域平滑传播.具体来讲,SLPDAL算法分为3步:首先,基于领域间数据分布均值差最小化准则寻求一个优化的核空间,并将领域数据嵌入到该核空间;然后,在该嵌入核空间,基于l₁-范最小化准则计算各领域数据的核稀疏重构系数;最后,通过保留领域数据间核稀疏重构系数约束,实现源领域数据标签向目标领域的传播.最后,将SLPDAL算法推广到多核学习框架,提出一个SLPDAL多核学习模型.在鲁棒人脸识别、视频概念检测和文本分类等领域适应学习任务上进行比较实验,所提出的方法取得了优于或可比较的学习性能.     

A unified framework for semi-supervised PU learning

Traditional supervised classifiers use only labeled data (features/label pairs) as the training set, while the unlabeled data is used as the testing set. In practice, it is often the case that the labeled data is hard to obtain and the unlabeled data contains the instances that belong to the predefined class but not the labeled data categories. This problem has been widely studied in recent years and the semi-supervised PU learning is an efficient solution to learn from positive and unlabeled examples. Among all the semi-supervised PU learning methods, it is hard to choose just one approach to fit all unlabeled data distribution. In this paper, a new framework is designed to integrate different semi-supervised PU learning algorithms in order to take advantage of existing methods. In essence, we propose an automatic KL-divergence learning method by utilizing the knowledge of unlabeled data distribution. Meanwhile, the experimental results show that (1) data distribution information is very helpful for the semi-supervised PU learning method; (2) the proposed framework can achieve higher precision when compared with the state-of-the-art method.     

一种基于聚类核的半监督支持向量机分类方法

为了在标记样本数目有限时尽可能地提高支持向量机的分类精度,提出了一种基于聚类核的半监督支持向量机分类方法。该算法依据聚类假设,即属于同一类的样本点在聚类中被分为同一类的可能性较大的原则去对核函数进行构造。采用K-均值聚类算法对已有的标记样本和所有的无标记样本进行多次聚类,根据最终的聚类结果去构造聚类核函数,从而更好地反映样本间的相似程度,然后将其用于支持向量机的训练和分类。理论分析和计算机仿真结果表明,该方法充分利用了无标记样本信息,提高了支持向量机的分类精度。     

Integrating Spectral Kernel Learning and Constraints in Semi-Supervised Classification

Recently, integrating new knowledge sources such as pairwise constraints into various classification tasks with insufficient training data has been actively studied in machine learning. In this paper, we propose a novel semi-supervised classification approach, called semi-supervised classification with enhanced spectral kernel, which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first design a non-parameter spectral kernel learning model based on the squared loss function. Then we develop an efficient semi-supervised classification algorithm which takes advantage of Laplacian spectral regularization: semi-supervised classification with enhanced spectral kernel under the squared loss (ESKS). Finally, we conduct many experiments on a variety of synthetic and real-world data sets to demonstrate the effectiveness of the proposed ESKS algorithm.     

面向问答社区的中文问题分类

传统的问题分类体系大都基于事实类问题,传统的问题分类方法也比较依赖于疑问词这一分类特征,但问答社区(CQA)中非事实类问题居多,且许多问题并不包含疑问词,为此,提出一种面向问答社区的粗粒度分类体系,并在此基础上提出一种基于疑问词的层次化结构问题分类方法。该方法首先自动识别问题中的疑问词,若疑问词存在,则用支持向量机(SVM)模型进行分类;而对没有疑问词的问题,则用所构造的基于焦点词的分类器进行分类。通过在从中文问答社区知乎中所爬取的问题数据集上进行实验,与传统的基于SVM模型的分类方法相比,该方法的分类准确率提高了4.7个百分点。实验结果表明,这种根据问题是否含有疑问词而选择不同分类器的方法,减轻了分类方法对疑问词的依赖,能有效提高问答社区中问题分类的准确率。     

Semi-Supervised Learning on Riemannian Manifolds

We consider the general problem of utilizing both labeled and unlabeled data to improve classification accuracy. Under the assumption that the data lie on a submanifold in a high dimensional space, we develop an algorithmic framework to classify a partially labeled data set in a principled manner. The central idea of our approach is that classification functions are naturally defined only on the submanifold in question rather than the total ambient space. Using the Laplace-Beltrami operator one produces a basis (the Laplacian Eigenmaps) for a Hilbert space of square integrable functions on the submanifold. To recover such a basis, only unlabeled examples are required. Once such a basis is obtained, training can be performed using the labeled data set. Our algorithm models the manifold using the adjacency graph for the data and approximates the Laplace-Beltrami operator by the graph Laplacian. We provide details of the algorithm, its theoretical justification, and several practical applications for image, speech, and text classification.     

领域自适应研究综述

经典机器学习算法假设训练数据和测试数据具有相同的输入特征空间和数据分布,但在很多现实应用中这一假设通常并不成立,导致经典机器学习算法失效。领域自适应是一种新的机器学习策略,其关键技术在于通过学习新的特征表达来对齐源域和目标域的数据分布,使得在有标签源域中训练的模型可以直接迁移到没有标签的目标域上,且不会引起模型性能的明显下降。介绍领域自适应的定义、分类和代表性算法,讨论基于度量学习和基于对抗学习的两类领域自适应算法。在此基础上,分析领域自适应的典型应用和现存挑战,并对其发展趋势及未来研究方向进行展望。     

Transfer Learning from Unlabeled Data via Neural Networks

A machine learning framework which uses unlabeled data from a related task domain in supervised classification tasks is described. The unlabeled data come from related domains, which share the same class labels or generative distribution as the labeled data. Patterns in the unlabeled data are learned via a neural network and transferred to the target domain from where the labeled data are generated, so as to improve the performance of the supervised learning task. We call this approach self-taught transfer learning from unlabeled data. We introduce a general-purpose feature learning algorithm producing features that retain information from the unlabeled data. Information preservation assures that the features obtained will be useful for improving the classification performance of the supervised tasks.