核学习机研究

该文概述了近年来机器学习研究领域的一个热点问题———核学习机。首先分析了核方法的主要思想,然后着重介绍了几种新近发展的核学习机,包括支持向量机、核的Fisher判别分析等有监督学习算法及核的主分量分析等无监督学习算法,最后讨论了其应用及前景展望。     

基于极限学习机的无监督领域适应分类器

在构建基于极限学习机的无监督自适应分类器时, 隐含层的参数通常都是随机选取的, 而随机选取的参数不具备领域适应能力. 为了增强跨领域极限学习机的知识迁移能力,提出一种新的基于极限学习机的无监督领域适应分类器学习方法, 该方法主要利用自编码极限学习机对源域和目标域数据进行重构学习, 从而可以获得具有领域不变特性的隐含层参数. 进一步, 结合联合概率分布匹配和流形正则的思想, 对输出层权重进行自适应调整. 所提出算法能对极限学习机的两层参数均赋予领域适应能力,在字符数据集和对象识别数据集上的实验结果表明其具有较高的跨领域分类精度.     

半监督极限学习机及其在近红外光谱数据分析中的应用

当数据集中包含的训练信息不充分时,监督的极限学习机较难应用,因此将半监督学习应用到极限学习机,提出一种半监督极限学习机分类模型;但其模型是非凸、非光滑的,很难直接求其全局最优解。为此利用组合优化方法,将提出的半监督极限学习机化为线性混合整数规划,可直接得到其全局最优解。进一步,利用近红外光谱技术,将半监督极限学习机应用于药品和杂交种子的近红外光谱数据的模式分类。与传统方法相比,在不同的光谱区域的数值实验结果显示:当数据集中包含训练信息不充分时,提出的半监督极限学习机提高了模型的推广能力,验证了所提出方法的可行性和有效性。     

基于深度学习的行人重识别研究综述

随着深度学习的发展,研究人员开始探索将深度学习应用于行人重识别任务并提出了大量方法,随之也迎来了新的挑战。为系统地了解这一领域的研究现状和发展趋势,首先对行人重识别任务以及存在的问题进行简单介绍;其次,根据训练方式的不同,分别探讨监督学习、半监督学习/弱监督学习以及无监督学习上行人重识别任务的研究进展,并根据现有研究热度介绍生成对抗网络和注意力机制在行人重识别上的应用;之后,列举了该领域中常用的经典数据集,并对比了深度模型在这些经典数据集（Market-1501、CUHK03等）上的表现;最后,对行人重识别领域的未来方向进行了展望。     

一种基于标签传播的半监督核学习算法

一个好的核函数能提升机器学习模型的有效性,但核函数的选择并不容易,其与问题背景密切相关,且依赖于领域知识和经验。核学习是一种通过训练数据集寻找最优核函数的机器学习方法,能通过有监督学习的方式寻找到一组基核函数的最优加权组合。考虑到训练数据集获取标签的代价,提出一种基于标签传播的半监督核学习方法,该方法能够同时利用有标签数据和无标签数据进行核学习,通过半监督学习中被广泛使用的标签传播方法结合和谐函数获得数据集统一的标签分布。在UCI数据集上对提出的算法进行性能评估,结果表明该方法是有效的。     

基于半监督极限学习机的轴承故障诊断

鉴于在实际的应用中滚动轴承的故障信号所属的类别往往是未知的,而且为了得到一定的测试数据需要花费大量的时间,甚至对机械设备造成了一些损害.利用极限学习机训练速度快且泛化能力强的特点,提出了一种基于半监督极限学习机的滚动轴承故障诊断方法,该方法允许在有少量带标签的轴承故障数据的情况下,将带标签的历史数据与新采集到的部分未带标签的数据一起用来训练得到一个最优的诊断模型.首先通过相空间重构将原始一维信号映射到一个高维的相空间,在相空间中提取初始的轴承特征集,然后将特征集输入半监督的极限学习机中进行训练和测试.实验结果表明,这种基于半监督算法的诊断模型简单,在神经元个数较少的情况下仍然具有很好的泛化能力,具有一定的应用价值.     

拉普拉斯多层极速学习机

极速学习机不仅仅是有效的分类器,还能应用到半监督学习中.但是,半监督极速学习机和拉普拉斯光滑孪生支持向量机一样是一种浅层学习算法.深度学习实现了复杂函数的逼近并缓解了以前多层神经网络算法的局部最小性问题,目前在机器学习领域中引起了广泛的关注.多层极速学习机（ML-ELM）是根据深度学习和极速学习机的思想提出的算法,通过堆叠极速学习机-自动编码器算法（ELM-AE）构建多层神经网络模型,不仅实现复杂函数的逼近,并且训练过程中无需迭代,学习效率高.我们把流形正则化框架引入ML-ELM中提出拉普拉斯多层极速学习机算法（Lap-ML-ELM）.然而,ELM-AE不能很好的解决过拟合问题,针对这一问题我们把权值不确定引入ELM-AE中提出权值不确定极速学习机-自动编码器算法（WU-ELM-AE）,它学习到更为鲁棒的特征.最后,我们在前面两种算法的基础上提出权值不确定拉普拉斯多层极速学习机算法（WUL-ML-ELM）,它堆叠WU-ELM-AE构建深度模型,并且用流形正则化框架求取输出权值,该算法在分类精度上有明显提高并且不需花费太多的时间.实验结果表明,Lap-ML-ELM与WUL-ML-ELM都是有效的半监督学习算法.     

基于深度学习的复杂网络实时Sybil攻击检测算法

针对复杂网络中Sybil攻击检测速度较慢的问题,提出一种基于深度学习的复杂网络实时Sybil攻击检测方案。从网络中采集数据,提取合适的特征;通过深度学习技术预测网络中的攻击行为。基于多层核极限学习机的深度学习技术包括无监督表示学习与监督特征分类两个阶段。通过低秩逼近法计算近似的经验核映射,代替原极限学习机随机生成的隐层。将经验核映射-自动编码的栈式自编码器作为表示学习,对极限学习机的时间效率与存储成本实现了显著的提高。基于实际社交数据的实验结果表明,该方案有效地降低了Sybil攻击的检测时间,并且保持了较好的检测效果。     

流形极限学习机自编码特征表示

极限学习机（ELM）作为一种无监督分类方法,具有学习速度快、泛化性能高、逼近能力好的优点。随着无监督学习的发展,将ELM与自动编码器集成已成为无标签数据集提取特征的新视角,如极限学习机自动编码器（ELM-AE）是一种无监督的神经网络,无需迭代即可找到代表原始样本和其学习过程的主要成分。其重建输入信号获取原始样本的主要特征,且考虑了原始数据的全局信息以避免信息的丢失,然而这类方法未考虑数据的固有流形结构即样本间的近邻结构关系。借鉴极限学习机自动编码器的思想,提出了一种基于流形的极限学习机自动编码器算法（M-ELM）。该算法是一种非线性无监督特征提取方法,结合流形学习保持数据的局部信息,且在特征提取过程中同时对相似度矩阵进行学习。通过在IRIS数据集、脑电数据集和基因表达数据集上进行实验,将该算法与其他无监督学习方法PCA、LPP、NPE、LE和ELM-AE算法经过[k]-means聚类后的准确率进行了比较,以表明该算法的有效性。     

弱监督场景下的行人重识别研究综述

近年来随着智能监控领域的不断发展,行人重识别问题逐渐受到学术界和工业界的广泛关注,其主要研究将不同摄像头下相同身份的行人图像进行关联.当前大部分研究工作关注在有监督场景下,即给定的训练数据都存在标记信息,然而考虑到数据标注工作的高成本,这在现实应用中往往是难以拓展的.本文关注于弱监督场景下的行人重识别算法,包括无监督场景和半监督场景,并且对当前先进的方法进行了分类和描述.对于无监督场景的行人重识别算法,本文根据其技术类型划分为五类,分别为基于伪标记的方法、基于图像生成的方法、基于实例分类的方法、基于领域自适应的方法和其他方法;对于半监督场景的行人重识别方法,本文根据其场景类型划分为四类,分别为少量的人有标记的场景、每一个人有少量标记的场景、基于tracklet的学习的场景和摄像头內有标记但摄像头间无标记的场景.最后,本文对当前行人重识别的相关数据集进行了整理,并对现有的弱监督方法的实验结果进行分析与总结.     

A novel robust kernel for visual learning problems

A major challenge to appearance-based learning techniques is the robustness against data corruption and irrelevant within-class data variation. This paper presents a robust kernel for kernel-based approach to achieving better robustness on several visual learning problems. Incorporating a robust error function used in robust statistics together with a deformation invariant distance measure, the proposed kernel is shown to be insensitive to noise and robust to intra-class variations. We prove that this robust kernel satisfies the requirements for a valid kernel, so it has good properties when used with kernel-based learning machines. In the experiments, we validate the superior robustness of the proposed kernel over the state-of-the-art algorithms on several applications, including hand-written digit classification, face recognition and data visualization.     

基于核空间中最优变换和聚类中心的鉴别特征提取

应用统计学习理论中的核化原理,可以将许多线性特征提取算法推广至非线性特征提取算法.提出了基于核化原理的最优变换与聚类中心算法,即通过非线性变换,将数据映射到高维核空间,应用最优变换算法,实现原空间数据的非线性特征提取,而求解过程却借助"核函数",回避了复杂非线性变换的具体表达形式.新算法可提取稳健的非线性鉴别特征,从而解决复杂分布数据的模式分类问题.大量数值实验表明新算法比传统的最优变换与聚类中心算法更有效,甚至优于经典的核Fisher判别分析.     

An introduction to kernel-based learning algorithms

This paper provides an introduction to support vector machines, kernel Fisher discriminant analysis, and kernel principal component analysis, as examples for successful kernel-based learning methods. We first give a short background about Vapnik-Chervonenkis theory and kernel feature spaces and then proceed to kernel based learning in supervised and unsupervised scenarios including practical and algorithmic considerations. We illustrate the usefulness of kernel algorithms by discussing applications such as optical character recognition and DNA analysis.     

A survey of the state of the art in learning the kernels

In recent years, the machine learning community has witnessed a tremendous growth in the development of kernel-based learning algorithms. However, the performance of this class of algorithms greatly depends on the choice of the kernel function. Kernel function implicitly represents the inner product between a pair of points of a dataset in a higher dimensional space. This inner product amounts to the similarity between points and provides a solid foundation for nonlinear analysis in kernel-based learning algorithms. The most important challenge in kernel-based learning is the selection of an appropriate kernel for a given dataset. To remedy this problem, algorithms to learn the kernel have recently been proposed. These methods formulate a learning algorithm that finds an optimal kernel for a given dataset. In this paper, we present an overview of these algorithms and provide a comparison of various approaches to find an optimal kernel. Furthermore, a list of pivotal issues that lead to efficient design of such algorithms will be presented.     

核函数支持向量机

概述了基于核函数方法的支持向量机。首先简要叙述支持向量机的基本思想和核特征空间,然后重点介绍核函数支持向量机的前沿理论与领先技术,同时描述了核函数支持向量机在关键领域的应用。     

Kernel-based transition probability toward similarity measure for semi-supervised learning

For improving the classification performance on the cheap, it is necessary to exploit both labeled and unlabeled samples by applying semi-supervised learning methods, most of which are built upon the pair-wise similarities between the samples. While the similarities have so far been formulated in a heuristic manner such as by k-NN, we propose methods to construct similarities from the probabilistic viewpoint. The kernel-based formulation of a transition probability is first proposed via comparing kernel least squares to variational least squares in the probabilistic framework. The formulation results in a simple quadratic programming which flexibly introduces the constraint to improve practical robustness and is efficiently computed by SMO. The kernel-based transition probability is by nature favorably sparse even without applying k-NN and induces the similarity measure of the same characteristics. Besides, to cope with multiple types of kernel functions, the multiple transition probabilities obtained correspondingly from the kernels can be probabilistically integrated with prior probabilities represented by linear weights. We propose a computationally efficient method to optimize the weights in a discriminative manner. The optimized weights contribute to a composite similarity measure straightforwardly as well as to integrate the multiple kernels themselves as multiple kernel learning does, which consequently derives various types of multiple kernel based semi-supervised classification methods. In the experiments on semi-supervised classification tasks, the proposed methods demonstrate favorable performances, compared to the other methods, in terms of classification performances and computation time.     

Kernel-based learning and feature selection analysis for cancer diagnosis

DNA microarray is a very active area of research in the molecular diagnosis of cancer. Microarray data are composed of many thousands of features and from tens to hundreds of instances, which make the analysis and diagnosis of cancer very complex. In this case, gene/feature selection becomes an elemental and essential task in data classification. In this paper, we propose a complete cancer diagnostic process through kernel-based learning and feature selection. First, support vector machines recursive feature elimination (SVM-RFE) is used to prefilter the genes. Second, the SVM-RFE is enhanced by using binary dragonfly (BDF), which is a recently developed metaheuristic that has never been benchmarked in the context of feature selection. The objective function is the average of classification accuracy rate generated by three kernel-based learning methods. We conducted a series of experiments on six microarray datasets often used in the literature. Experiment results demonstrate that this approach is efficient and provides a higher classification accuracy rate using a reduced number of genes.     

基于核学习方法的短时交通流量预测

基于核学习的强大非线性映射性能,针对短时交通流量预测,提出一类基于核学习方法的预测模型。核递推最小二乘(KRLS)基于近似线性依赖(approximate linear dependence,ALD) 技术可降低计算复杂度及存储量,是一种在线核学习方法,适用于较大规模数据集的学习;核偏最小二乘(KPLS)方法将输入变量投影在潜在变量上,利用输入与输出变量之间的协方差信息提取潜在特征;核极限学习机(KELM)方法用核函数表示未知的隐含层非线性特征映射,通过正则化最小二乘算法计算网络的输出权值,能以极快的学习速度获得良好的推广性。为验证所提方法的有效性,将KELM、KPLS、ALD-KRLS用于不同实测交通流数据中,在同等条件下,与现有方法进行比较。实验结果表明,不同核学习方法的预测精度和训练速度均有提高,体现了核学习方法在短时交通流量预测中的应用潜力。     

Multiple kernel clustering based on centered kernel alignment

Multiple kernel clustering (MKC), which performs kernel-based data fusion for data clustering, is an emerging topic. It aims at solving clustering problems with multiple cues. Most MKC methods usually extend existing clustering methods with a multiple kernel learning (MKL) setting. In this paper, we propose a novel MKC method that is different from those popular approaches. Centered kernel alignment—an effective kernel evaluation measure—is employed in order to unify the two tasks of clustering and MKL into a single optimization framework. To solve the formulated optimization problem, an efficient two-step iterative algorithm is developed. Experiments on several UCI datasets and face image datasets validate the effectiveness and efficiency of our MKC algorithm.