一种基于样本加权的合成多核学习方法

多核学习是目前基于内核学习的机器学习领域中的一个新的研究热点。内核学习方法可以把数据映射到高维空间来增加线性分类器如支持向量机的计算性能,它是目前处理非线性模式识别与分类问题的一种便捷、高效的方法。然而,在某些特殊情况下,基于单一核函数的内核学习方法并不能完全有效地处理如数据异构或者不规则、样本规模大、样本分布不平坦等实际问题,所以通过将多个核函数以加权的形式合成一个核函数,来得到更好的识别精度以及效率,是当前研究的一个发展趋势。因此,提出一种基于样本加权的合成多核学习方法,通过单一核函数对样本的拟合以及适应能力(对样本的学习精度),对每一个核函数按照对应的拟合以及适应能力加权,最终得到基于样本加权的合成多核决策函数。为了验证基于样本加权的合成多核学习方法的有效性和可靠性,在多个数据集上进行了实验分析,实验结果表明,与已有的多核学习方法相比较,本文提出的方法取得了更好的分类结果。     

稀疏Tikhonov正则化多分类支持向量机

Tikhonov正则化多分类支持向量机是一种将多分类问题简化为单个优化问题的新型支持向量机.由于Tikhonov正则化多分类支持向量机利用全部类别数据样本构建核函数矩阵,因此不适合大规模数据集的模式分类问题,鉴于该原因,一种稀疏Tikhonov正则化多分类支持量机被建立,其训练算法首先构建样本重要性评价标准,在标准下通过迭代学习获取约简集,最后利用约简集构建核函数矩阵并训练支持向量机.仿真实验结果表明稀疏Tikhonov正则化多分类支持向量机在训练速度和稀疏性方面具有很大的优越性.     

基于特征贡献度加权高斯核函数的粗糙one-class支持向量机

粗糙one-class支持向量机(ROCSVM)是一种一类支持向量机,它通过核函数映射,定义上近似超平面和下近似超平面,使得训练样本能根据在粗糙间隔中的位置,自适应地对决策超平面产生影响.由于ROCSVM训练集只有正类样本,因此充分挖掘和利用训练样本的分类特征对于提高ROCSVM的分类性能有重要意义.为此,提出了一种基于训练样本分类特征贡献度的加权高斯核函数(λRBF):先对训练样本做主成分分析(PCA)得到按特征值排序的向量集,以此向量集构造核函数,使得特征值较大的维度在核函数中起较大的作用.在UCI标准数据集和仿真数据上的实验结果表明:与一般RBF的ROCSVM相比,基于λ-RBF的ROCSVM有着更好的泛化性和更高的识别率.     

一种混合核函数的支持向量机

为了提高支持向量机分类效果、学习能力和外推能力,分析了两种支持向量机的核函数:K型核函数和logistic核函数,构造出一种新的混合核函数的支持向量机,并且对其性能进行了理论分析。把构造出的混合核函数支持向量机与常用核函数构造的向量机应用到二维数据分类与图片分类中进行实验对比。二维数据点和图片分类的实验结果表明,混合核函数的支持向量机的分类效果、学习能力和外推能力,明显优于常用核函数构造的支持向量机。     

基于多核学习支持向量机的音乐流派分类

针对不同特征向量下选择最优核函数的学习方法问题,将多核学习支持向量机(MK-SVM)应用于音乐流派自动分类中,提出了将最优核函数进行加权组合构成合成核函数进行流派分类的方法。多核分类学习能够针对不同的声学特征采用不同的最优核函数,并通过学习得到各个核函数在分类中的权重,从而明确各声学特征在流派分类中的权重,为音乐流派分类中特征向量的分析和选择提供了一个清晰、明确的结果。在ISMIR 2011竞赛数据集上验证了提出的基于多核学习支持向量机(MKL-SVM)的分类方法,并与传统的基于单核支持向量机的方法进行了比较分析。实验结果表明基于MKL-SVM的音乐流派自动分类准确率比传统单核支持向量机的分类准确率提高了6.58%,且该方法与传统的特征选择结果比较,更清楚地解释了所选择的特征向量对流派分类的影响大小,通过选择影响较大的特征组合进行分类,分类结果也有了明显的提升。     

SVM在多源遥感图像分类中的应用研究

在利用遥感图像进行土地利用/覆盖分类过程中,可采用以下两种途径来提高分类精度:一是通过增加有利于分类的数据源,引入地理辅助数据和归一化植被指数(NDVI)来进行多源信息融合;二是选择更好的分类方法,例如支持向量机(SVM)学习方法,由于该方法克服了最大似然法和神经网络的弱点,非常适合高维、复杂的小样本多源数据的分类。为了提高多源遥感图像分类的精度,还研究了支持向量机在遥感图像分类中模型的选择,包括多类模型和核函数的选择。分类结果表明,支持向量机比传统的分类方法具有更高的精度,尤其是基于径向基核函数和一对一多类方法的支持向量机模型更适合多源遥感图像分类,因此,基于支持向量机的多源土地利用/覆盖分类能大大提高分类精度。     

滚动轴承技术故障诊断的支持向量机方法研究

针对当前故障诊断中几种常用方法的不足,首次提出将支持向量机方法应用于滚动轴承技术故障诊断。该文提出的两种算法其核心均是利用支持向量机方法对样本进行分类。支持向量机方法基于小学习样本条件下,通过寻求结构风险最小,以期获得良好的分类效果和泛化能力。两种实验结果表明,在选用合适核函数及参数条件下,支持向量机具有学习速度快、诊断正确率高的优良性能,这一结论表明了该文所提出方法的优越性。     

多尺度核方法的自适应序列学习及应用

多尺度核方法是当前核机器学习领域的一个热点。通常多尺度核的学习在多核处理时存在诸如多核平均组合、迭代学习时间长、经验选择合成系数等弊端。文中基于核目标度量规则,提出一种多尺度核方法的自适应序列学习算法,实现多核加权系数的自动快速求取。实验表明,该方法在回归精度、分类正确率方面比单核支持向量机方法结果更优,函数拟合与分类稳定性更强,证明该算法具有普遍适用性。     

特征加权的核学习方法

提出了一种特征加权的核学习方法,其主要为了解决当前核方法在分类任务中对所有数据特征的同等对待的不足。在分类任务中,数据样本的每个特征所起的作用并不是相同的,有些特征对分类任务有促进作用,应该给予更多的关注。提出的算法集成了多核学习的优势,以加权的方式组合不同的核函数,但所需的计算复杂度更低。实验结果证明,提出的算法与支持向量机、多核学习算法相比,分类准确度优于支持向量机和多核学习算法,在计算复杂度上略高于支持向量机,但远远低于多核学习算法。     

支持向量机核函数选择研究与仿真

支持向量机是一种基于核的学习方法,核函数选取对支持向量机性能有着重要的影响,如何有效地进行核函数选择是支持向量机研究领域的一个重要问题。目前大多数核选择方法不考虑数据的分布特征,没有充分利用隐含在数据中的先验信息。为此,引入能量熵概念,借助超球体描述和核函数蕴藏的度量特征,提出一种基于样本分布能量熵的支持向量机核函数选择方法,以提高SVM学习能力和泛化能力。数值实例仿真验证表明了该方法的可行性和有效性。     

面向乳腺癌辅助诊断的改进支持向量机方法

根据针吸细胞学方法影像中提取的特征值, 设计了一种改进的支持向量机分类方法, 并应用于乳腺癌的辅助诊断。通过对几种常用核函数的对比分析, 所建立的新核函数在诊断中具有很好的综合性能。使用实际临床数据分析显示, 该方法比模因佩雷托(memetic Pareto artificial neural network, MPANN)与一种改进型人工神经网络(evolutionary artificial neural network, EANN)方法在乳腺癌辅助诊断中具有更好的效果, 可以为医疗机构对该疾病的诊断提供有力的决策支持。     

A wavelet extreme learning machine

Extreme learning machine (ELM) has been widely used in various fields to overcome the problem of low training speed of the conventional neural network. Kernel extreme learning machine (KELM) introduces the kernel method to ELM model, which is applicable in Stat ML. However, if the number of samples in Stat ML is too small, perhaps the unbalanced samples cannot reflect the statistical characteristics of the input data, so that the learning ability of Stat ML will be influenced. At the same time, the mix kernel functions used in KELM are conventional functions. Therefore, the selection of kernel function can still be optimized. Based on the problems above, we introduce the weighted method to KELM to deal with the unbalanced samples. Wavelet kernel functions have been widely used in support vector machine and obtain a good classification performance. Therefore, to realize a combination of wavelet analysis and KELM, we introduce wavelet kernel functions to KELM model, which has a mix kernel function of wavelet kernel and sigmoid kernel, and introduce the weighted method to KELM model to balance the sample distribution, and then we propose the weighted wavelet–mix kernel extreme learning machine. The experimental results show that this method can effectively improve the classification ability with better generalization. At the same time, the wavelet kernel functions perform very well compared with the conventional kernel functions in KELM model.     

Kernel linear regression for face recognition

Linear regression uses the least square algorithm to solve the solution of linear regression equation. Linear regression classification (LRC) shows good classification performance on face image data. However, when the axes of linear regression of class-specific samples have intersections, LRC could not well classify the samples that distribute around intersections. Moreover, the LRC could not perform well at the situation of severe lighting variations. This paper proposes a new classification method, kernel linear regression classification (KLRC), based on LRC and the kernel trick. KLRC is a nonlinear extension of LRC and can offset the drawback of LRC. KLRC implicitly maps the data into a high-dimensional kernel space by using the nonlinear mapping determined by a kernel function. Through this mapping, KLRC is able to make the data more linearly separable and can perform well for face recognition with varying lighting. For comparison, we conduct on three standard databases under some evaluation protocols. The proposed methodology not only outperforms LRC but also takes the better performance than typical kernel methods such as kernel linear discriminant analysis and kernel principal component analysis.     

弹性网络核极限学习机的多标记学习算法

将正则化极限学习机或者核极限学习机理论应用到多标记分类中,一定程度上提高了算法的稳定性。但目前这些算法关于损失函数添加的正则项都基于L2正则,导致模型缺乏稀疏性表达。同时,弹性网络正则化既保证模型鲁棒性且兼具模型稀疏化学习,但结合弹性网络的极限学习机如何解决多标记问题鲜有研究。基于此,本文提出一种对核极限学习机添加弹性网络正则化的多标记学习算法。首先,对多标记数据特征空间使用径向基核函数映射;随后,对核极限学习机损失函数施加弹性网络正则项;最后,采用坐标下降法迭代求解输出权值以得到最终预测标记。通过对比试验和统计分析表明,提出的算法具有更好的性能表现。     

Multiple kernel ensemble learning for software defect prediction

Software defect prediction aims to predict the defect proneness of new software modules with the historical defect data so as to improve the quality of a software system. Software historical defect data has a complicated structure and a marked characteristic of class-imbalance; how to fully analyze and utilize the existing historical defect data and build more precise and effective classifiers has attracted considerable researchers’ interest from both academia and industry. Multiple kernel learning and ensemble learning are effective techniques in the field of machine learning. Multiple kernel learning can map the historical defect data to a higher-dimensional feature space and make them express better, and ensemble learning can use a series of weak classifiers to reduce the bias generated by the majority class and obtain better predictive performance. In this paper, we propose to use the multiple kernel learning to predict software defect. By using the characteristics of the metrics mined from the open source software, we get a multiple kernel classifier through ensemble learning method, which has the advantages of both multiple kernel learning and ensemble learning. We thus propose a multiple kernel ensemble learning (MKEL) approach for software defect classification and prediction. Considering the cost of risk in software defect prediction, we design a new sample weight vector updating strategy to reduce the cost of risk caused by misclassifying defective modules as non-defective ones. We employ the widely used NASA MDP datasets as test data to evaluate the performance of all compared methods; experimental results show that MKEL outperforms several representative state-of-the-art defect prediction methods.     

面向数据流的多任务多核在线学习算法

多任务多核学习已逐渐成为在线学习算法研究的热点。对于数据流的处理,现有的在线学习算法在准确性上有一定的欠缺,因此提出一种新的多任务多核在线学习模型用于提高数据流预测的准确性。在保持多任务多核学习的基础上,将其扩展到在线学习中,从而得到一个新的在线学习算法;同时为输入数据保持一定大小的数据窗口,用较小空间换取数据的完整性。实验部分对核函数的选取以及训练样本集的大小进行了较为详细的分析,通过对UCI数据和实际的机场客流量数据进行分析,很好地保障了流数据处理的准确性及实时性,有一定的实际应用价值。     

A set of new Chebyshev kernel functions for support vector machine pattern classification

In this study, we introduce a set of new kernel functions derived from the generalized Chebyshev polynomials. The proposed generalized Chebyshev polynomials allow us to derive different kernel functions. By using these polynomial functions, we generalize recently introduced Chebyshev kernel function for vector inputs and, as a result, we obtain a robust set of kernel functions for Support Vector Machine (SVM) classification. Thus in this study, besides clarifying how to apply the Chebyshev kernel functions on vector inputs, we also increase the generalization capability of the previously proposed Chebyshev kernels and show how to derive new kernel functions by using the generalized Chebyshev polynomials. The proposed set of kernel functions provides competitive performance when compared to all other common kernel functions on average for the simulation datasets. The results indicate that they can be used as a good alternative to other common kernel functions for SVM classification in order to obtain better accuracy. Moreover, test results show that the generalized Chebyshev kernel approaches to the minimum support vector number for classification in general.     

Edit distance-based kernel functions for structural pattern classification

A common approach in structural pattern classification is to define a dissimilarity measure on patterns and apply a distance-based nearest-neighbor classifier. In this paper, we introduce an alternative method for classification using kernel functions based on edit distance. The proposed approach is applicable to both string and graph representations of patterns. By means of the kernel functions introduced in this paper, string and graph classification can be performed in an implicit vector space using powerful statistical algorithms. The validity of the kernel method cannot be established for edit distance in general. However, by evaluating theoretical criteria we show that the kernel functions are nevertheless suitable for classification, and experiments on various string and graph datasets clearly demonstrate that nearest-neighbor classifiers can be outperformed by support vector machines using the proposed kernel functions.     

多标记数据特征提取方法的核改进

针对多标记数据特征提取方法中输出核函数没有准确刻画标记间的相关性的问题,在充分度量标记间相关性的基础上,提出了两种新的输出核函数构造方法。第一种方法首先将多标记数据转化为单标记数据,并使用标记集合来刻画标记间的相关性;然后从损失函数的角度出发定义新的输出核函数。第二种方法是利用互信息来度量标记间的两两相关性,在此基础上进一步构造新的输出核函数。3个多标记数据集上2种分类器的实验结果表明,与原有核函数对应的多标记特征提取方法相比,基于损失函数的输出核函数对应的特征提取方法性能最好,5个评价指标的性能平均提高了10%左右, 尤其在Yeast数据集上,Coverage指标下降幅度达到了30%左右;基于互信息的输出核函数次之,性能平均提高了5%左右。实验结果表明,基于新的输出核函数的特征提取方法能够更加有效地提取特征,并进一步简化分类器的学习过程,提高分类器的泛化性能。     

Kernel-based linear classification on categorical data

Kernel-based methods have been widely investigated in the soft-computing community. However, they focus mainly on numeric data. In this paper, we propose a novel method for kernel learning on categorical data, and show how the method can be used to derive effective classifiers for linear classification. Based on kernel density estimation for categorical attributes, three popular classification methods, i.e., Naive Bayes, nearest neighbor and prototype-based classification, are effectively extended to classify categorical data. We also propose two data-driven approaches to the bandwidth selection problem, with one aimed at minimizing the mean squared error of the kernel estimate and the other endeavored to attribute weights optimization. Theoretical analysis indicates that, as in the numeric case, kernel learning of categorical attributes is capable to make the classes to be more separable, resulting in outstanding performances of the new classifiers on various real-world data sets.