支持向量机分类与回归算法的关系研究

基于统计学习理论的支持向量机算法以其优秀的学习性能已广泛用于解决分类与回归问题.分类算法通过求两类样本之间的最大间隔来获得最优分离超平面,其几何意义相当直观,而回归算法的几何意义就不那么直观了.另外,有些适用于分类问题的快速优化算法却不能用于回归算法中.研究了分类与回归算法之间的关系,为快速分类算法应用于回归模型提供了一定的理论依据.     

连续超松弛支持向量机回归算法应用研究

支持向量回归问题的研究,对函数拟合(回归逼近)具有重要的理论和应用意义.借鉴分类问题的有效算法,将其推广到回归问题中来,针对用于分类问题的SOR支持向量机有效算法,提出了SORR支持向量回归算法.在若干不同维数的数据集上,对SORR算法、ASVR算法和LibSVM算法进行数值试验,并进行比较分析.数值实验结果表明,SORR算法是有效的,与当前流行的支持向量机回归算法相比,在回归精度和学习速度上都有一定的优势.     

角分类前向神经网络研究

角分类算法是一类快速分类算法，以其为学习算法的前向神经网络，在信息检索，特别是在线信息检索等领域有着重要的应用．通过对CC4学习算法的分析，揭示了泛化距离在角分类神经网络中的意义．针对文本数据的快速分类要求，提出了新的角分类网络TextCC．为解决数据的多类别判定问题，给出了新的角分类神经网络隐层与输出层之间连接矩阵的学习算法．实验表明，新的角分类神经网络隐层与输出层之间连接矩阵的学习算法有效，TextCC的分类精度教CC4的分类精度显著的提高．     

基于支持向量机分类的回归方法

支持向量机(support vector machine,简称SVM)是一种基于结构风险最小化原理的分类技术,也是一种新的具有很好泛化性能的回归方法.提出了一种将回归问题转化为分类问题的新思想.这种方法具有一定的理论依据,与SVM回归算法相比,其优化问题几何意义清楚明确.     

SORR回归算法在胆固醇测定中的应用

借鉴分类问题的算法,推广到回归问题中去,针对用于分类问题的SOR(successive overrelaxation for support vector)支持向量机算法,提出SORR(successive overrelaxation for support vector regression)支持向量回归算法,并应用于医学上三类血浆脂蛋白(VLDL、LDL、HDL)测定样本中胆固醇的含量。数值实验表明:SORR算法有效,与标准的支持向量回归SVR算法相比,保持了相同的回归精度,提高了学习速度,为临床上测定胆固醇含量提供新的有效方法。     

大样本多源域与小目标域的跨领域快速分类学习

传统的跨领域分类学习一般考虑均衡的单一源域到单一目标域的学习,但在现实世界中数据往往是不平衡的.当用于解决不平衡分类问题时,由于分类器的偏向性,其分类精度、抗噪性能往往有不同程度的下降.为了克服域间不平衡性,提出了一种不平衡多源跨领域分类算法(imbalance multisource classfication on cross-domain learning,IMCCL),该算法依据被众多实验证明有效的"逻辑回归模型"与"后验概率最大法则"构建多个训练域分类器并综合指导目标域的数据分类.为了充分高效利用大样本的源域数据,满足大样本的快速运算,在结合CDdual算法的基础上,提出了IMCCL的快速算法(IMCCL-CDdual).将其应用到文本数据分类与图像识别分类的实验结果表明:该算法具有较高的识别率、快速的识别速度和抗干扰性和领域自适应性.     

一种选择性SER-BagBoosting Trees集成学习研究

集成学习已成为机器学习研究的一大热点.提出了一种综合Bagging和Boosting技术特点,以分类回归树为基学习器构造一种新的相似度指标用于聚类并利用聚类技术和贪婪算法进行选择性集成学习的算法--SER-BagBoosting Trees算法.算法主要应用于回归问题.实验表明,该算法往往比其它算法具有更好的泛化性能和更高的运行效率.     

权核Logistic回归模型的分类和特征选择算法

监督学习情况下,经常遇到样例的维数远远大于样本个数的学习情况。此时,样例中存在许多与样例类标签无关的特征,研究如何同时实现稀疏特征选择并具有更好的分类性能的算法具有优势。提出了基于权核逻辑斯蒂非线性回归模型的分类和特征选择算法。权对角矩阵的对角元素在0到1之间取值,对角元素的取值作为学习参数由最优化过程确定,讨论了提出的快速轮转优化算法。提出的算法在十个实际数据集上进行了测试,实验结果显示,提出的分类算法与L1,L2,Lp正则化逻辑斯蒂模型分类算法比较具有优势。     

基于流形正则化的分类与回归算法及应用

利用流形正则化的思想,围绕半监督学习,提出了一种针对流形正则化的模式分类和回归分析的新算法.该算法基于流形上的正则化项和传统的正则化项相结合的方法,利用支持向量机分类与回归已有的结果,解决半监督学习的分类与回归问题,提高了泛化能力.该算法实现简单,无需调用其他程序.通过数值试验,验证了该算法具有较好的泛化能力,对噪音具有较强的鲁棒性.且在分类问题上,该算法在输入极少数有标签样本时,也能保持较好的分类效果;在回归问题上,也具有较好的学习精度,尤其在输入带有噪音的流形数据上时,表现就更为突出.     

基于几何思想的快速支持向量机算法

为了快速地进行分类,根据几何思想来训练支持向量机,提出了一种快速而简单的支持向量机训练算法——几何快速算法。由于支持向量机的最优分类面只由支持向量决定,因此只要找出两类样本中所有支持向量,那么最优分类面就可以完全确定。该新的算法根据两类样本的几何分布,先从两类样本的最近点开始;然后通过不断地寻找违反KKT条件的样本点来找出支持向量;最后确定最优分类面。为了验证新算法的有效性,分别利用两个公共数据库,对新算法与SMO算法及DIRECTSVM算法进行了实验对比,实验结果显示,新算法的分类精度虽与其他两个方法相当,但新算法的运算速度明显比其他两个算法快。     

A geometric approach to Support Vector Machine (SVM) classification

The geometric framework for the support vector machine (SVM) classification problem provides an intuitive ground for the understanding and the application of geometric optimization algorithms, leading to practical solutions of real world classification problems. In this work, the notion of "reduced convex hull" is employed and supported by a set of new theoretical results. These results allow existing geometric algorithms to be directly and practically applied to solve not only separable, but also nonseparable classification problems both accurately and efficiently. As a practical application of the new theoretical results, a known geometric algorithm has been employed and transformed accordingly to solve nonseparable problems successfully.     

一个分子模型的生成及显示系统

本文介绍了一个实现分子模型的生成及显示的软件包,该软件包能对三维分子模型(空间填充模型)进行动态显示,随时进行几何变换及开窗变换,交互性能好.文章详细介绍了该软件包所采用的新的快速的算法,算法包括线框图显示及真实感图显示,与国外同类算法相比,研制的算法速度非常快.     

基于等高测度矩阵辨识不规则封闭图形

针对现有算法对不规则封闭图形轮廓特征辨识效率不高的问题,提出一种基于自定义等高测度矩阵的快速辨识算法.自定义等高测度矩阵从轮廓形态上最优地描述了不规则封闭图形的几何特征;基于该矩阵自定义图形单元的偏心率族参数,并将该参数的概率密度函数作为样本分类器设计依据;结合Bayes理论提出了最小误差概率分类器的设计方法.实际应用验证表明,该算法对轮廓特征的表述具有较好的平移、尺度不变性,且具有30倍于傅里叶描述子的运算效率.     

Connection between trinomial trees and finite difference methods for option pricing with state-dependent switching rates

Tree approaches (binomial or trinomial trees) are very popularly used in finance industry to price financial derivatives. Such popularity stems from their simplicity and clear financial interpretation of the methodology. On the other hand, PDE (partial differential equation) approaches, with which standard numerical procedures such as the finite difference method (FDM), are characterized with the wealth of existing theory, algorithms and numerical software that can be applied to solve the problem. For a simple geometric Brownian motion model, the connection between these two approaches is studied, but it is lower-order equivalence. Moreover such a connection for a regime-switching model is not so clear at all. This paper presents the high-order equivalence between the two for regime-switching models. Moreover the convergence rates of trinomial trees for pricing options with state-dependent switching rates are first proved using the theory of the FDMs.     

线性支持向量机优化问题的极大熵方法

支特向量机是一种新的机器学习方法，已成功地应用于模式分类、回归分析和密度估计等问题中．本文依据统计学习理论和最优化理论建立了线性支特向量机的无约束优化模型，并给出了一种有效的近似解法一极大熵方法,为求解支持向量机优化问题提供了一种新途径,本文方法特别易于计算机实现。数值实验结果表明了模型和算法的可行性和有效性．     

Ensemble of Efficient Minimal Learning Machines for Classification and Regression

Minimal Learning Machine (MLM) is a recently proposed supervised learning algorithm with performance comparable to most state-of-the-art machine learning methods. In this work, we propose ensemble methods for classification and regression using MLMs. The goal of ensemble strategies is to produce more robust and accurate models when compared to a single classifier or regression model. Despite its successful application, MLM employs a computationally intensive optimization problem as part of its test procedure (out-of-sample data estimation). This becomes even more noticeable in the context of ensemble learning, where multiple models are used. Aiming to provide fast alternatives to the standard MLM, we also propose the Nearest Neighbor Minimal Learning Machine and the Cubic Equation Minimal Learning Machine to cope with classification and single-output regression problems, respectively. The experimental assessment conducted on real-world datasets reports that ensemble of fast MLMs perform comparably or superiorly to reference machine learning algorithms.     

Nonlinear Dimension Reduction with Kernel Sliced Inverse Regression

Sliced inverse regression (SIR) is a renowned dimension reduction method for finding an effective low-dimensional linear subspace. Like many other linear methods, SIR can be extended to nonlinear setting via the ldquokernel trick.rdquo The main purpose of this paper is two-fold. We build kernel SIR in a reproducing kernel Hilbert space rigorously for a more intuitive model explanation and theoretical development. The second focus is on the implementation algorithm of kernel SIR for fast computation and numerical stability. We adopt a low-rank approximation to approximate the huge and dense full kernel covariance matrix and a reduced singular value decomposition technique for extracting kernel SIR directions. We also explore kernel SIR's ability to combine with other linear learning algorithms for classification and regression including multiresponse regression. Numerical experiments show that kernel SIR is an effective kernel tool for nonlinear dimension reduction and it can easily combine with other linear algorithms to form a powerful toolkit for nonlinear data analysis.     

A simple decomposition algorithm for support vector machines with polynomial-time convergence

Support vector machines (SVMs) are a new and important tool in data classification. Recently much attention has been devoted to large scale data classifications where decomposition methods for SVMs play an important role.So far, several decomposition algorithms for SVMs have been proposed and applied in practice. The algorithms proposed recently and based on rate certifying pair/set provide very attractive features compared with many other decomposition algorithms. They converge not only with finite termination but also in polynomial time. However, it is difficult to reach a good balance between low computational cost and fast convergence.In this paper, we propose a new simple decomposition algorithm based on a new philosophy on working set selection. It has been proven that the working set selected by the new algorithm is a rate certifying set. Further, compared with the existing algorithms based on rate certifying pair/set, our algorithm provides a very good feature in combination of lower computational complexity and faster convergence.