共查询到20条相似文献,搜索用时 781 毫秒
1.
序贯最小优化的改进算法 总被引:26,自引:0,他引:26
序贯最小优化(sequential minimal optimization,简称SMO)算法是目前解决大量数据下支持向量机(support vector machine,简称SVM)训练问题的一种十分有效的方法,但是确定工作集的可行方向策略会降低缓存的效率.给出了SMO的一种可行方向法的解释,进而提出了一种收益代价平衡的工作集选择方法,综合考虑与工作集相关的目标函数的下降量和计算代价,以提高缓存的效率.实验结果表明,该方法可以提高SMO算法的性能,缩短SVM分类器的训练时间,特别适用于样本较多、支持向量较多、非有界支持向量较多的情况. 相似文献
2.
改进工作集选择策略的序贯最小优化算法 总被引:3,自引:0,他引:3
针对标准序贯最小优化(sequential minimal optimization, SMO)算法采用可行方向工作集选择策略所带来的缓存命中率低下问题,给出了SMO类型算法每次迭代所带来的目标函数下降量的二阶表达式,并据此提出了一种改进的工作集选择策略.新策略综合考虑算法收敛所需的迭代次数及缓存效率,从总体上减少了核函数计算次数,因此极大提高了训练效率,并且,它在理论上具有严格的收敛保障.实验结果表明,核函数越复杂,样本维度越高,缓存容量相对训练样本的规模越小,改进工作集选择策略的SMO算法相较于标准SMO算法的性能提高就越显著. 相似文献
3.
近年来,随着序列最小优化分类算法SMO等一系列快速算法的推出,支持向量机在自动文本分类研究领域取得了很大的成功。大多数文本分类问题是线性可分的,使用线性核函数的SMO算法能够取得非常好的分类效果。但是文本向量是一种非常稀疏的向量,采用线性核函数的SMO算法对噪声样本非常敏感,容易产生发散的问题。文章分析证明了噪声如何影响SMO算法收敛性。为了解决训练样本中噪声样本影响SMO算法收敛的问题,设计了一个消除噪声样本的算法,取得了非常好的效果。 相似文献
4.
针对基于递推下降法的多输出支持向量回归算法在模型参数拟合过程中收敛速度慢、预测精度低的情况,使用一种基于秩2校正规则且具有二阶收敛速度的修正拟牛顿算法(BFGS)进行多输出支持向量回归算法的模型参数拟合,同时为了保证模型迭代过程中的下降量和全局收敛性,应用非精确线性搜索技术确定步长因子。通过分析支持向量机(SVM)中核函数的几何结构,构造数据依赖核函数替代传统核函数,生成多输出数据依赖核支持向量回归模型。将模型与基于梯度下降法、修正牛顿法拟合的多输出支持向量回归模型进行对比。实验结果表明,在200个样本下该算法的迭代时间为72.98 s,修正牛顿法的迭代时间为116.34 s,递推下降法的迭代时间为2065.22 s。所提算法能够减少模型迭代时间,具有更快的收敛速度。 相似文献
5.
First and Second Order SMO Algorithms for LS-SVM Classifiers 总被引:1,自引:0,他引:1
Least squares support vector machine (LS-SVM) classifiers have been traditionally trained with conjugate gradient algorithms.
In this work, completing the study by Keerthi et al., we explore the applicability of the SMO algorithm for solving the LS-SVM
problem, by comparing First Order and Second Order working set selections concentrating on the RBF kernel, which is the most
usual choice in practice. It turns out that, considering all the range of possible values of the hyperparameters, Second Order
working set selection is altogether more convenient than First Order. In any case, whichever the selection scheme is, the
number of kernel operations performed by SMO appears to scale quadratically with the number of patterns. Moreover, asymptotic
convergence to the optimum is proved and the rate of convergence is shown to be linear for both selections. 相似文献
6.
MLSVM4--一种多乘子协同优化的SVM快速学习算法 总被引:4,自引:0,他引:4
贯序最小优化(SMO)算法是解决大数据集支持向量机学习问题的一种有效方法,但SMO选择工作集的策略是选择数据集中最违背KKT条件的两个样本,而且还使用了随机函数,使得优化过程具有很大的随机性,影响了学习效率.在多拉格朗日乘子协同优化的通用公式基础上,吸收了Keerthi所提出的SMO修改算法中双阈值的优点,给出了乘子数为4时的一个算法MLSVM4,由于能更加精确地确定待优化样本的拉格朗日乘子值,使得学习收敛速度大大提高,特别是在使用线性核的场合下效果更加明显,在Adult、Web、手写体数字数据集上的实验结果表明,MLSVM4算法速度超过了SMO算法3到42倍. 相似文献
7.
蜘蛛猴算法(Spider Monkey Optimization,SMO)是受蜘蛛猴觅食行为启发提出的一种群集智能优化算法,为增强蜘蛛猴算法的局部搜索性能,提出一种基于动态自适应惯性权重的SMO算法(DWSMO)。通过在惯性权重中引入目标函数值,使得惯性权重随着目标函数值的变化而动态改变,从而减少惯性权重变化的盲目性,有效平衡算法的全局探索能力以及局部开发能力。将改进的蜘蛛猴算法在函数优化问题上进行测试,仿真实验结果表明,改进的蜘蛛猴算法可有效提高函数寻优精度,加快收敛速度,且具有较强的稳定性。 相似文献
8.
将求解SVC模型的算法运用到求解SVR模型中一般要SVR模型的核函数正定且满足Mercer条件,而实际应用中利用几何框架将SVC模型转换成相应的SVR模型时,通常无法保证经转换得到的SVR模型的核函数具有正定性,从而导致SVR模型不是凸规划模型而无法求解。为解决上述问题,本文提出了一种运用扩展的序列最小最优化方法(SMO)来求解基于非正定核的SVR模型,设计了算法中工作集的选择准则,解决了算法中如何选择工作集变量当前的最优值问题。由于该算法不要求核函数具有正定性,从而拓宽了SVR模型核函数的选择范围。实验表明,该算法对基于正定或非正定核的SVR模型都具有很好的泛化性能和回归精度,具有一定的理论意义和实用价值。 相似文献
9.
The generalized LASSO 总被引:6,自引:0,他引:6
In the last few years, the support vector machine (SVM) method has motivated new interest in kernel regression techniques. Although the SVM has been shown to exhibit excellent generalization properties in many experiments, it suffers from several drawbacks, both of a theoretical and a technical nature: the absence of probabilistic outputs, the restriction to Mercer kernels, and the steep growth of the number of support vectors with increasing size of the training set. In this paper, we present a different class of kernel regressors that effectively overcome the above problems. We call this approach generalized LASSO regression. It has a clear probabilistic interpretation, can handle learning sets that are corrupted by outliers, produces extremely sparse solutions, and is capable of dealing with large-scale problems. For regression functionals which can be modeled as iteratively reweighted least-squares (IRLS) problems, we present a highly efficient algorithm with guaranteed global convergence. This defies a unique framework for sparse regression models in the very rich class of IRLS models, including various types of robust regression models and logistic regression. Performance studies for many standard benchmark datasets effectively demonstrate the advantages of this model over related approaches. 相似文献
10.
Improvements to the SMO algorithm for SVM regression 总被引:27,自引:0,他引:27
Shevade S.K. Keerthi S.S. Bhattacharyya C. Murthy K.R.K. 《Neural Networks, IEEE Transactions on》2000,11(5):1188-1193
This paper points out an important source of inefficiency in Smola and Scholkopf's (1998) sequential minimal optimization (SMO) algorithm for support vector machine regression that is caused by the use of a single threshold value. Using clues from the Karush-Kuhn-Tucker conditions for the dual problem, two threshold parameters are employed to derive modifications of SMO for regression. These modified algorithms perform significantly faster than the original SMO on the datasets tried. 相似文献
11.
最小二乘分解算法在车型识别中的应用 总被引:1,自引:0,他引:1
提出一种最小二乘支持向量机的序贯最小分类分解算法.针对最小二乘支持向量机,通过对核函数的相关变换,将二阶的误差信息归结到优化方程的一阶信息中,从而简化运算过程.采用最优函数梯度二阶信息选择工作集,实现最小二乘支持向量机分解算法,提高了算法的收敛性.采用径向基核函数和交叉验证网格搜索的方法验证算法的分类准确性.实验结果表明,提出的分类算法应用于车型识别中,可以得到比其他分类方法更好的分类准确度. 相似文献
12.
SMO algorithm for least-squares SVM formulations 总被引:9,自引:0,他引:9
This article extends the well-known SMO algorithm of support vector machines (SVMs) to least-squares SVM formulations that include LS-SVM classification, kernel ridge regression, and a particular form of regularized kernel Fisher discriminant. The algorithm is shown to be asymptotically convergent. It is also extremely easy to implement. Computational experiments show that the algorithm is fast and scales efficiently (quadratically) as a function of the number of examples. 相似文献
13.
14.
15.
16.
17.
The current state of knowledge regarding nonstationary processes is significantly poorer then in the case of stationary signals. In many applications, signals are treated as stationary only because in this way it is easier to analyze them; in fact, they are nonstationary. Nonstationary processes are undoubtedly more difficult to analyze and their diversity makes application of universal tools impossible. In this paper we propose a new class of generalized regression neural networks working in nonstationary environment. The generalized regession neural networks (GRNN) studied in this paper are able to follow changes of the best model, i.e., time-varying regression functions. The novelty is summarized as follows: 1) We present adaptive GRNN tracking time-varying regression functions. 2) We prove convergence of the GRNN based on general learning theorems presented in Section IV. 3) We design in detail special GRNN based on the Parzen and orthogonal series kernels. In each case we precise conditions ensuring convergence of the GRNN to the best models described by regression function. 4) We investigate speed of convergence of the GRNN and compare performance of specific structures based on the Parzen kernel and orthogonal series kernel. 5) We study various nonstationarities (multiplicative, additive, "scale change," "movable argument") and design in each case the GRNN based on the Parzen kernel and orthogonal series kernel. 相似文献
18.
基于插值的核函数构造 总被引:16,自引:3,他引:16
近年来,统计学习(SLT)和支持向量机(SVM)理论的研究日益受到当前国际机器学习领域的重视.有关核函数的研究则一直是研究的重点.这是因为不同的核函数会导致SVM的泛化能力有很大的不同.如何根据所给数据选择合适的核函数成为人们所关注的核心问题.该文首先指出满足Mercer条件的核函数的具体表达式并非问题关键,在此基础上,该文进一步提出利用散乱数据插值的办法确定特征空间中感兴趣点的内积值以代替传统核函数的一般表达式所起的作用.实验表明该方法不仅能够有效改善支持向量机的设计训练过程中的不确定性,而且泛化能力要优于绝大部分的基于传统核函数的支持向量机. 相似文献
19.
This paper describes an implementation of a client caching system which supports disconnected and partially connected modes of operation while preserving the traditional role of client caching (improved response time) in the fully connected mode. The system is designed to facilitate various server-dependent verification protocols. The need for the heterogeneous server support stems from the fact that the mobile user is likely to access various network services, transfer data between them and use the data in the local unit application programs. The unique feature of this project is its implementation which does not require modification of the native system's kernel source code. The paper shows that an installable file system is a viable mechanism for providing such file system extensions without source code modifications. The results of our preliminary experiments indicate efficiency of the design and the feasibility of the implementation mechanism used. 相似文献