最小二乘隐空间支持向量机

在隐空间中采用最小二乘损失函数,提出了最小二乘隐空间支持向量机（LSHSSVMs）．同隐空间支持向量机（HSSVMs）一样,最小二乘隐空间支持向量机不需要核函数满足正定条件,从而扩展了支持向量机核函数的选择范围．由于采用了最小二乘损失函数,最小二乘隐空问支持向量机产生的优化问题为无约束凸二次规划,这比隐空间支持向量机产生的约束凸二次规划更易求解．仿真实验结果表明所提算法在计算时间和推广能力上较隐空间支持向量机存在一定的优势．     

KPCA-LSSVM建模方法及在钢材淬透性中的应用研究

通过等式约束条件修改普通的支持向量机可以得到最小二乘支持向量机,不需要再次求解复杂的二次规划问题,提出了利用核主元分析进行特征提取,在高维特征空间中计算主元,降低样本的维数,然后用最小二乘支持向量机进行建模．仿真结果表明了该方法的有效性和优越性．     

一类非线性系统基于最小二乘支持向量机的自适应H∞控制

提出一类非线性不确定系统基于最小二乘支持向量机的白适应H∞控制方法．该方法基于最小二乘支持向量机估计对象的未知非线性函数,并给出了最小二乘支持向量机权向量和偏移值的在线学习规则．引入H∞控制器用于减弱外部干扰及最小二乘支持向量机近似误差对输出误差的影响．利用李亚普诺夫理论证明了整个闭环系统一致最终有界稳定．仿真结果表明了该方法的有效性．     

支持向量机的多层动态自适应参数优化

首先提出了基于多层动态自适应搜索技术的最小二乘支持向量机参数优化方法，然后采用最小二乘支持向量机对典型非线性控制系统的辨识进行了研究．辨识结果表明，最小二乘支持向量机可以用于非线性控制系统辨识，多层动态自适应搜索方法确定了最优支持向量机参数，从而获得精确的非线性控制系统辨识结果．     

最小二乘Littlewood-Paley小波支持向量机

基于小波分解理论和支持向量机核函数的条件，提出了一种多维允许支持向量核函数——Littlewood-Paley小波核函数．该核函数不仅具有平移正交性，而且可以以其正交性逼近二次可积空间上的任意曲线，从而提升了支持向量机的泛化性能．在Littlewood-Paley小波函数作为支持向量核函数的基础上，提出了最小二乘Littlewood-Paley小波支持向量机(LS-LPWSVM)．实验结果表明，LS-LPWSVM在同等条件下比最小二乘支持向量机的学习精度要高，因而更适用于复杂函数的学习问题．     

一种新型的多元分类支持向量机

最小二乘支持向量机采用最小二乘线性系统代替传统的支持向量机采用二次规划方法解决模式识别问题。该文详细推理和分析了二元分类最小二乘支持向量机算法，构建了多元分类最小二乘支持向量机，并通过典型样本进行测试，结果表明采用多元分类最小二乘支持向量机进行模式识别是有效、可行的。     

一种稀疏最小二乘支持向量分类机

一般的支持向量分类机需要求解二次规划问题,最小二乘支持向量机只需求解一个线性方程组,但其缺乏稀疏性.为了改进最小二乘支持向量分类机,本文结合中心距离比值及增量学习的思想提出一种基于预选、筛选支持向量的稀疏最小二乘支持向量机.该方法既能弥补最小二乘向量机的稀疏性,减少计算机的存储量和计算量,加快最小二乘支持向量机的训练速度和决策速度,又能对非均衡训练数据造成的分类面的偏移进行纠正,还不影响最小二乘支持向量机的分类能力.3组实验结果也证实了这一点.     

可保证分类性能的最小二乘支持向量机

当前支持向量机是分类研究与应用的一个热点。提出了一个新的最小二乘支持向量机算法,该算法向最小二乘支持向量机（LS-SVM）优化模型中融入了类内散度（VSLSVM）思想,即用优化准则Min w′Mw对原LS-SVM进行重组合,w为对应LS-SVM中的权向量,M是类内散度矩阵。提出的方法仅仅需要求解一个线性系统而不是凸规划问题,实验主要对SVM和Suykens等人的方法进行了比较,并验证了提出的算法的有效性。     

基于边界近邻的最小二乘支持向量机实现

最小二乘支持向量机采用最小二乘线性系统代替传统的支持向量即采用二次规划方法解决模式识别问题,能够有效地减少计算的复杂性.但最小二乘支持向量机失去了对支持向量的稀疏性.文中提出了一种基于边界近邻的最小二乘支持向量机,采用寻找边界近邻的方法对训练样本进行修剪,以减少了支持向量的数目.将边界近邻最小二乘支持向量机用来解决由1-a-r(one-against-rest)方法构造的支持向量机分类问题,有效地克服了用1-a-r(one-against-rest)方法构造的支持向量机分类器训练速度慢、计算资源需求比较大、存在拒分区域等缺点.实验结果表明,采用边界近邻最小二乘支持向量机分类器,识别精度和识别速度都得到了提高.     

基于边界近邻的最小二乘支持向量机实现

最小二乘支持向量机采用最小二乘线性系统代替传统的支持向量即采用二次规划方法解决模式识别问题,能够有效地减少计算的复杂性。但最小二乘支持向量机失去了对支持向量的稀疏性。文中提出了一种基于边界近邻的最小二乘支持向量机,采用寻找边界近邻的方法对训练样本进行修剪,以减少了支持向量的数目。将边界近邻最小二乘支持向量机用来解决由1-a-r(one-against-rest)方法构造的支持向量机分类问题,有效地克服了用1-a-r(one-against-rest)方法构造的支持向量机分类器训练速度慢、计算资源需求比较大、存在拒分区域等缺点。实验结果表明,采用边界近邻最小二乘支持向量机分类器,识别精度和识别速度都得到了提高。     

Multiplicative updates for nonnegative quadratic programming

Many problems in neural computation and statistical learning involve optimizations with nonnegativity constraints. In this article, we study convex problems in quadratic programming where the optimization is confined to an axis-aligned region in the nonnegative orthant. For these problems, we derive multiplicative updates that improve the value of the objective function at each iteration and converge monotonically to the global minimum. The updates have a simple closed form and do not involve any heuristics or free parameters that must be tuned to ensure convergence. Despite their simplicity, they differ strikingly in form from other multiplicative updates used in machine learning. We provide complete proofs of convergence for these updates and describe their application to problems in signal processing and pattern recognition.     

基于乘性规则的支持向量域分类器

该文提出了一种基于支持向量域描述(SVDD)的学习分类器．在两类样本分类中，该算法在训练时通过对1类样本的描述求取包含1类样本的球形边界．然后通过该边界对两类样本数据进行分类，并且在求取边界的优化问题中，采用乘性规则来直接求取Lagrange乘子，而不是用传统的二次优化方法．该文所获得的学习算法和支持向量机(SVM)与序列最小优化(SMO)算法相比，不仅降低了样本的采集代价，而且在优化速度上有了很大提高．通过CBCL人脸库的仿真实验．将该算法和SVM、SOM算法的实验结果进行对比，说明了该学习算法的有效性．     

Group consensus algorithms based on preference relations

In many group decision-making situations, decision makers’ preferences for alternatives are expressed in preference relations (including fuzzy preference relations and multiplicative preference relations). An important step in the process of aggregating preference relations, is to determine the importance weight of each preference relation. In this paper, we develop a number of goal programming models and quadratic programming models based on the idea of maximizing group consensus. Our models can be used to derive the importance weights of fuzzy preference relations and multiplicative preference relations. We further develop iterative algorithms for reaching acceptable levels of consensus in group decision making based on fuzzy preference relations or multiplicative preference relations. Finally, we include an illustrative example.     

L-Broyden methods: a generalization of the L-BFGS method to the limited-memory Broyden family

The paper derives a multiplicative update equation for the convex Broyden family of quasi-Newton (QN) updates. The well-known multiplicative Broyden-fletcher-Goldfarb-Shanno (BFGS) update is a special case of this. Using a self-scaling parameter, the formula is extended to the SR1 update. It is shown that for each Broyden update, a pair of multiplicative update formulae can be defined (coincident in the case of Davidon-fletcher-Powell (DFP)). The multiplicative updates are used to define limited-memory QN algorithms, denoted the L-Broyden methods, of which L-BFGS is a special case. Numerical results show that the L-Broyden methods are competitive with extensions of the variable storage conjugate gradients limited memory QN method to other Broyden updates, but that L-BFGS with Shanno scaling remains the most efficient and reliable method in the L-Broyden family.     

The multiplicative update rule for an extension of the iterative constrained endmembers algorithm

Based on the multiplicative update rule of nonnegative matrix factorization (NMF), we add the nonnegative constraint of endmember matrix to the objective function of iterative constrained endmembers (ICE), and propose a simple multiplicative update rule for ICE, named NMF-ICE. Our method avoids the use of quadratic programming method in ICE, therefore can greatly improve the computational efficiency. Experiments using both simulated and real hyperspectral data show that NMF-ICE is effective in endmember generation.     

On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is useful to find basis information of nonnegative data. Currently, multiplicative updates are a simple and popular way to find the factorization. However, for the common NMF approach of minimizing the Euclidean distance between approximate and true values, no proof has shown that multiplicative updates converge to a stationary point of the NMF optimization problem. Stationarity is important as it is a necessary condition of a local minimum. This paper discusses the difficulty of proving the convergence. We propose slight modifications of existing updates and prove their convergence. Techniques invented in this paper may be applied to prove the convergence for other bound-constrained optimization problems.     

Proportioning with second-order information for model predictive control

We propose an algorithm for the effective solution of quadratic programming (QP) problems arising from model predictive control (MPC). MPC is a modern multivariable control method which gives the solution for a QP problem at each sample instant. Our algorithm combines the active-set strategy with the proportioning test to decide when to leave the actual active set. For the minimization in the face, we use a direct solver implemented by the Cholesky factors updates. The performance of the algorithm is illustrated by numerical experiments, and the results are compared with the state-of-the-art solvers on benchmarks from MPC.     

Optimal filtering for polynomial system states with polynomial multiplicative noise

In this paper, the optimal filtering problem for polynomial system states with polynomial multiplicative noise over linear observations is treated proceeding from the general expression for the stochastic Ito differential of the optimal estimate and the error variance. As a result, the Ito differentials for the optimal estimate and error variance corresponding to the stated filtering problem are first derived. The procedure for obtaining a closed system of the filtering equations for any polynomial state with polynomial multiplicative noise over linear observations is then established, which yields the explicit closed form of the filtering equations in the particular cases of a linear state equation with linear multiplicative noise and a bilinear state equation with bilinear multiplicative noise. In the example, performance of the designed optimal filter is verified for a quadratic state with a quadratic multiplicative noise over linear observations against the optimal filter for a quadratic state with a state‐independent noise and a conventional extended Kalman–Bucy filter. Copyright © 2006 John Wiley & Sons, Ltd.     

一类双线性随机系统M量测不定LQ控制

研究了带有乘性噪声和受扰动观测的离散时间随机系统不定线性二次(Linear quadratic, LQ) 最优输出反馈控制问题. 对此类问题而言,二次成本函数的加权矩阵不定号,并且最优控制具有对偶效果.为在最优性和计算复杂度间 进行折衷,本文采用了一种M量测反馈控制设计方法.基于动态规划方法,将未来的测量结合到当前控制 计算当中的M量测反馈控制可以通过倒向求解一类与原系统维数相同的广义差分Riccati方程(Generalized difference Riccati equation, GDRE)得到.仿真结果 表明本文提出的算法与目前普遍采用的确定等价性方法相比具有优越性.     

求解Minimax优化问题的Newton型算法

Newton法是求解无约束优化问题的最有效的算法,但由于需要计算目标函数的Hesse矩阵计算量大,因此人们大多采用拟Newton(变度量法)求解无约束问题。近些年来,由于自动微分(Automatic Differentiation)技术的提出和计算机速度与内存的不断提高,