Preconditioned Conjugate Gradient Method and Generalized Successive Overrelaxation Method for the Least Squares Problems

In a recent paper [4], Li et al . gave a generalized successive overrelaxation (GSOR) method for the least squares problems. In this paper, the connection between the GSOR method and the preconditioned conjugate gradient (PCG) method for the normal equations is investigated. It is shown that the PCG method is at least as fast as the GSOR method. Numerical examples demonstrates that the PCG method is much faster than the GSOR method.     

A Layer-by-Layer Least Squares based Recurrent Networks Training Algorithm: Stalling and Escape

The limitations of the least squares based training algorithm is dominated by stalling problem and evaluation error by transformation matrix to obtain an unacceptable solution. This paper presents a new approach for the recurrent networks training algorithm based upon the Layer-by-Layer Least Squares based algorithm to overcome the aforementioned problems. In accordance with our proposed algorithm, all the weights are evaluated by the least squares method without the evaluation of transformation matrix to speed up the rate of convergence. A probabilistic mechanism, based upon the modified weights updated equations, is introduced to eliminate the stalling problem experienced by the pure least squares type computation. As a result, the merits of the proposed algorithm are capable of providing an ability of escaping from local minima to obtain a good optimal solution and still maintaining the characteristic of fast convergence.     

基于最小二乘支持向量机的函数拟合的研究

函数拟合通常要在有限的训练样本下对函数变量之间的关系做出预测,在实践中由于训练样本有限,并且训练样本本身存在噪音和孤立点,用传统的方法进行函数拟合的结果往往不能满足要求.本文主要利用最小二乘支持向量机对函数进行拟合.首先介绍了最小二乘支持向量机的工作原理,并对参数选择方法进行了研究,然后通过仿真实验对利用最小二乘支持向量机进行函数拟合的效果加以对比说明.     

Fictitious Domain Approach Via Lagrange Multipliers with Least Squares Spectral Element Method

We consider the application of Fictitious Domain approach combined with Least Squares Spectral Elements for the numerical solution of partial differential equations. Fictitious Domain methods allow problems formulated on a complicated shaped domain Ω to be solved on a simpler domain Π containing Ω. Least Squares Spectral Element Method has been used to develop the discrete model, as this scheme combines the generality of finite element methods with the accuracy of spectral methods. Moreover the least squares methods have theoretical and computational advantages in the algorithmic design and implementation. This paper presents the formulation and validation of the Fictitious Domain/Least Squares Spectral Element approach. The convergence of the relative energy norm η is verified computing smooth solutions to two-dimensional first and second-order differential equations, demonstrating the predictive capability of the proposed formulation.     

Simulating free surface problems using Discrete Least Squares Meshless method

A Discrete Least Squares Meshless (DLSM) method is presented here for the simulation of incompressible free surface flows. The governing equations of the mass and momentum conservations are solved in a Lagrangian form using a pressure projection method. Since there are no particles in the outer region of the free surface, the particle density will drop significantly. Free surfaces are, therefore, resolved by tracking particles with highly reduced density. A fully least squares approach is used in both function approximation and the discretization of the governing differential equations in space. The meshless shape functions are derived using the Moving Least Squares (MLS) method of function approximation. The discretized equations are obtained via a discrete least squares method in which the sum of the squared residuals are minimized with respect to unknown nodal parameters. The method enjoys the advantage of producing symmetric, positive definite matrixes for the cases considered. The method can be viewed as a truly meshless method since it does not need any mesh for both field variable approximation and the construction of system matrices. Two free surface problems namely dam break and evolution of a drop with an initial known velocity are solved to test the accuracy of the proposed method. The results show the ability of the proposed method to solve complex fluid dynamic problems with moving free surface boundaries.     

改进最小二乘变点识别法在负荷分解的应用

随着全世界正进行的大规模智能电表的推广安装,使用非侵入式负荷监测分解方法,总电能消耗分解为单独设备的消耗,成为最近的研究热点。而变点识别是负荷分解方法中的第一步。精确的变点检测为后续提取特征以及识别负荷,打下了坚实的基础。提出了一种基于均值变点模型的识别算法,通过滑动窗口,利用最小二乘法计算目标函数,以确定变点个数。最后,提出假设检验,来验证变点检测的准确性。它能根据相关信号准确检测到负荷投切等引起的电气量变化、发生时刻等重要信息,并记录下来,然后为后续的负荷识别和分解提供保障。最后以某商业写字楼为例,通过测量该商业部分用电负荷数据,从而验证了该算法的可行性。     

基于动态遗忘因子最小二乘与EKF的电池SOC估计

电池荷电状态SOC（State Of Charge）作为电池管理系统中尤为重要的一部分,其准确估计成为锂离子电池研究的重点。为了提高动态工况下的SOC估计精度,对锂离子电池等效模型进行分析,基于AIC（赤池信息）准则确定二阶RC电路为等效电路模型,使用递推最小二乘算法对模型参数进行在线辨识,为提高辨识精度,提出了改进带动态遗忘因子递推最小二乘算法,对算法加入遗忘因子,通过电压结果误差实时动态调整算法遗忘因子取值。将递推最小二乘算法和含动态遗忘因子最小二乘算法分别与扩展卡尔曼滤波（EKF）算法进行SOC联合估计,并对比其预测效果,结果表明含有动态遗忘因子最小二乘与EKF联合估计模型具有更高的精度和鲁棒性。     

局部约束加强的最小二乘回归子空间聚类*

针对最小二乘回归子空间聚类算法存在的数据局部相关性信息缺失、系数矩阵稀疏性不足的缺点,提出局部约束加强的最小二乘回归子空间聚类算法.在原始的最小二乘回归子空间聚类算法的基础上加入数据局部相关性约束,使表示系数矩阵的块对角性质更明显.同时,提出相似度矩阵构造方法,有效提高类内相似度,降低类间相似度.实验表明文中算法可以有效提高聚类的精确度,从而验证算法有效可行.     

最小二乘半监督支持张量机学习算法*

支持张量机(STM)受限于迭代操作,训练时间较长.针对这一缺点,改进STM的目标规划,将训练过程由解决一组二次规划改为计算线性方程组,并引入直推式的思想解决半监督问题,提出最小二乘半监督支持张量机学习算法.在人脸识别和时间序列分类上对比文中算法与传统算法,实验证明文中算法不仅减少运算时间,而且提高识别率.     

一种训练前馈神经网络的局部线性化最小二乘算法

提出了一种新的训练前馈神经网络的局部线性化最小二乘算法。新算法将整个网络的权值训练看作非线性系统的参数识别，并将非线性系统的全局参数识别转化为一系列局部子系统的参数识别，最终将局部子系统的参数识别转化为线性化的最小二乘问题。     

基于GPU的最小二乘蒙特卡罗算法期权定价

期权是以金融产品作为行权品种的交易合约。随着期权交易规模和交易量的迅速增长,期权定价的计算量越来越大,在传统CPU平台上对期权进行定价变得越来越困难。图形处理器（GPU）平台的出现和发展为解决期权定价计算提供了解决方案。在GPU上使用最小二乘蒙特卡罗算法（Least Squares Monte Carlo,LSM）实现了对一维和四维美式期权定价计算：首先利用CURAND库产生大量随机数,然后并行化期权标的价格变化路径,最后对最小二乘法和贴现定价进行并行化。为提高GPU平台上LSM方法的计算效率,对整个过程进行了优化。实际测试结果表明,在CPU+GPU上实现一维和四维美式期权定价相对CPU平台的加速比最高分别达到20.275和47.538,且比其他文献的方法整体性能有较大的提升。     

基于PLS分析的高压绝缘子污秽等级判定

针对污秽绝缘子红外热像特征数据具有多重相关性的特点,提出基于PLS(Partial Least Squares, PLS)回归分析的高压绝缘子污秽等级判定方法。在最大限度保留原有数据信息的前提下,建立起高压绝缘子污秽特征量与污秽等级之间的PLS回归模型方程,通过对回归模型方程进行变量投影重要性指标分析,可以得到各个特征量对污秽等级判定结果的影响程度。此方法有效解决了自变量之间的多重相关性问题,量化了污秽特征量与污秽等级之间的关系。测试结果表明,将PLS回归分析应用于高压绝缘子污秽等级的判定,科学可靠,准确率高,具有较强的实用性。     

稀疏结构化最小二乘双支持向量回归机

最小二乘双支持向量回归机（LSTSVR）通过引入最小二乘损失将双支持向量回归机（TSVR）中的二次规划问题简化为两个线性方程组的求解,从而大大减少了训练时间。然而,LSTSVR最小化基于最小二乘损失的经验风险易导致以下不足：（1）“过学习”问题;（2）模型的解缺乏稀疏性,难以训练大规模数据。针对（1）,提出结构化最小二乘双支持向量回归机（S-LSTSVR）以提升模型的泛化能力;针对（2）,进一步利用不完全Choesky分解对核矩阵进行低秩近似,给出求解S-LSTSVR的稀疏算法SS-LSTSVR,使模型能有效地训练大规模数据。人工数据和UCI数据集中的实验证明SS-LSTSVR不但可以避免“过学习”,而且能够高效地解决大规模训练问题。     

Relations between Bias-Eliminating Least Squares, the Frisch scheme and Extended Compensated Least Squares methods for identifying errors-in-variables systems

There are many methods for identifying errors-in-variables systems. Among them Bias-Eliminating Least Squares (BELS), the Frisch scheme and Extended Compensated Least Squares (ECLS) methods are attractive approaches because of their simplicity and good estimation accuracy. These three methods are all based on a Bias-Compensated Least-Squares (BCLS) principle. In this paper, the relationships between them are considered. In particular, the set of nonlinear equations utilized in these three methods are proved to be equivalent under different noise conditions also for finite samples. It is shown that BELS, Frisch and ECLS methods have the same asymptotic estimation accuracy providing the same extended vector is used.     

A Dynamic Method for Weighted Linear Least Squares Problems

A new method for solving the weighted linear least squares problems with full rank is proposed. Based on the theory of Liapunov's stability, the method associates a dynamic system with a weighted linear least squares problem, whose solution we are interested in and integrates the former numerically by an A-stable numerical method. The numerical tests suggest that the new method is more than comparative with current conventional techniques based on the normal equations. Received August 4, 2000; revised August 29, 2001 Published online April 25, 2002     

合作SCC系统的总LS信号检测方法

为了克服已有信道估计算法无法及时跟踪信道变化的缺陷,在合作OFDM系统中引入了总LS（Total Least Squares,TLS）信号检测方法来实现信道状态信息估计。TLS方法同时考虑了信道噪声和信道时变特性,能够对信道和信号的变化同时进行跟踪估计。因为充分考虑了信道的时变性,且复杂度较低、收敛速度较快,所以在高速移动通信环境下,TLS方法能够很好地估计信道信息。仿真结果表明,与传统的LS法和ML法相比,该算法在改善误码性能方面优势明显。     

移动最小二乘法在多功能传感器数据重构中的应用

针对传统最小二乘法全局拟合的局限性, 将一种新型的数值算法---移动最小二乘法应用于非线性多功能传感器的信号重构. 通过详细研究插值函数的构造方法及性质, 合理地选取基函数和权函数, 求出试函数的系数, 进而得到信号的重构值. 详细分析了基函数维数、影响域节点数及权函数因子对计算结果的影响, 并对最小二乘法以及移动最小二乘法的重构数据进行了对比, 重构的相对误差分别小于 15.3 % 和 1.03 %, 结果表明移动最小二乘法更适合非线性曲面拟合, 且适当地增加基函数维数或影响域节点数可以进一步提高数据重构的精度.     

最小二乘支持向量机算法研究

1 引言支持向量机(SVM,Support Vector Machines)是基于结构风险最小化的统计学习方法,它具有完备的统计学习理论基础和出色的学习性能,在模式识别和函数估计中得到了有效的应用(Vapnik,1995,1998)。支持向量机方法一方面通过把数据映射到高维空间,解决原始空间中数据线性不可分问题;另一方面,通过构造最优分类超平面进行数据分类。神经网络通过基于梯度迭代的方法进行数据学习,容易陷入局部最小值,支持向量机是通过解决一个二次规划问题,来获得     

基于假设检验及异常点剔除的稳健LS-SVM回归

提出一种稳健的LS-SVM回归算法。该算法建立在异常样本逐步剔除的框架上,每次循环中选择误差最大的样本加以考察,然后使用统计假设检验方法对其进行诊断。若样本被诊断为异常样本,则将其剔除,并重新训练LS-SVM,为下一轮的异常点诊断和剔除提供更准确的信息。同时为了减少运算复杂度,我们还将减量学习引入到算法的重新训练过程中,从而保证算法的附加复杂度不超过O(N³)。仿真数据集和实际数据集上的详细实验证实该算法的优越性,并提供一种使用该算法建立异常样本检测器的思路。     

基于最小二乘算法的最优适应控制器

采用"输入匹配"的方法,建立了"一步超前"最小二乘算法,得以参数估计的收敛速度. 证明了闭环适应系统是全局稳定的,且适应控制收敛于"一步超前"最优控制.