Outliers, inliers and the generalized least trinuned squares estimator in system identification

The least trimmed squares estimator (LTS) is a well known robust estimator in terms of protecting the estimate from the outliers. Its high computational complexity is however a problem in practice. We show that the LTS estimate can be obtained by a simple algorithm with the complexity O( N In N) for large N, where N is the number of measurements. We also show that though the LTS is robust in terms of the outliers, it is sensitive to the inliers. The concept of the inliers is introduced. Moreover, the Generalized Least Trimmed Squares estimator (GLTS) together with its solution are presented that reduces the effect of both the outliers and the inliers.     

On strong consistency of least squares identification algorithms

In this paper almost sure convergence results are derived for least squares identification algorithms. The convergence conditions expressed in terms of the measurable signal model states derived for asymptotically stable signal models and possibly nonstationary processes are in essence the same as those previously given, but are derived more directly. Strong consistency results are derived for the case of signal models with unstable modes and exponential rates of convergence to the unstable modes are demonstrated. These latter convergence results are stronger than those earlier ones in which weak consistency conditions are given and there is also less restriction on the noise disturbances than in earlier theories. The derivations in the paper appeal to martingale convergence theorems and the Toeplitz lemma.     

Outliers, inliers and the generalized least trimmed squares estimator in system identification

The least trimmed squares estimator (LTS) is a well known robust estinaator in terms of protecting the estimatefrom the outliers. Its high computational complexity is however a problem in practice. We show that the LTS estimate can be obtained by a simple algorithm with the complexity O( N In N) for large N, where N is the number of measurements. We also showthat though the LTS is robust in terms of the outliers, it is sensitive to the inliers. The concept of the inliers is introduced. Moreover, the Generalized Least Trimmed Squares estimator (GLTS) together with its solution are presented that reduces the effect of both the outliers and the inliers.     

l1鲁棒辨识:最小二乘算法及试验设计

现有的l^1鲁棒辨识方法依赖于观测数据窗的起始时刻因而不能用来辨识时变系统,针对该问题基于最小二乘法提出了一种l^1鲁棒辨识算法．该算法与观测窗的起始时刻无关,可用于时变系统的辨识．证明了当试验输入为持续激励信号时所提出的算法为本质最优算法,进一步证明了周期持续激励序列为最优试验信号,并给出了辨识误差紧界的计算公式．最后利用提出的算法研究了慢时变系统的l^1鲁棒辨识问题．     

改进的鲁棒总体最小二乘无源定位算法

针对机载无源定位易受异常误差影响的问题,提出一种基于角度信息的鲁棒递推总体最小二乘定位（RRTLS）算法。建立机载无源定位模型,得出总体最小二乘（TLS）解,根据机载定位的实时性、低复杂度要求将其转化为加权递推形式;根据广义M估计原理构建鲁棒TLS极值准则,利用其性质将RRTLS定位问题转化为等价权函数的设计问题;验证了利用残差识别异常误差的合理性,在此基础上建立了等价权函数。仿真结果表明,不存在异常误差时,递推总体最小二乘（RTLS）算法和RRTLS算法均能较好收敛;存在异常误差时,递推最小二乘（RLS）和RTLS定位结果受到扭曲,而RRTLS算法能够获得理想的估值,具有较强的鲁棒性。     

Stochastic adaptive control using a modified least squares algorithm

Recent papers on stochastic adaptive control have established global convergence for algorithms using a stochastic approximation iteration. However, to date, global convergence has not been established for algorithms incorporating a least squares iteration. This paper establishes global convergence for a slightly modified least squares stochastic adaptive control algorithm. It is shown that, with probability one, the algorithm will ensure that the system inputs and outputs are sample mean square bounded and the mean square output tracking error achieves its global minimum possible value for linear feedback control.     

新的正交最小二乘方法

正交最小二乘是一种贪婪算法,采用逐步回归建模,每一步利用搜索算法找到最小化残差的一个回归项。将其拓展为每一步搜索多个最优的回归项,从而得到一种稀疏的回归方法,并将其应用于谐波分量提取中。仿真实验说明,新方法不仅能够较为精确地逐项估计出分量的参数,而且可以对分量个数进行有效的估计。     

一种用于数据校正的最小二乘与准最小二乘组合方法

过程系统的控制与优化要求可靠的过程数据。通过测量得到的过程数据含有随机误差和过失误差,采用数据校正技术可有效地减小过程测量数据的误差,从而提高过程控制与优化的准确性。针对传统基于最小二乘的数据校正方法:和基于准最小二乘的鲁棒数据校正方法:,分析了它们的优缺点,并提出了一种最小二乘与准最小二乘组合方法:。该方法:先采用准最小二乘估计器检测过失误差并剔除,然后再采用最小二乘估计器进行数据校正,可以综合前两种方法:各自的优点,使得数据校正结果:更加准确。将提出最小二乘与准最小二乘组合方法:应用于线性与非线性系统的数据校正中,通过校正结果:的比较说明此方法:的具有较好的过失误差检测能力和较准确的数据校正结果:。最后将此方法:应用于实际过程系统空气分离流程的数据校正中,结果:说明了此方法:的有效性。     

Accuracy analysis of bias-eliminating least squares estimates for errors-in-variables systems

The bias-eliminating least squares (BELS) method is one of the consistent estimators for identifying dynamic errors-in-variables systems. In this paper, we investigate the accuracy properties of the BELS estimates. An explicit expression for the normalized asymptotic covariance matrix of the estimated parameters is derived and supported by some numerical examples.     

基于整体最小二乘法的高分辨率图像重建算法

提出一种用于高分辨率图像重建的整体最小二乘算法。在现有多数重建算法中,假设系统矩阵是精确的而误差主要源于采样图像,但实际上抖动误差也出现在系统矩阵中。该方法能同时最小化这两种误差,采用基于正则化的Rayleigh商来光滑解,用共轭梯度算法来迭代求解该正则化Rayleigh商的最小化函数。实验证明该方法对于抖动系统矩阵是稳定和精确的。     

Decomposition-based recursive least squares identification methods for multivariate pseudo-linear systems using the multi-innovation

This paper studies the parameter estimation algorithms of multivariate pseudo-linear autoregressive systems. A decomposition-based recursive generalised least squares algorithm is deduced for estimating the system parameters by decomposing the multivariate pseudo-linear autoregressive system into two subsystems. In order to further improve the parameter accuracy, a decomposition based multi-innovation recursive generalised least squares algorithm is developed by means of the multi-innovation theory. The simulation results confirm that these two algorithms are effective.     

基于最小二乘和EKF的伪卫星定位算法

空基伪卫星由于自身机动性以及受到诸如气流、压力、温度等外界因素的影响使得其位置存在着偏移。因此,精确确定空基伪卫星的位置是其增强现有导航系统或独立组网进行导航定位的前提。针对扩展Kalman滤波对初值的要求和最小二乘法估计性好的特点,提出了一种混合算法,该算法用逆定位原理建立伪距观测方程组并采用最小二乘法解算出初值,运用扩展Kalman滤波进行定位。仿真表明,混合算法优于最小二乘法,定位精度得到了提高。     

The structured total least squares algorithm research for passive location based on angle information

Based on the constrained total least squares (CTLS) passive location algorithm with bearing-only measurements, in this paper, the same passive location problem is transformed into the structured total least squares (STLS) problem. The solution of the STLS problem for passive location can be obtained using the inverse iteration method. It also expatiates that both the STLS algorithm and the CTLS algorithm have the same location mean squares error under certain condition. Finally, the article presents a kind of location and tracking algorithm for moving target by combining STLS location algorithm with Kalman filter (KF). The efficiency and superiority of the proposed algorithms can be confirmed by computer simulation results.     

A class of small sample nonlinear least squares problems

This paper discusses the least squares fitting of models consisting of a weighted sum of functions, belonging to the same nonlinearly parametric family, to relatively small numbers of observations disturbed by additive errors.Numerical examples show that the number of distinct functions in the solution may, through coincidence of some of the parameters, become smaller than the number of functions in the fitted model, even if the latter number is the true one.An explanation of this phenomenon is given.     

Detection techniques in least squares identification

Parameter estimation schemes based on least squares identification and detection ideas are proposed for ease of computation, reduced numerical difficulties, and bias reduction in the presence of colored noise correlated with the states of the signal generating system. The algorithms are simpler because in the calculations, the state vector is at one point replaced by a quantized version. This technique avoids to some extent numerical difficulties associated with ill-conditioning in least squares schemes and thus obviates the need for square root algorithms and the need for high order precision calculations. In recursive form, the schemes are designed to yield parameter estimates with negligible bias without the additional computational effort or instability risks associated with generalized and extended least squares, recursive maximum likelihood schemes, or the method of instrumental variables. Nonrecursive schemes are designed to minimize computational effort in a batch processing situation while at the same time giving some reduction of bias in the state dependent colored noise situation.

The novel algorithms have the limitation that they are suboptimal and there is thus a consequent reduction in the speed of convergence for some applications. The merits of the proposed schemes are assessed via simulation studies in this paper and an adaptive equalization application in a companion paper.     

Hierarchical Newton and least squares iterative estimation algorithm for dynamic systems by transfer functions based on the impulse responses

This paper develops a parameter estimation algorithm for linear continuous-time systems based on the hierarchical principle and the parameter decomposition strategy. Although the linear continuous-time system is a linear system, its output response is a highly nonlinear function with respect to the system parameters. In order to propose a direct estimation algorithm, a criterion function is constructed between the response output and the observation output by means of the discrete sampled data. Then a scheme by combining the Newton iteration and the least squares iteration is builded to minimise the criterion function and derive the parameter estimation algorithm. In light of the different features between the system parameters and the output function, two sub-algorithms are derived by using the parameter decomposition. In order to remove the associate terms between the two sub-algorithms, a Newton and least squares iterative algorithm is deduced to identify system parameters. Compared with the Newton iterative estimation algorithm without the parameter decomposition, the complexity of the hierarchical Newton and least squares iterative estimation algorithm is reduced because the dimension of the Hessian matrix is lessened after the parameter decomposition. The experimental results show that the proposed algorithm has good performance.     

小波分析和偏最小二乘法相结合用于示波计时电位同时测定铅和铊中的研究

提出了基于小波分析和偏最小二乘(Partial Least Squares,PLS)基础上的化学计量学方法用于示波计时电位同时测定铅和铊的研究。利用小波变换可方便地从dE/dt-E信号中滤噪,提取与去极剂浓度变化有关的信号,获得利于多组分测定的示波图。该方法为示波过程分析奠定了一定的基础。     

Face recognition using partial least squares components

The paper considers partial least squares (PLS) as a new dimension reduction technique for the feature vector to overcome the small sample size problem in face recognition. Principal component analysis (PCA), a conventional dimension reduction method, selects the components with maximum variability, irrespective of the class information. So PCA does not necessarily extract features that are important for the discrimination of classes. PLS, on the other hand, constructs the components so that the correlation between the class variable and themselves is maximized. Therefore PLS components are more predictive than PCA components in classification. The experimental results on Manchester and ORL databases show that PLS is to be preferred over PCA when classification is the goal and dimension reduction is needed.     

The regularized least squares algorithm and the problem of learning halfspaces

We provide sample complexity of the problem of learning halfspaces with monotonic noise, using the regularized least squares algorithm in the reproducing kernel Hilbert spaces (RKHS) framework.     

Sequential weighted least squares algorithm for PET image reconstruction

The weighted least squares (WLS) algorithm has proven useful for modern positron emission tomography (PET) scanners to approach reconstructions with non-Poisson precorrected measurement data. In this paper, we propose a new time recursive sequential WLS algorithm whose derivation uses the time-varying property of data acquisition of PET scanning. It ties close relationship with the time-varying Kalman filtering and can be extended appropriately to an iteration fashion as the absence of proper a priori initializations. The performance of sequential WLS is evaluated experimentally. The results show its fast convergence over both the multiplicative and coordinate-based iterative WLS methods. It also produces relative uniform estimate variances that makes it more suitable for routine applications.