Maximum likelihood-based recursive least-squares estimation for multivariable systems using the data filtering technique

For multivariable equation-error systems with an autoregressive moving average noise, this paper applies the decomposition technique to transform a multivariable model into several identification sub-models based on the number of the system outputs, and derives a data filtering and maximum likelihood-based recursive least-squares algorithm to reduce the computation complexity and improve the parameter estimation accuracy. A multivariable recursive generalised extended least-squares method and a filtering-based recursive extended least-squares method are presented to show the effectiveness of the proposed algorithm. The simulation results indicate that the proposed method is effective and can produce more accurate parameter estimates than the compared methods.     

Genetic least squares for system identification

The recursive least-squares algorithm with a forgetting factor has been extensively applied and studied for the on-line parameter estimation of linear dynamic systems. This paper explores the use of genetic algorithms to improve the performance of the recursive least-squares algorithm in the parameter estimation of time-varying systems. Simulation results show that the hybrid recursive algorithm (GARLS), combining recursive least-squares with genetic algorithms, can achieve better results than the standard recursive least-squares algorithm using only a forgetting factor.     

Gradient based and least-squares based iterative identification methods for OE and OEMA systems

Gradient based and least-squares based iterative identification algorithms are developed for output error (OE) and output error moving average (OEMA) systems. Compared with recursive approaches, the proposed iterative algorithms use all the measured input–output data at each iterative computation (at each iteration), and thus can produce highly accurate parameter estimation. The basic idea of the iterative methods is to adopt the interactive estimation theory: the parameter estimates relying on unknown variables are computed by using the estimates of these unknown variables which are obtained from the preceding parameter estimates. The simulation results confirm theoretical findings.     

基于KRLS的非均匀采样非线性系统辨识

利用提升技术可将非均匀采样非线性系统离散化为一个多输入单输出传递函数模型,从而将系统输出表示为非均匀刷新非线性输入和输出回归项的线性参数模型,进一步基于非线性输入的估计或过参数化方法进行辨识.然而,当非线性环节结构未知或不能被可测非均匀输入参数化表示时,上述辨识方法将不再适用.为了解决这个问题,利用核方法将原始非线性数据投影到高维特征空间中使其线性可分,再对投影后的数据应用递推最小二乘算法进行辨识,提出基于核递推最小二乘的非均匀采样非线性系统辨识方法.此外,针对系统含有有色噪声干扰的情况,参考递推增广最小二乘算法的思想,利用估计残差代替不可测噪声,提出核递推增广最小二乘算法.最后,通过仿真例子验证所提算法的有效性.     

The relationship between a continuous-time identification algorithm based on the deterministic filter and least-squares methods

It is shown that an identification technique recently derived from the continuous-time Kalman filter may also be deduced from recursive algorithms for least-squares and minimum-variance methods of parameter estimation. Preliminary repeated integration, as a method for the identification of continuous systems, is enhanced by its incorporation into the procedure. The investigation permits a greater appreciation of certain features of the least-squares and minimum-variance methods, including the inference that the recursive minimum-variance algorithm can only exist when the measurement noise is an uncorrelated sequence. A study of well-known starting procedures relate these techniques to a deterministic state estimator derived from stability considerations.     

Auxiliary model-based RELS and MI-ELS algorithm for Hammerstein OEMA systems

This paper considers the identification problem for Hammerstein output error moving average (OEMA) systems. An auxiliary model-based recursive extended least-squares (RELS) algorithm and an auxiliary model-based multi-innovation extended least-squares (MI-ELS) algorithm are presented using the multi-innovation identification theory. The basic idea is to express the system output as a linear combination of the parameters by using the key-term separation principle and auxiliary model method. The proposed algorithms can give highly accurate parameter estimates. The simulation results show the effectiveness of the proposed algorithms.     

Recursive least squares parameter identification algorithms for systems with colored noise using the filtering technique and the auxilary model

This paper focuses on the parameter estimation problems of output error autoregressive systems and output error autoregressive moving average systems (i.e., the Box–Jenkins systems). Two recursive least squares parameter estimation algorithms are proposed by using the data filtering technique and the auxiliary model identification idea. The key is to use a linear filter to filter the input–output data. The proposed algorithms can identify the parameters of the system models and the noise models interactively and can generate more accurate parameter estimates than the auxiliary model based recursive least squares algorithms. Two examples are given to test the proposed algorithms.     

Maximum likelihood parameter estimation algorithm for controlled autoregressive autoregressive models

A maximum likelihood parameter estimation algorithm is derived for controlled autoregressive autoregressive (CARAR) models based on the maximum likelihood principle. In this derivation, we use an estimated noise transfer function to filter the input–output data. The simulation results show that the proposed estimation algorithm can effectively estimate the parameters of such class of CARAR systems and give more accurate parameter estimates than the recursive generalized least-squares algorithm.     

Convergence of adaptive control schemes using least-squaresparameter estimates

The stability, convergence, asymptotic optimality, and self-tuning properties of stochastic adaptive control schemes based on least-squares estimates of the unknown parameters are examined. It is assumed that the additive noise is i.i.d. and Gaussian, and that the true system is of minimum phase. The Bayesian embedding technique is used to show that the recursive least-squares parameter estimates converge in general. The normal equations of least squares are used to establish that all stable control law designs used in a certainty-equivalent (i.e. indirect) procedure generally yield a stable adaptive control system. Four results are given to characterize the limiting behavior precisely. A certainty-equivalent self-tuning regulator is shown to yield strongly consistent parameter estimates when the delay is strictly greater than one, even without any excitation in the reference trajectory     

Parameter identification of distributed parameter systems in the presence of measurement noise

A new approach to recursive parameter identification of second-order distributed parameter systems in the presence of measurement noise under unknown initial and boundary conditions is proposed. A two-dimensional low-pass filter is introduced to pre-filter the observed data corrupted by measurement noise. The low-pass filter is designed in the continuous time-space domain and discretized by bilinear transformation. Thus a discrete estimation model of the system under study is easily constructed with filtered input-output data for recursive identification algorithms. The recursive least squares method is still efficient in the presence of low measurement noise if the filter parameters are designed so that the noise effects are reduced sufficiently. Using filtered input data as instrumental variables, a recursive instrumental variable method is also presented to obtain consistent estimates when the digital low-pass filters are not designed successfully or when the output data is corrupted by high measurement noise. Illustrative examples are given to demonstrate the applicability of the proposed methods.     

Recursive Iterative Algorithms for Adaptive Identification of Nonlinear Concentrated Dynamic Systems

Identification algorithms for concentrated dynamic systems are studied. The apparatus of pseudo-inversion of block matrices underlies adaptive identification. It is used to construct a recursive algorithm for linear least-squares problems and a recursive iterative algorithm for nonlinear least-squares problems.     

Recursive estimation methods for discrete systems

In system identification, the error evolution is composed of two decoupled parts: one is the identifying information on the current estimation residual, while the other is past arithmetic errors. Previous recursive algorithms only considered how to update current prediction errors. Up to now, research has mostly been based on recursive least-squares (RLS) methods. In this note, a general recursive identification method is proposed for discrete systems. Using this new algorithm, a recursive empirical frequency-domain optimal parameter (REFOP) estimate is established. The REFOP method has the advantage of resisting disturbance noise. Some simulations are included to illustrate the new method's reliability.     

Bias-compensation-based least-squares estimation with a forgetting factor for output error models with white noise

In this paper, the bias-compensation-based recursive least-squares (LS) estimation algorithm with a forgetting factor is proposed for output error models. First, for the unknown white noise, the so-called weighted average variance is introduced. With this weighted average variance, a bias-compensation term is first formulated to achieve the bias-eliminated estimates of the system parameters. Then, the weighted average variance is estimated. Finally, the final estimation algorithm is obtained by combining the estimation of the weighted average variance and the recursive LS estimation algorithm with a forgetting factor. The effectiveness of the proposed identification algorithm is verified by a numerical example.     

Continuous-time constrained least-squares algorithms for recursive parameter estimation of stochastic linear systems by a stabilized output-error method

Discrete-time least-squares algorithms for recursive parameter estimation have continuous-time counterparts, which minimize a quadratic functional. The continuous-time algorithms can also include (in)equality constraints. Asymptotic convergence is demonstrated by means of Lyapunov methods. The constrained algorithms are applied in a stabilized output-error configuration for parameter estimation in stochastic linear systems.     

Iterative and recursive least squares estimation algorithms for moving average systems

An iterative least squares algorithm and a recursive least squares algorithms are developed for estimating the parameters of moving average systems. The key is use the least squares principle and to replace the unmeasurable noise terms in the information vector. The steps and flowcharts of computing the parameter estimates are given. The simulation results validate that the proposed algorithms can work well.     

The parallel implementation of a cascade adaptive identification algorithm

This note presents a new method of parameter estimation, called cascading, for use in adaptive control. The algorithm is shown to be superior to a simple recursive least-squares estimator especially for a system characterized by noisy measurements. The algorithm can be implemented easily on a parallel processor such as ORAC [1], [2] or any sequential processor. When the algorithm is implemented on a parallel processor such as ORAC the real time used to compute the parameter estimates is of the same order as a recursive least-squares estimator.     

Input–output data filtering based recursive least squares identification for CARARMA systems

This paper uses an estimated noise transfer function to filter the input–output data and presents filtering based recursive least squares algorithms (F-RLS) for controlled autoregressive autoregressive moving average (CARARMA) systems. Through the data filtering, we obtain two identification models, one including the parameters of the system model, and the other including the parameters of the noise model. Thus, the recursive least squares method can be used to estimate the parameters of these two identification models, respectively, by replacing the unmeasurable variables in the information vectors with their estimates. The proposed F-RLS algorithm has a high computational efficiency because the dimensions of its covariance matrices become small and can generate more accurate parameter estimation compared with other existing algorithms.     

Online identification of continuous-time systems with unknown time delay

In this note, we present a recursive algorithm for online identification of systems with unknown time delay. The proposed algorithm can be interpreted as an approximate nonlinear least-squares, corresponding to modified normalized or un-normalized least-squares when the normalizing factor /spl gamma/>0 or /spl gamma/=0, respectively. Both algorithms are essentially extensions to general least-squares methodology. Simulation studies demonstrate the characteristics and performance of these algorithms.     

基于数据滤波的带协方差重置的递推贝叶斯算法

针对传统最小二乘算法计算量大、在有色噪声干扰下估计有误差的问题,提出了一种基于滤波技术的带协方差重置的递推贝叶斯算法。该算法首先使用一个动态非线性滤波器对输入输出数据进行滤波,然后使用贝叶斯方法进行参数估计。同时,为了加快参数的收敛速度,在算法中加入了一种新型的协方差重置策略。计算量分析表明,当过程模型和噪声模型的阶数分别为6和4的时候,所提算法可以减少约62.35%的计算量。仿真结果显示,所提算法与传统最小二乘算法在采样数据长度为3000时的估计误差分别为0.771%和1.118%。因此,所提算法具有较高的计算效率,并且可以给出精度较高的参数估计值。     

Identification of continuous-time systems with multiple unknown time delays by global nonlinear least-squares and instrumental variable methods

This paper considers the identification problem of multiple input single output (MISO) continuous-time systems with unknown time delays of the inputs, from sampled input-output data. An iterative global separable nonlinear least-squares (GSEPNLS) method which estimates the time delays and transfer function parameters separably is derived to significantly reduce the possibility of convergence to a local minimum, by using the stochastic global-optimization techniques. Furthermore, the GSEPNLS method is modified to a novel global separable nonlinear instrumental variable (GSEPNIV) method to remove the biases of the estimates in the presence of high measurement noise. Simulation results show that the proposed algorithms work quite well.