Least squares based iterative algorithms for identifying Box–Jenkins models with finite measurement data

A least squares based iterative identification algorithm is developed for Box–Jenkins models (or systems). The proposed iterative algorithm can produce highly accurate parameter estimation compared with recursive approaches. The basic idea of the proposed iterative method is to adopt the interactive estimation theory: the parameter estimates relying on unknown variables are computed by using the estimates of these unknown variables which are obtained from the preceding parameter estimates. The numerical example indicates that the proposed iterative algorithm has fast convergence rates compared with the gradient based iterative algorithm.     

Parameter estimation for block‐oriented nonlinear systems using the key term separation

This article considers the parameter estimation problems of block‐oriented nonlinear systems. By using the key term separation, the system output is represented as a linear combination of unknown parameters. We give a key term separation auxiliary model gradient‐based iterative (KT‐AM‐GI) identification algorithm and propose a key term separation auxiliary model three‐stage gradient‐based iterative (KT‐AM‐3S‐GI) identification algorithm by using the hierarchical identification principle. Meanwhile, the multiinnovation theory is used to derived the key term separation auxiliary model three‐stage multiinnovation gradient‐based iterative (KT‐AM‐3S‐MIGI) algorithm. The analysis shows that compared with the KT‐AM‐GI algorithm, the KT‐AM‐3S‐GI algorithm can improve the parameter estimation accuracy and reduce the computational burden. In addition, the KT‐AM‐3S‐MIGI can give more accurate parameter estimates than the KT‐AM‐3S‐GI algorithm and can track time‐varying parameters based on the dynamical window data. This work provides a reference for improving the identification performance of multiinput nonlinear output‐error systems or multivariable nonlinear systems. The simulation results confirm the effectiveness of the proposed algorithm.     

Data filtering based least squares iterative algorithm for parameter identification of output error autoregressive systems

This letter derives a data filtering based least squares iterative identification algorithm for output error autoregressive systems. The basic idea is to use the data filtering technique to transform the original identification model to an equation error model and to estimate the parameters of this model. The proposed algorithm is more efficient and can produce more accurate parameter estimation than the existing least squares iterative algorithm.     

Recursive methods for estimating the radial basis function‐based state‐dependent autoregressive model

Identifying a nonlinear radial basis function‐based state‐dependent autoregressive (RBF‐AR) time series model is the basis for solving the corresponding prediction and control problems. This paper studies some recursive parameter estimation algorithms for the RBF‐AR model. Considering the difficulty of the nonlinear optimal problem arising in estimating the RBF‐AR model, an overall forgetting gradient algorithm is deduced based on the negative gradient search. A numerical method with a forgetting factor is provided to solve the problem of determining the optimal convergence factor. In order to improve the parameter estimation accuracy, the multi‐innovation identification theory is applied to develop an overall multi‐innovation forgetting gradient (O‐MIFG) algorithm. The simulation results indicate that the estimation model based on the O‐MIFG algorithm can capture the dynamics of the RBF‐AR model very well.     

Gradient‐based back‐propagation dynamical iterative learning scheme for the neuro‐fuzzy inference system

In this paper, a gradient‐based back propagation dynamical iterative learning algorithm is proposed for structure optimization and parameter tuning of the neuro‐fuzzy system. Premise and consequent parameters of the neuro‐fuzzy model are initialized randomly and then tuned by the proposed iterative algorithm. The learning algorithm is based on the first order partial derivative of the output with respect to the structure parameters. The first order derivative of the model output with respect to the structure parameters determines the sensitivity of the model to structure parameters. The sensitivity values are then used to set the tuning factors and parameters updating step sizes. Therefore, an adaptive dynamical iterative scheme is achieved which adapts the learning procedure to the current state of the performance during the optimization process. Larger tuning step sizes make the convergence speed higher and vice versa. In this regard, this parameter is treated according to the calculated sensitivity of the model to the parameter. The proposed learning algorithm is compared with the least square back propagation method, genetic algorithm and chaotic genetic algorithm in the neuro‐fuzzy model structure optimization. Smaller mean square error and shorter learning time are sought in this paper, and the performance of the proposed learning algorithm is versified regarding these criteria.     

Two‐stage auxiliary model gradient‐based iterative algorithm for the input nonlinear controlled autoregressive system with variable‐gain nonlinearity

This article focuses on the parameter estimation problem of the input nonlinear system where an input variable‐gain nonlinear block is followed by a linear controlled autoregressive subsystem. The variable‐gain nonlinearity is described analytical by using an appropriate switching function. According to the gradient search technique and the auxiliary model identification idea, an auxiliary model‐based stochastic gradient algorithm with a forgetting factor is presented. For the sake of improving the parameter estimation accuracy, an auxiliary model gradient‐based iterative algorithm is proposed by utilizing the iterative identification theory. To further optimize the performance of the algorithm, we decompose the identification model of the system into two submodels and derive a two‐stage auxiliary model gradient‐based iterative (2S‐AM‐GI) algorithm by using the hierarchical identification principle. The simulation results confirm the effectiveness of the proposed algorithms and show that the 2S‐AM‐GI algorithm has higher identification efficiency compared with the other two algorithms.     

A model identification scheme for driver-following dynamics in road traffic

The driver-following, or car-following, model is one of the most fundamental driver behavior models that are applied in intelligent transport applications. Its fidelity determines the applicability of microscopic traffic simulators, where the model is often implemented to mimic real traffic. Meanwhile, the behavioral model is fundamental to the development of advanced driving assistance systems (ADAS). This paper develops a dynamic model identification approach based on iterative usage of the extended Kalman Filtering (EKF) algorithm. Among other things, this allows to carry out model identification using a rather general optimization objective on the whole physical states of the following vehicle. In particular, the method is established on the basis of the equivalence between the Kalman filter and the recursive least squares (RLS) method in a specific context of parameter identification. To illustrate the method, two car-following models are studied in numerical experiments using real car-following data. The method has shown advantages in replication and prediction of vehicle dynamics in car-following over the conventional approaches. It has also the potential to be further extended for building tactical driving controllers in intelligent transportation applications.     

An Efficient on‐Line Parameter Identification Algorithm for Nonlinear Servomechanisms with an Algebraic Technique for State Estimation

This paper presents a methodology for on‐line closed‐loop identification of a class of nonlinear servomechanisms. First, a system is defined with the same structure as the actual servomechanism, but using time‐varying estimated parameters. No coupling between the actual and the estimation systems is present. Position, velocity and acceleration errors, defined as the difference of the actual respective signals and the signals coming from the estimation system, are required in the identification method. Then, a recursive algorithm for on‐line identification of the system parameters is derived from a cost function depending on a linear combination of all the estimation errors. Velocity and acceleration estimates, required in the proposed parameter identification algorithm, are obtained by using an algebraic methodology. The identification algorithm is compared by means of real‐time experiments with an on‐line least squares algorithm with forgetting factor and an off‐line least squares algorithm with data preprocessing. Experimental results show that the proposed approach has a performance comparable to that obtained with the off‐line least squares algorithm, but with the advantage of avoiding any preprocessing.     

Two-stage least squares based iterative identification algorithm for controlled autoregressive moving average (CARMA) systems

A two-stage least squares based iterative (two-stage LSI) identification algorithm is derived for controlled autoregressive moving average (CARMA) systems. The basic idea is to decompose a CARMA system into two subsystems and to identify each subsystem, respectively. Because the dimensions of the involved covariance matrices in each subsystem become small, the proposed algorithm has a high computational efficiency. The simulation results indicate that the proposed algorithm is effective.     

Adaptive parameter estimation for a general dynamical system with unknown states

This paper is concerned with the design of a state filter for a time‐delay state‐space system with unknown parameters from noisy observation information. The key is to investigate new identification algorithms for interactive state and parameter estimation of the considered system. Firstly, an observability canonical state‐space model is derived from the original model by linear transformation for the purpose of simplifying the model structure. Secondly, a direct state filter is formulated by minimizing the state estimation error covariance matrix on the basis of the Kalman filtering principle. Thirdly, once the unknown states are estimated, a state filter–based recursive least squares algorithm is proposed for parameter estimation using the least squares principle. Then, a state filter–based hierarchical least squares algorithm is derived by decomposing the original system into several subsystems for improving the computational efficiency. Finally, the numerical examples illustrate the effectiveness and robustness of the proposed algorithms.     

Maximum likelihood based identification methods for rational models

ABSTRACT

In a rational model, some terms of the information vector are correlated with the noise, which makes the traditional least squares based iterative algorithms biased. In order to overcome this shortcoming, this paper develops two recursive algorithms for estimating the rational model parameters. These two algorithms, based on the maximum likelihood principle, have three integrated key features: (1) to establish two unbiased maximum likelihood recursive algorithms, (2) to develop a maximum likelihood recursive least squares (ML-RLS) algorithm to decrease the computational efforts, (3) to update the parameter estimates by the ML-RLS based particle swarm optimisation (ML-RLS-PSO) algorithm when the noise-to-output ratio is large. Comparative studies demonstrate that (1) the ML-RLS algorithm is only valid for rational models when the noise-to-output ratio is small, (2) the ML-RLS-PSO algorithm is effective for rational models with random noise-to-output ratio, but at the cost of heavy computational efforts. Furthermore, the simulations provide cases for potential expansion and applications of the proposed algorithms.     

Recursive Identification Methods for Multivariate Output-error Moving Average Systems Using the Auxiliary Model

This paper studies the parameter identification problems of multivariate output-error moving average systems. An auxiliary model based extended stochastic gradient algorithm and based recursive extended least squares algorithm are proposed for estimating the parameters of the multivariate output-error moving average systems. By using the multi-innovation identification theory, an auxiliary model based multi-innovation extended stochastic gradient algorithm is derived for improving the parameter estimation accuracy. Finally, the simulation results indicate that the proposed algorithms can work well.     

Time-varying parametric system multiresolution identification by wavelets

In this paper, the problem of time-varying parametric system identification by wavelets is discussed. Employing wavelet operator matrix representation, we propose a new multiresolution least squares (MLS) algorithm for time-varying AR (ARX) system identification and a multiresolution least mean squares (MLMS) algorithm for the refinement of parameter estimation. These techniques can achieve the optimal tradeoff between the over-fitted solution and the poorly represented identification. The main features of time-varying model parameters are extracted in a multiresolution way, which can be used to represent the smooth trends as well as track the rapidly changing components of time-varying parameters simultaneously and adaptively. Further, a noisy time-varying AR (ARX) model can also be identified by combining the total least squares algorithm with the MLS algorithm. Based on the proposed AR (ARX) model parameter estimation algorithm, a novel identification scheme for time-varying ARMA (ARMAX) system is presented. A higher-order time-varying AR (ARX) model is used to approximate the time-varying ARMA (ARMAX) system and thus obtain an initial parameter estimation. Then an iterative algorithm is applied to obtain the consistent and efficient estimates of the ARMA (ARMAX) system parameters. This ARMA (ARMAX) identification algorithm requires linear operations only and thus greatly saves the computational load. In order to determine the time-varying model order, some modified AIC and MDL criterions are developed based on the proposed wavelet identification schemes. Simulation results verify that our methods can track the rapidly changing of time-varying system parameters and attain the best balance between parsimonious modelling and accurate identification.     

Iterative identification algorithms for input nonlinear output error autoregressive systems

This paper focuses on the parameter estimation problems of input nonlinear output error autoregressive systems. Based on the key variables separation technique and the auxiliary model identification idea, the output of the system is expressed as a linear combination of all the system parameters, the unknown inner variables in the information vector are replaced with the outputs of the auxiliary model and a gradient based and a least squares based iterative identification algorithms are derived. Simulation example is provided to illustrate the effectiveness of the proposed algorithms.     

基于KRLS的非均匀采样非线性系统辨识

利用提升技术可将非均匀采样非线性系统离散化为一个多输入单输出传递函数模型,从而将系统输出表示为非均匀刷新非线性输入和输出回归项的线性参数模型,进一步基于非线性输入的估计或过参数化方法进行辨识.然而,当非线性环节结构未知或不能被可测非均匀输入参数化表示时,上述辨识方法将不再适用.为了解决这个问题,利用核方法将原始非线性数据投影到高维特征空间中使其线性可分,再对投影后的数据应用递推最小二乘算法进行辨识,提出基于核递推最小二乘的非均匀采样非线性系统辨识方法.此外,针对系统含有有色噪声干扰的情况,参考递推增广最小二乘算法的思想,利用估计残差代替不可测噪声,提出核递推增广最小二乘算法.最后,通过仿真例子验证所提算法的有效性.     

Three‐stage multi‐innovation parameter estimation for an exponential autoregressive time‐series model with moving average noise by using the data filtering technique

This paper studies the data filtering‐based identification algorithms for an exponential autoregressive time‐series model with moving average noise. By means of the data filtering technique and the hierarchical identification principle, the identification model is transformed into three sub‐identification (Sub‐ID) models, and a filtering‐based three‐stage extended stochastic gradient algorithm is derived for identifying these Sub‐ID models. In order to improve the parameter estimation accuracy, a filtering‐based three‐stage multi‐innovation extended stochastic gradient (F‐3S‐MIESG) algorithm is developed by using the multi‐innovation identification theory. The simulation results indicate that the proposed F‐3S‐MIESG algorithm can work well.     

M-estimator and D-optimality model construction using orthogonal forward regression.

This correspondence introduces a new orthogonal forward regression (OFR) model identification algorithm using D-optimality for model structure selection and is based on an M-estimators of parameter estimates. M-estimator is a classical robust parameter estimation technique to tackle bad data conditions such as outliers. Computationally, The M-estimator can be derived using an iterative reweighted least squares (IRLS) algorithm. D-optimality is a model structure robustness criterion in experimental design to tackle ill-conditioning in model structure. The orthogonal forward regression (OFR), often based on the modified Gram-Schmidt procedure, is an efficient method incorporating structure selection and parameter estimation simultaneously. The basic idea of the proposed approach is to incorporate an IRLS inner loop into the modified Gram-Schmidt procedure. In this manner, the OFR algorithm for parsimonious model structure determination is extended to bad data conditions with improved performance via the derivation of parameter M-estimators with inherent robustness to outliers. Numerical examples are included to demonstrate the effectiveness of the proposed algorithm.     

Moving data window gradient‐based iterative algorithm of combined parameter and state estimation for bilinear systems

The combined iterative parameter and state estimation problem is considered for bilinear state‐space systems with moving average noise in this paper. There are the product terms of state variables and control variables in bilinear systems, which makes it difficult for the parameter and state estimation. By designing a bilinear state estimator based on the Kalman filtering, the states are estimated using the input‐output data. Furthermore, a moving data window (MDW) is introduced, which can update the dynamical data by removing the oldest data and adding the newest measurement data. A state estimator‐based MDW gradient‐based iterative (MDW‐GI) algorithm is proposed to estimate the unknown states and parameters jointly. Moreover, given the extended gradient‐based iterative (EGI) algorithm as a comparison, the MDW‐GI algorithm can reduce the impact of noise to parameter estimation and improve the parameter estimation accuracy. The numerical simulation examples validate the effectiveness of the proposed algorithm.     

一种基于参数辨识的迭代学习控制算法及应用

针对间歇过程中模型参数变化的问题,提出了一种基于遗忘因子最小二乘法辨识的迭代学习控制算法。迭代学习律的参数随模型参数变化而更新,利用遗忘因子大大减小参数变化时"错误"数据对算法的影响,使算法具有更强的自适应性。把这一算法应用于黄酒发酵过程,提高了发酵过程的优化控制效果。仿真结果表明当模型参数随着批次变化时,系统的跟踪性能得到了改进。