Two‐stage auxiliary model gradient‐based iterative algorithm for the input nonlinear controlled autoregressive system with variable‐gain nonlinearity

This article focuses on the parameter estimation problem of the input nonlinear system where an input variable‐gain nonlinear block is followed by a linear controlled autoregressive subsystem. The variable‐gain nonlinearity is described analytical by using an appropriate switching function. According to the gradient search technique and the auxiliary model identification idea, an auxiliary model‐based stochastic gradient algorithm with a forgetting factor is presented. For the sake of improving the parameter estimation accuracy, an auxiliary model gradient‐based iterative algorithm is proposed by utilizing the iterative identification theory. To further optimize the performance of the algorithm, we decompose the identification model of the system into two submodels and derive a two‐stage auxiliary model gradient‐based iterative (2S‐AM‐GI) algorithm by using the hierarchical identification principle. The simulation results confirm the effectiveness of the proposed algorithms and show that the 2S‐AM‐GI algorithm has higher identification efficiency compared with the other two algorithms.     

Parameter estimation for block‐oriented nonlinear systems using the key term separation

This article considers the parameter estimation problems of block‐oriented nonlinear systems. By using the key term separation, the system output is represented as a linear combination of unknown parameters. We give a key term separation auxiliary model gradient‐based iterative (KT‐AM‐GI) identification algorithm and propose a key term separation auxiliary model three‐stage gradient‐based iterative (KT‐AM‐3S‐GI) identification algorithm by using the hierarchical identification principle. Meanwhile, the multiinnovation theory is used to derived the key term separation auxiliary model three‐stage multiinnovation gradient‐based iterative (KT‐AM‐3S‐MIGI) algorithm. The analysis shows that compared with the KT‐AM‐GI algorithm, the KT‐AM‐3S‐GI algorithm can improve the parameter estimation accuracy and reduce the computational burden. In addition, the KT‐AM‐3S‐MIGI can give more accurate parameter estimates than the KT‐AM‐3S‐GI algorithm and can track time‐varying parameters based on the dynamical window data. This work provides a reference for improving the identification performance of multiinput nonlinear output‐error systems or multivariable nonlinear systems. The simulation results confirm the effectiveness of the proposed algorithm.     

Performance Analysis of The Auxiliary‐Model‐Based Multi‐Innovation Stochastic Newton Recursive Algorithm for Dual‐Rate Systems

The stochastic Newton recursive algorithm is studied for dual‐rate system identification. Owing to a lack of intersample measurements, the single‐rate model cannot be identified directly. The auxiliary model technique is adopted to provide the intersample estimations to guarantee the recursion process continues. Intersample estimations have a great influence on the convergence of parameter estimations, and one‐step innovation may lead to a large fluctuation or even divergence during the recursion. In the meantime, the sample covariance matrix may appear singular. The recursive process would cease for these reasons. In order to guarantee the recursion process and to also improve estimation accuracy, multi‐innovation is utilized for correcting the parameter estimations. Combining the auxiliary model and multi‐innovation theory, the auxiliary‐model‐based multi‐innovation stochastic Newton recursive algorithm is proposed for time‐invariant dual‐rate systems. The consistency of this algorithm is analyzed in detail. The final simulations confirm the effectiveness of the proposed algorithm.     

Recursive methods for estimating the radial basis function‐based state‐dependent autoregressive model

Identifying a nonlinear radial basis function‐based state‐dependent autoregressive (RBF‐AR) time series model is the basis for solving the corresponding prediction and control problems. This paper studies some recursive parameter estimation algorithms for the RBF‐AR model. Considering the difficulty of the nonlinear optimal problem arising in estimating the RBF‐AR model, an overall forgetting gradient algorithm is deduced based on the negative gradient search. A numerical method with a forgetting factor is provided to solve the problem of determining the optimal convergence factor. In order to improve the parameter estimation accuracy, the multi‐innovation identification theory is applied to develop an overall multi‐innovation forgetting gradient (O‐MIFG) algorithm. The simulation results indicate that the estimation model based on the O‐MIFG algorithm can capture the dynamics of the RBF‐AR model very well.     

Maximum likelihood‐based gradient estimation for multivariable nonlinear systems using the multiinnovation identification theory

This article considers the identification problems of multivariable input nonlinear systems with unmeasured disturbances. For the identification difficulty caused by the crossproducts between the parameters of the linear block and the nonlinear block, the key term separation technique is adopted to separate the parameters of the nonlinear block from the parameters of the linear block. By combining the model decomposition technique and the hierarchical identification principle, a key term separation‐based maximum likelihood recursive extended stochastic gradient algorithm with reduced computational complexity is presented to estimate all the parameters directly. By introducing the multiinnovation identification theory, a key term separation‐based maximum likelihood multiinnovation extended stochastic gradient algorithm is proposed to improve the parameter estimation accuracy. The simulation results illustrate the effectiveness of the proposed methods.     

Parameter estimation for nonlinear systems by using the data filtering and the multi-innovation identification theory

For Hammerstein output-error autoregressive systems, a decomposition based multi-innovation stochastic gradient (D-MISG) identification algorithm and a data filtering based multi-innovation stochastic gradient (F-MISG) identification algorithm are derived by means of the key-term separation principle and the multi-innovation identification theory. The D-MISG algorithm uses the decomposition technique to transform a Hammerstein system into two subsystems and requires less computational cost, and the F-MISG algorithm uses a linear filter to filter the input-output data and has a higher estimation accuracy for larger innovation lengths. The simulation results show that the proposed two algorithm can give satisfactory parameter estimates.     

Decomposition‐based multiinnovation gradient identification algorithms for a special bilinear system based on its input‐output representation

This article considers the parameter estimation for a special bilinear system with colored noise. Its input‐output representation is derived by eliminating the state variables in the bilinear system. Based on the input‐output representation of the bilinear system, a multiinnovation generalized extended stochastic gradient (MI‐GESG) algorithm is proposed by using the multiinnovation identification theory. Furthermore, a decomposition‐based multiinnovation (ie, hierarchical multiinnovation) generalized extended stochastic gradient identification (H‐MI‐GESG) algorithm is derived to enhance the parameter estimation accuracy by using the hierarchical identification principle, and a GESG algorithm is presented for comparison. Compared with the existing identification algorithms for the bilinear system, the proposed MI‐GESG and H‐MI‐GESG algorithms can generate more accurate parameter estimation. Finally, a simulation example is provided to verify the effectiveness of the proposed algorithms.     

MISO系统辅助模型多新息增广随机梯度算法

对于有色噪声干扰的输出误差多输入单输出（MISO）系统,常规的递推最小二乘辨识方法给出的参数估计是有偏的。为了提高随机梯度辨识方法的收敛精度和速度,用辅助模型的输出代替辨识模型信息向量中的未知不可测变量,推导出其辅助模型增广随机梯度辨识算法;再引入新息长度扩展标量新息为新息向量,提出了基于辅助模型的MISO系统多新息增广随机梯度辨识算法。所得算法在每一次的迭代中不仅使用了当前数据和新息,而且使用了过去数据和新息,提高了参数估计精度和收敛速度。仿真例子验证了算法的有效性。     

基于辅助模型的多新息广义增广随机梯度算法

将辅助模型辨识思想与多新息辨识理论相结合,利用系统可测信忠建立一个辅助模型.分别用辅助模型输出和噪声估计值代替辨识模型信忠向量中未知真实输出变量和不可测噪声项,并引入新忠长度扩展标量新息为新息向量,提出了Box-lenkins模型的辅助模型多新忠广义增广随机梯度辨识方法.所提出方法重复使用系统数据,能够改善参数估计精度,加快算法的收敛速度.     

A novel data filtering based multi-innovation stochastic gradient algorithm for Hammerstein nonlinear systems

The identification of nonlinear systems is a hot topic in the identification fields. In this paper, a data filtering based multi-innovation stochastic gradient algorithm is derived for Hammerstein nonlinear controlled autoregressive moving average systems by adopting the key-term separation principle and the data filtering technique. The proposed algorithm provides a reference to improve the identification accuracy of the nonlinear systems with colored noise. The simulation results show that the new algorithm can more effectively estimate the parameters of the Hammerstein nonlinear systems than the multi-innovation stochastic gradient algorithm.     

Recursive parameter estimation methods and convergence analysis for a special class of nonlinear systems

This paper is concerned with the joint estimation of states and parameters of a special class of nonlinear systems, ie, bilinear systems. The key is to investigate new estimation methods for interactive state and parameter estimation of the considered system based on the interactive estimation theory. Because the system states are unknown, a bilinear state observer is established based on the Kalman filtering principle. Then, the unavailable states are updated by the state observer outputs recursively. Once the state estimates are obtained, the bilinear state observer–based hierarchical stochastic gradient algorithm is developed by using the gradient search. For the purpose of improving the convergence rate and the parameter estimation accuracy, a bilinear state observer–based hierarchical multi‐innovation stochastic gradient algorithm is proposed by expanding a scalar innovation to an innovation vector. The convergence analysis indicates that the parameter estimates can converge to their true values. The numerical example illustrates the effectiveness of the proposed algorithms.     

Maximum likelihood least squares‐based iterative methods for output‐error bilinear‐parameter models with colored noises

This article is concerned with the parameter identification of output‐error bilinear‐parameter models with colored noises from measurement data. An auxiliary model least squares‐based iterative method is developed through the overparameterization model. It examines the difficulty of estimating the overparameterized vector, which usually presents a heavy computational burden in the identification process. To overcome this drawback, a parameter separation technique is introduced and the nonlinear model is reformulated as a refined identification model through eliminating the crossmultiplying terms. In this regard, a parameter separation least squares‐based iterative (PS‐LSI) algorithm is derived by avoiding estimating the redundant parameters. On the basis of the PS‐LSI algorithm, we derive a maximum likelihood least squares‐based iterative method to further improve the numerical accuracy. The identification is dependent on the formulation of a pseudolinear regression relationship, which contains two linear prefilters constructed from the system and noise models. The performance of this proposed method is confirmed by the numerical simulations as well as direct comparisons with other existing algorithms.     

Convergence Analysis of Weighted Stochastic Gradient Identification Algorithms Based on Latest‐Estimation for ARX Models

In this paper, weighted stochastic gradient (WSG) algorithms for ARX models are proposed by modifying the standard stochastic gradient identification algorithms. In the proposed algorithms, the correction term is a weighting combination of the correction terms of the standard stochastic gradient (SG) algorithm in the current and last recursive steps. In addition, a latest estimation based WSG (LE‐WSG) algorithm is also established. The convergence performance of the proposed LE‐WSG algorithm is then analyzed. It is shown by a numerical example that both the WSG and LE‐WSG algorithms can possess faster convergence speed and higher convergence precision compared with the standard SG algorithms if the weighting factor is appropriately chosen.     

非均匀采样系统多新息随机梯度辨识性能分析

针对一类非均匀采样系统,提出了其输入输出表达的多新息随机梯度辨识方法．该方法将随机梯度算法中的新息项扩展为向量,有效利用了历史新息所包含的信息,从而提高辨识精度和算法的收敛速度,同时又保留了随机梯度算法计算量小的优点．仿真例子通过改变新息长度,验证了所提出辨识算法性能的优越性．     

Recursive Identification Methods for Multivariate Output-error Moving Average Systems Using the Auxiliary Model

This paper studies the parameter identification problems of multivariate output-error moving average systems. An auxiliary model based extended stochastic gradient algorithm and based recursive extended least squares algorithm are proposed for estimating the parameters of the multivariate output-error moving average systems. By using the multi-innovation identification theory, an auxiliary model based multi-innovation extended stochastic gradient algorithm is derived for improving the parameter estimation accuracy. Finally, the simulation results indicate that the proposed algorithms can work well.     

Performance analysis of multi-innovation gradient type identification methods

It is well-known that the stochastic gradient (SG) identification algorithm has poor convergence rate. In order to improve the convergence rate, we extend the SG algorithm from the viewpoint of innovation modification and present multi-innovation gradient type identification algorithms, including a multi-innovation stochastic gradient (MISG) algorithm and a multi-innovation forgetting gradient (MIFG) algorithm. Because the multi-innovation gradient type algorithms use not only the current data but also the past data at each iteration, parameter estimation accuracy can be improved. Finally, the performance analysis and simulation results show that the proposed MISG and MIFG algorithms have faster convergence rates and better tracking performance than their corresponding SG algorithms.     

Moving data window gradient‐based iterative algorithm of combined parameter and state estimation for bilinear systems

The combined iterative parameter and state estimation problem is considered for bilinear state‐space systems with moving average noise in this paper. There are the product terms of state variables and control variables in bilinear systems, which makes it difficult for the parameter and state estimation. By designing a bilinear state estimator based on the Kalman filtering, the states are estimated using the input‐output data. Furthermore, a moving data window (MDW) is introduced, which can update the dynamical data by removing the oldest data and adding the newest measurement data. A state estimator‐based MDW gradient‐based iterative (MDW‐GI) algorithm is proposed to estimate the unknown states and parameters jointly. Moreover, given the extended gradient‐based iterative (EGI) algorithm as a comparison, the MDW‐GI algorithm can reduce the impact of noise to parameter estimation and improve the parameter estimation accuracy. The numerical simulation examples validate the effectiveness of the proposed algorithm.     

Extended stochastic gradient identification algorithms for Hammerstein–Wiener ARMAX systems

An extended stochastic gradient algorithm is developed to estimate the parameters of Hammerstein–Wiener ARMAX models. The basic idea is to replace the unmeasurable noise terms in the information vector of the pseudo-linear regression identification model with the corresponding noise estimates which are computed by the obtained parameter estimates. The obtained parameter estimates of the identification model include the product terms of the parameters of the original systems. Two methods of separating the parameter estimates of the original parameters from the product terms are discussed: the average method and the singular value decomposition method. To improve the identification accuracy, an extended stochastic gradient algorithm with a forgetting factor is presented. The simulation results indicate that the parameter estimation errors become small by introducing the forgetting factor.     

Identification of hybrid and linear parameter‐varying models via piecewise affine regression using mixed integer programming

This article presents a two‐stage algorithm for piecewise affine (PWA) regression. In the first stage, a moving horizon strategy is employed to simultaneously estimate the model parameters and to classify the training data by solving a small‐size mixed‐integer quadratic programming problem. In the second stage, linear multicategory separation methods are used to partition the regressor space. The framework of PWA regression is adapted to the identification of PWA AutoRegressive with eXogenous input (PWARX) models as well as linear parameter‐varying (LPV) models. The performance of the proposed algorithm is demonstrated on an academic example and on two benchmark experimental case studies. The first experimental example concerns modeling the placement process in a pick‐and‐place machine, while the second one consists in the identification of an LPV model describing the input‐output relationship of an electronic bandpass filter with time‐varying resonant frequency.     

双输入多率系统辅助模型增广随机梯度算法

针对有色噪声干扰的双输入多率系统,为解决辨识模型信息向量中存在未知变量和不可测噪声项的问题,结合辅助模型思想和递推增广随机梯度算法的优点,用辅助模型的输出代替系统的未知变量,用估计残差代替信息向量中的不可测噪声项,进而提出了双输入多率系统的辅助模型增广随机梯度算法。为了提高辨识算法的收敛速度和改善参数估计精度,在算法中引入遗忘因子,得到相应的辅助模型带遗忘因子增广随机梯度算法。仿真实例说明,引入遗忘因子,能加快算法的收敛性,提高参数估计精度。