Iterative identification methods for a class of bilinear systems by using the particle filtering technique

This article mainly studies the iterative parameter estimation problems of a class of nonlinear systems. Based on the auxiliary model identification idea, this article utilizes the estimated parameters to construct an auxiliary model, and uses its outputs to replace the unknown noise-free process outputs, and develops an auxiliary model least squares-based iterative (AM-LSI) identification algorithm. For further improving the parameter estimation accuracy, we use a particle filter to estimate the unknown noise-free process outputs, and derive a particle filtering least squares-based iterative (PF-LSI) identification algorithm. During each iteration, the AM-LSI and PF-LSI algorithms can make full use of the measured input–output data. The simulation results indicate that the proposed algorithms are effective for identifying the nonlinear systems, and can generate more accurate parameter estimates than the auxiliary model-based recursive least squares algorithm.     

On some parameter estimation algorithms for the nonlinear exponential autoregressive model

Modeling an exponential autoregressive (ExpAR) time series is the basis of solving the corresponding prediction and control problems. This paper investigates the hierarchical parameter estimation methods for the ExpAR model. By the hierarchical identification principle, the original nonlinear optimization problem is transformed into the combination of a linear and nonlinear optimization problem, and then, we derive a hierarchical least squares and stochastic gradient (LS‐SG) algorithm. Given the difficulty of determining the step‐size in the hierarchical LS‐SG algorithm, an approach is proposed to obtain the optimal step‐size. To improve the parameter estimation accuracy, the multi‐innovation identification theory is employed to develop a hierarchical least squares and multi‐innovation stochastic gradient algorithm for the ExpAR model. Two simulation examples are provided to test the effectiveness of the proposed algorithms.     

Hierarchical recursive least squares algorithms for Hammerstein nonlinear autoregressive output-error systems

This article considers the parameter estimation problem of Hammerstein nonlinear autoregressive output-error systems with autoregressive moving average noises. Applying the key term separation technique, the original system is decomposed into three subsystems: the first subsystem contains the unknown parameters related to the output, the second subsystem contains the unknown parameters related to the input, and the third subsystem contains the unknown parameters related to the noise model. A hierarchical recursive least squares algorithm is proposed based on the hierarchical identification principle for interactively identifying each subsystem. The simulation results confirm that the proposed algorithm is effective in estimating the parameters of Hammerstein nonlinear autoregressive output-error systems.     

A dual filtering scheme for nonlinear active noise control

Active noise control problems are often affected by nonlinear effects such as distortion and saturation of measurement and actuation devices, which call for suitable nonlinear models and algorithms. The active noise control problem can be interpreted as an indirect model identification problem, due to the secondary path dynamics that follow the control filter block. This complicates the weight update mechanism in the nonlinear case, in that the error gradient depends on the secondary path gradient through nonlinear recursions. A simpler and computationally less demanding approach is here proposed that employs the updating scheme of the standard filtered‐x least mean squares (LMS) or filtered‐u LMS algorithm. As in those schemes, the calculation of the error gradient requires a signal filtering through an auxiliary system, here obtained through a secondary adaptation loop. The resulting dual filtering LMS algorithm performs the adaptation of the controller parameters in a direct identification mode and can therefore be easily coupled with adaptive model structure selection schemes to provide online tuning of the model structure, for improved model robustness. Copyright © 2013 John Wiley & Sons, Ltd.     

The filtering‐based maximum likelihood iterative estimation algorithms for a special class of nonlinear systems with autoregressive moving average noise using the hierarchical identification principle

For a special class of nonlinear systems (ie, bilinear systems) with autoregressive moving average noise, this paper gives the input‐output representation of the bilinear systems through eliminating the state variables in the model. Based on the obtained model and the maximum likelihood principle, a filtering‐based maximum likelihood hierarchical gradient iterative algorithm and a filtering‐based maximum likelihood hierarchical least squares iterative algorithm are developed for identifying the parameters of bilinear systems with colored noises. The original bilinear systems are divided into three subsystems by using the data filtering technique and the hierarchical identification principle, and they are identified respectively. Compared with the gradient‐based iterative algorithm and the multi‐innovation stochastic gradient algorithm, the proposed algorithms have higher computational efficiency and parameter estimation accuracy. The simulation results indicate that the proposed algorithms are effective for identifying bilinear systems.     

A search algorithm for the identification of multiple inputs nonlinear systems using orthogonal least squares estimator

A search algorithm for the identification of multiple inputs nonlinear systems using the orthogonal least squares estimator is derived. Because of the high dimensionality of general nonlinear systems the forward regression algorithm is used to detect the plausible size of the final fitted model and a variation of the forward regression algorithm is proposed. Instead of choosing the best candidate term at each iteration, top few candidate terms which have the largest error reduction ratios are investigated at each iteration. A search algorithm coupled with the model predicted output is derived which will sort through all plausible candidate terms to produce an optimal solution for the problem. Simulated and experimental examples are included to demonstrate the effectiveness of the proposed algorithm.     

Stable adaptive identification of fully-coupled second-order 6 degree-of-freedom nonlinear plant models for underwater vehicles: Theory and experimental evaluation

This article reports the development, stability analysis, and experimental evaluation of a novel adaptive identification (AID) algorithm for underwater vehicles (UVs) for on-line estimation of plant parameters (hydrodynamic mass, quadratic drag, righting moment, and buoyancy parameters) that enter linearly into 6 degree-of-freedom (6-DOF) second-order rigid-body UV plant dynamic models. The reported UV AID method does not require instrumentation of vehicle acceleration as is required of other standard plant parameter identification methods such as conventional least squares. All but one previously reported adaptive methods for second-order nonlinear plants have addressed the problem of model-based adaptive tracking control—approaches in which adaptive plant model identification is performed simultaneously with model-based trajectory-tracking control of fully-actuated second-order plants; however, these approaches are not applicable when the plant is either uncontrolled, under open-loop control, underactuated, or using any control law other than an algorithm-specific adaptive tracking controller. The UV AID algorithm reported herein does not require simultaneous reference trajectory-tracking control, nor does it require instrumentation of linear acceleration or angular acceleration; thus this novel approach complements previously reported adaptive tracking methods and is applicable to a broader class of UV applications for which fully-actuated tracking control is impractical or infeasible. We report a experimental performance analysis of the UV AID algorithm in comparison to conventional least-square identification methods, including comparison in cross-validation where the performance of the experimentally identified plant models obtained in identification trials are compared to experimental trials differing from the identification trials.     

A fuzzy C‐regression model algorithm using a new PSO algorithm

In this paper, a new methodology is introduced for the identification of the parameters of the multiple‐input–multiple‐output local linear Takagi‐Sugeno fuzzy models using the weighted recursive least squares (WRLS). The WRLS is sensitive to initialization, which leads to no convergence. In order to overcome this problem, adaptive chaos particle swarm optimization is proposed to optimize the initial states of WRLS. This new algorithm is improved versions of the original particle swarm optimization algorithm. Finally, comparative experiments are designed to verify the validity of the proposed clustering algorithm and the Takagi‐Sugeno fuzzy model identification method, and the results show that the new method is effective in describing a complicated nonlinear system with significantly high accuracies compared with approaches in the literature.     

The modified extended Kalman filter based recursive estimation for Wiener nonlinear systems with process noise and measurement noise

This article develops the modified extended Kalman filter based recursive estimation algorithms for Wiener nonlinear systems with process noise and measurement noise. The prior estimate of the linear block output is computed based on the auxiliary model, and the posterior estimate is updated by designing a modified extended Kalman filter. A multi-innovation gradient algorithm and a recursive least squares algorithm are derived to estimate the parameters of the linear subsystem, respectively. The simulation examples are provided to demonstrate the effectiveness of the proposed algorithms.     

Least‐squares parameter estimation for systems with irregularly missing data

This paper considers the problems of parameter identification and output estimation with possibly irregularly missing output data, using output error models. By means of an auxiliary model (or reference model) approach, we present a recursive least‐squares algorithm to estimate the parameters of missing data systems, and establish convergence properties for the parameter and missing output estimation in the stochastic framework. The basic idea is to replace the unmeasurable inner variables with the output of an auxiliary model. Finally, we test the effectiveness of the algorithm with an example system. Copyright © 2009 John Wiley & Sons, Ltd.     

Maximum likelihood hierarchical least squares-based iterative identification for dual-rate stochastic systems

For a dual-rate sampled-data stochastic system with additive colored noise, a dual-rate identification model is obtained by using the polynomial transformation technique, which is suitable for the available dual-rate measurement data. Based on the obtained model, a maximum likelihood least squares-based iterative (ML-LSI) algorithm is presented for identifying the parameters of the dual-rate sampled-data stochastic system. In order to improve the computation efficiency of the algorithm, the identification model of a dual-rate sampled-data stochastic system is divided into two subidentification models with smaller dimensions and fewer parameters, and a maximum likelihood hierarchical least squares-based iterative (H-ML-LSI) algorithm is proposed for these subidentification models by using the hierarchical identification principle. The simulation results indicate that the proposed algorithms are effective for identifying dual-rate sampled-data stochastic systems and the H-ML-LSI algorithm has a higher computation efficiency than the ML-LSI algorithm.     

Hierarchical recursive signal modeling for multifrequency signals based on discrete measured data

This paper studies the problem of parameter estimation for the multifrequency sine signals, which have multiple characteristic parameters such as the amplitudes, phases, and frequencies. It is noted that the signal output is nonlinear with respect to the phase and frequency parameters while it is linear with respect to the amplitude parameters. This feature inspires us to separate all of the characteristic parameters into a linear parameter set and a nonlinear parameter set, where the linear set is composed of the amplitude parameters and the nonlinear set is composed of the phase parameters and the frequency parameters. After the parameter separation, two identification submodels are constructed for optimizing the linear parameter set and the nonlinear parameter set. Then the nonlinear identification model becomes a linear identification submodel and a nonlinear identification submodel. Therefore, the nonlinear optimization for minimizing the objective function is converted into the combination of the quadratic optimization and nonlinear optimization. Based on the separable identification submodels, a recursive least squares subalgorithm and a recursive gradient subalgorithm are proposed for identifying the linear parameters and nonlinear parameters, respectively. Moreover, an interactive estimation algorithm is designed to remove the related parameter sets between the subalgorithms and a hierarchical identification method is presented by combining the subalgorithms. For the purpose of tracking the time-varying, a forgetting factor is introduced to improve the convergence speed. The numerical examples are provided to qualify the performance of the proposed method based on some performance measures.     

Finite‐model adaptive control using an LS‐like algorithm

Adaptive control problem of a class of discrete‐time nonlinear uncertain systems, of which the internal uncertainty can be characterized by a finite set of functions, is formulated and studied by using an least squares (LS)‐like algorithm to design the feedback control law. For the finite‐model adaptive control problem, this algorithm is proposed as an extension of counterpart of traditional LS algorithm. Stability in sense of pth mean for the closed‐loop system is proved under a so‐called linear growth assumption, which is shown to be necessary in general by a counter‐example constructed in this paper. The main results have been also applied to parametric cases, which demonstrate how to bridge the non‐parametric case and parametric case. Copyright © 2006 John Wiley & Sons, Ltd.     

Identification of dual‐rate systems based on finite impulse response models

Two identification algorithms, a least squares and a correlation analysis based, are developed for dual‐rate stochastic systems in which the output sampling period is an integer multiple of the input updating period. The basic idea is to use auxiliary FIR models to predict unmeasurable noise‐free (true) outputs, and then use these and system inputs to identify parameters of underlying fast single‐rate models. The simulation results indicate that the proposed algorithms are effective. Copyright © 2004 John Wiley & Sons, Ltd.     

Adaptive filtering‐based multi‐innovation gradient algorithm for input nonlinear systems with autoregressive noise

In this paper, by means of the adaptive filtering technique and the multi‐innovation identification theory, an adaptive filtering‐based multi‐innovation stochastic gradient identification algorithm is derived for Hammerstein nonlinear systems with colored noise. The new adaptive filtering configuration consists of a noise whitening filter and a parameter estimator. The simulation results show that the proposed algorithm has higher parameter estimation accuracies and faster convergence rates than the multi‐innovation stochastic gradient algorithm for the same innovation length. As the innovation length increases, the filtering‐based multi‐innovation stochastic gradient algorithm gives smaller parameter estimation errors than the recursive least squares algorithm.     

Combined state and parameter estimation for Hammerstein systems with time delay using the Kalman filtering

This paper discusses the state and parameter estimation problem for a class of Hammerstein state space systems with time delay. Both the process and the measurement noises are considered in the system. On the basis of the observable canonical state space form and the key term separation, a pseudolinear regressive identification model is obtained. For the unknown states in the information vector, the Kalman filter is used to search for the optimal state estimates. A Kalman filter–based least squares iterative and a recursive least squares algorithms are proposed. Extending the information vector to include the latest information terms, which are missed for the time delay, the Kalman filter–based recursive extended least squares algorithm is derived to obtain the estimates of the unknown time delay, parameters, and states. The numerical simulation results are given to illustrate the effectiveness of the proposed algorithms.     

A hybrid least squares-GA-based algorithm for harmonic estimation

Harmonic estimation in a distorted signal along with additive noise has been an area of interest for researchers in many disciplines of science and engineering. This paper presents a new algorithm to estimate the harmonics in power systems using genetic algorithms (GAs). The harmonic estimation problem is linear in amplitude and nonlinear in phase. The proposed hybrid algorithm takes advantage of this structure and iterates between linear least squares amplitude estimation and the nonlinear GA-based phase estimation. Improvement in both convergence for solution as well as processing time is demonstrated from this algorithm.     

Chaotic system identification using Wiener‐LSSVM model

In this paper, a new method is proposed for identifying chaotic system based on a Wiener‐least squares support vector machine (Wiener‐LSSVM) model. The model consists of a linear dynamic subsystem followed by a static nonlinear function, which is represented by LSSVM in this paper. The parameters of the linear dynamic part and those of LSSVM are estimated simultaneously by solving a set of linear equations using the least squares (LS) method. The proposed method incorporates partial structure information into the identification process and does not assume that the parameters of linear dynamic part are known. On the other hand, the LS algorithm is more efficient than gradient‐descendent‐based algorithms for estimating the parameters of Wiener‐LSSVM. Three identification examples are given to validate the effectiveness of the proposed method. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

Delta算子离散模型辨识的性能分析

研究了Delta算子描述的离散系统参数辨识问题，基于Delta算子矩阵求逆引理，给出Delta算子递推最小二乘(DRLS)估计公式；分析了DRLS算法的参数误差和预报误差特性。所得结论将连续与离散模型辨识的有关结果统一于Delta算子框架。     

Iterative parameter and order identification for fractional-order nonlinear finite impulse response systems using the key term separation

This article considers the parameter estimation for a fractional-order nonlinear finite impulse response system with colored noise. For the fractional-order systems, the challenge and difficulty are to identify the order and parameters of the systems simultaneously under colored noise disturbances. In order to reduce the problem of redundant parameter estimation, the output form of the system can be expressed by a linear combination of unknown parameters through the separation of the key term separation. A key term separation auxiliary model gradient-based iterative algorithm is derived by using the negative gradient search. Meanwhile, to achieve the higher estimation accuracy, we propose a key term separation auxiliary model multiinnovation gradient-based iterative algorithm by utilizing the multiinnovation theory. Finally, the simulation results test the effectiveness of the proposed algorithms.