Performance analysis of multi-innovation gradient type identification methods

It is well-known that the stochastic gradient (SG) identification algorithm has poor convergence rate. In order to improve the convergence rate, we extend the SG algorithm from the viewpoint of innovation modification and present multi-innovation gradient type identification algorithms, including a multi-innovation stochastic gradient (MISG) algorithm and a multi-innovation forgetting gradient (MIFG) algorithm. Because the multi-innovation gradient type algorithms use not only the current data but also the past data at each iteration, parameter estimation accuracy can be improved. Finally, the performance analysis and simulation results show that the proposed MISG and MIFG algorithms have faster convergence rates and better tracking performance than their corresponding SG algorithms.     

Identification of Hammerstein nonlinear ARMAX systems

Two identification algorithms, an iterative least-squares and a recursive least-squares, are developed for Hammerstein nonlinear systems with memoryless nonlinear blocks and linear dynamical blocks described by ARMAX/CARMA models. The basic idea is to replace unmeasurable noise terms in the information vectors by their estimates, and to compute the noise estimates based on the obtained parameter estimates. Convergence properties of the recursive algorithm in the stochastic framework show that the parameter estimation error consistently converges to zero under the generalized persistent excitation condition. The simulation results validate the algorithms proposed.     

Exact filters for Newton–Raphson parameter estimation algorithms for continuous-time partially observed stochastic systems

This paper presents explicit finite-dimensional filters for implementing Newton–Raphson (NR) parameter estimation algorithms. The models which exhibit nonlinear parameter dependence are stochastic, continuous-time and partially observed. The implementation of the NR algorithm requires evaluation of the log-likelihood gradient and the Fisher information matrix. Fisher information matrices are important in bounding the estimation error from below, via the Cramer–Rao bound. The derivations are based on relations between incomplete and complete data, likelihood, gradient and Hessian likelihood functions, which are derived using Girsanov's measure transformations.     

Frequency domain identification of Wiener models

This paper proposes a frequency domain algorithm for Wiener model identifications based on exploring the fundamental frequency and harmonics generated by the unknown nonlinearity. The convergence of the algorithm is established in the presence of white noise. No a priori knowledge of the structure of the nonlinearity is required and the linear part can be nonparametric.     

基于辅助模型的多新息广义增广随机梯度算法

将辅助模型辨识思想与多新息辨识理论相结合,利用系统可测信忠建立一个辅助模型.分别用辅助模型输出和噪声估计值代替辨识模型信忠向量中未知真实输出变量和不可测噪声项,并引入新忠长度扩展标量新息为新息向量,提出了Box-lenkins模型的辅助模型多新忠广义增广随机梯度辨识方法.所提出方法重复使用系统数据,能够改善参数估计精度,加快算法的收敛速度.     

Parameter identification of fractional-order Wiener system based on FF-ESG and GI algorithms

Fractional-order calculus has broad application scenarios in engineering and physics. Unlike integer-order calculus, fractional-order calculus has the ability to analyze nonclassical phenomena in science and engineering. For industrial processes with strong nonlinear characteristics, nonlinear models such as the Wiener model have become research hotspots. This paper studies the parameter identification of the fractional-order Wiener system. In this paper, the forgetting factor extended stochastic gradient (FF-ESG) algorithm and the gradient iterative (GI) algorithm are proposed to identify the parameters of the fractional-order Wiener system. Then, the convergence of the FF-ESG algorithm for the fractional-order Wiener system is analyzed. Both proposed algorithms can obtain exact parameter estimates, which are verified by a numerical example and a case study of a fluid control valve.     

Auxiliary model-based least-squares identification methods for Hammerstein output-error systems

The difficulty in identification of a Hammerstein (a linear dynamical block following a memoryless nonlinear block) nonlinear output-error model is that the information vector in the identification model contains unknown variables—the noise-free (true) outputs of the system. In this paper, an auxiliary model-based least-squares identification algorithm is developed. The basic idea is to replace the unknown variables by the output of an auxiliary model. Convergence analysis of the algorithm indicates that the parameter estimation error consistently converges to zero under a generalized persistent excitation condition. The simulation results show the effectiveness of the proposed algorithms.     

Least squares based iterative identification for a class of multirate systems

This paper studies modeling and identification problems for multi-input multirate systems with colored noises. The state-space models are derived for the systems with different input updating periods and furthermore the corresponding transfer functions are obtained. To solve the difficulty of identification models with unmeasurable noises terms, the least squares based iterative algorithm is presented by replacing the unmeasurable variables with their iterative estimates. Finally, the simulation results indicate that the proposed iterative algorithm has advantages over the recursive algorithms.     

MINLIP for the identification of monotone Wiener systems

This paper studies the MINLIP estimator for the identification of Wiener systems consisting of a sequence of a linear FIR dynamical model, and a monotonically increasing (or decreasing) static function. Given T observations, this algorithm boils down to solving a convex quadratic program with O(T) variables and inequality constraints, implementing an inference technique which is based entirely on model complexity control, a technique which is dubbed as Einstein’s razor.² This technique is proven to yield, here, almost consistent estimates in case the given data is suitably rich (local Persistently Exciting of sufficient order), and if the samples obey a ‘true’ monotone Wiener system satisfying some regularity conditions. It is then indicated how to extend the method in order to cope with noisy data, and empirical evidence is given in order to support the claim of efficiency.     

Maximum likelihood stochastic gradient estimation for Hammerstein systems with colored noise based on the key term separation technique

This paper considers the identification problems of Hammerstein finite impulse response moving average (FIR-MA) systems using the maximum likelihood principle and stochastic gradient method based on the key term separation technique. In order to improve the convergence rate, a maximum likelihood multi-innovation stochastic gradient algorithm is presented. The simulation results show that the proposed algorithms can effectively estimate the parameters of the Hammerstein FIR-MA systems.     

Performance analysis of stochastic gradient algorithms under weak conditions

By using the stochastic martingale theory, convergence properties of stochastic gradient （SG） identification algorithms are studied under weak conditions. The analysis indicates that the parameter estimates by the SG algorithms consistently converge to the true parameters, as long as the information vector is persistently exciting （i.e., the data product moment matrix has a bounded condition number） and that the process noises are zero mean and uncorrelated. These results remove the strict assumptions, made in existing references, that the noise variances and high-order moments exist, and the processes are stationary and ergodic and the strong persis- tent excitation condition holds. This contribution greatly relaxes the convergence conditions of stochastic gradient algorithms. The simulation results with bounded and unbounded noise variances confirm the convergence conclusions proposed.     

Strong consistence of recursive identification for Wiener systems

The paper concerns identification of the Wiener system consisting of a linear subsystem followed by a static nonlinearity f(·) with no invertibility and structure assumption. Recursive estimates are given for coefficients of the linear subsystem and for the value f(v) at any fixed v. The main contribution of the paper consists in establishing convergence with probability one of the proposed algorithms to the true values. This probably is the first strong consistency result for this kind of Wiener systems. A numerical example is given, which justifies the theoretical analysis.     

Convergence of normalized iterative identification of Hammerstein systems

An iterative identification algorithm of Hammerstein systems needs a proper initial condition to guarantee its convergence. In this paper, we propose a new algorithm by fixing the norm of the parameter estimates. The normalized algorithm ensures the convergence property under arbitrary nonzero initial conditions. The proofs of the property also give a geometrical explanation on why the normalization guarantees the convergence. An additional contribution is that the static function in the Hammerstein system is extended to square-integrable functions.     

Identification of Wiener systems with binary-valued output observations

This work is concerned with identification of Wiener systems whose outputs are measured by binary-valued sensors. The system consists of a linear FIR (finite impulse response) subsystem of known order, followed by a nonlinear function with a known parametrization structure. The parameters of both linear and nonlinear parts are unknown. Input design, identification algorithms, and their essential properties are presented under the assumptions that the distribution function of the noise is known and the nonlinearity is continuous and invertible. It is shown that under scaled periodic inputs, identification of Wiener systems can be decomposed into a finite number of core identification problems. The concept of joint identifiability of the core problem is introduced to capture the essential conditions under which the Wiener system can be identified with binary-valued observations. Under scaled full-rank conditions and joint identifiability, a strongly convergent algorithm is constructed. The algorithm is shown to be asymptotically efficient for the core identification problem, hence achieving asymptotic optimality in its convergence rate. For computational simplicity, recursive algorithms are also developed.     

Recursive identification of errors-in-variables Wiener systems

This paper considers the recursive identification of errors-in-variables (EIV) Wiener systems composed of a linear dynamic system followed by a static nonlinearity. Both the system input and output are observed with additive noises being ARMA processes with unknown coefficients. By a stochastic approximation incorporated with the deconvolution kernel functions, the recursive algorithms are proposed for estimating the coefficients of the linear subsystem and for the values of the nonlinear function. All the estimates are proved to converge to the true values with probability one. A simulation example is given to verify the theoretical analysis.     

非均匀Hammerstein-Wiener 系统的递阶随机梯度辨识算法

针对一类非均匀数据采样Hammerstein-Wiener 系统, 提出一种递阶多新息随机梯度算法. 首先基于提升技术, 推导出系统的状态空间模型, 并考虑因果约束关系, 将该模型分解成两个子系统, 利用多新息遗忘随机梯度算法辨识出此模型的参数; 然后, 引入可变遗忘因子, 提出一种修正函数并在线确定其大小, 提高了算法的收敛速度及抗干扰能力. 仿真实例验证了所提出算法的有效性和优越性.

     

Convergence of fixed‐point iteration for the identification of Hammerstein and Wiener systems

Convergence property of the iterative algorithm for Hammerstein or Wiener systems is generally hard to establish because of the existence the unmeasurable internal variables in such systems. In this paper, a fixed‐point iteration is introduced to identifying both Hammerstein and Wiener systems with a unified algorithm. This newly proposed estimation algorithm gives consistent estimates under arbitrary nonzero initial conditions. In addition, the errors of the estimates are established as functions of the noise variance, and thus how the noise affects the quality of parameter estimates for a finite number of data points is made clear. Copyright © 2012 John Wiley & Sons, Ltd.     

广义离散随机线性系统降阶Wiener滤波、平滑和预报器

应用现代时间序列方法 ,基于自回归滑动平均 (ARMA)新息模型、白噪声估值器和观测预报器 ,对于广义离散随机线性系统 ,提出了降阶Wiener状态估值器 ,可统一处理滤波、平滑和预报问题 ,并且能减少计算负担 .仿真例子说明了算法的有效性     

Parameter identification technique for multivariate stochastic systems

This paper considers the problem of obtaining accurate estimates of multivariate systems with reasonable computations. To avoid the structural identification problem which is associated with multivariate systems, we observe the system by a linear combination of the outputs. The two stage least square method is employed to estimate the model parameters. An optimum combination of the outputs is obtained such that the parameter estimates have the least asymptotic generalized variance. Computer simulations are provided to illustrate the usefulness of the proposed method.     

L2–L∞ filtering for nonlinear stochastic systems

A sufficient condition for a general nonlinear stochastic system to have L₂?L_∞ gain less than or equal to a prescribed positive number is established in terms of a certain Hamilton–Jacobi inequality (HJI). Based on this criterion, the existence of an L₂?L_∞ filter is given by a second‐order nonlinear HJI, and the filter matrices can be obtained by solving such an HJI. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society