A note on sampling and parameter estimation in linear stochasticsystems

Numerical differentiation formulas that yield consistent least squares parameter estimates from sampled observations of linear, time invariant higher order systems have been introduced previously by Duncan et al. (1994). The formulas given by Duncan et al. have the same limiting system of equations as in the continuous time case. The formula presented in this note can be characterized as preserving asymptotically a partial integration rule. It leads to limiting equations for the parameter estimates that are different from the continuous case, but they again imply consistency. The numerical differentiation formulas given here can be used for an arbitrary linear system, which is not the case in the previous paper by Duncan et al     

Identification of Hammerstein nonlinear ARMAX systems

Two identification algorithms, an iterative least-squares and a recursive least-squares, are developed for Hammerstein nonlinear systems with memoryless nonlinear blocks and linear dynamical blocks described by ARMAX/CARMA models. The basic idea is to replace unmeasurable noise terms in the information vectors by their estimates, and to compute the noise estimates based on the obtained parameter estimates. Convergence properties of the recursive algorithm in the stochastic framework show that the parameter estimation error consistently converges to zero under the generalized persistent excitation condition. The simulation results validate the algorithms proposed.     

一类有输入噪声扰动的逆系统无偏参数辨识算法研究

对逆系统建模时,原系统的输出作为逆系统参数辨识时的输入.由于原系统输出存在测量噪声,且噪声方差未知,采用普通最小二乘法辨识,无法得到逆系统参数的一致无偏估计.为此,本文研究了一种有输入扰动的的逆系统无偏参数辨识算法,该算法先通过小波变换估计输入信号噪声的方差,再由估计得到的方差,通过偏差消除的递推最小_乘法,对逆系统的参数进行无偏辨识.该算法降低了对输入辨识信号为白噪声的要求,具有较强的实用性.由于采用递推运算,该算法也可以用于逆系统参数的在线辨识.最后,通过实验验证了该算法的有效性.     

Bias-compensation-based least-squares estimation with a forgetting factor for output error models with white noise

In this paper, the bias-compensation-based recursive least-squares (LS) estimation algorithm with a forgetting factor is proposed for output error models. First, for the unknown white noise, the so-called weighted average variance is introduced. With this weighted average variance, a bias-compensation term is first formulated to achieve the bias-eliminated estimates of the system parameters. Then, the weighted average variance is estimated. Finally, the final estimation algorithm is obtained by combining the estimation of the weighted average variance and the recursive LS estimation algorithm with a forgetting factor. The effectiveness of the proposed identification algorithm is verified by a numerical example.     

Wavelet-Based Identification of Linear Discrete-Time Systems: Robustness Issue

Discrete-time linear time-varying systems are modeled by discrete-time wavelets. The output of the unknown system is corrupted by noise. The system model parameters are estimated by the least-squares method applied to the output error. Conditions are derived that provide vanishing influence of the output noise to the parameter estimates. Due to the time-frequency selectivity of wavelets, parameter estimates can be robust to narrow-band noise and/or impulse noise. This robustness is confirmed by simulations.     

Non-parametric orthogonal series identification of Hammerstein systems

The non-linearity in a discrete system governed by the Hammerstein functional is identified. The system is driven by a random while input signal and the output is disturbed by a random white noise. No parametric a priori information concerning the non-linearity is available and non-parametric algorithms are proposed. The algorithms are derived from the trigonometric as well as Hermite orthogonal series. It is shown that the algorithms converge to the unknown characteristic in a pointwise manner and that the mean integrated square error converges to zero as the number of observations tends to infinity. The rate of convergence is examined. A numerical example is also given.     

Recursive Bayesian Algorithm for Identification of Systems with Non-uniformly Sampled Input Data

To identify systems with non-uniformly sampled input data, a recursive Bayesian identification algorithm with covariance resetting is proposed. Using estimated noise transfer function as a dynamic filter, the system with colored noise is transformed into the system with white noise. In order to improve estimates, the estimated noise variance is employed as a weighting factor in the algorithm. Meanwhile, a modified covariance resetting method is also integrated in the proposed algorithm to increase the convergence rate. A numerical example and an industrial example validate the proposed algorithm.     

Estimation of the autoregressive parameters of a mixed autoregressive moving-average time series

The problem of estimating the autoregressive parameters of a mixed autoregressive moving-average (ARMA) time series (of known order) using the output data alone is treated. This problem is equivalent to the estimation of the denominator terms of the scalar transfer function of a stationary, linear discrete time system excited by an unobserved unenrrelated sequence input by employing only the observations of the scalar output. The solution of this problem solves the problem of the identification of the dynamics of a white-noise excited continuous-time linear stationary system using sampled data. The latter problem was suggested by Bartlett in 1946. The problem treated here has appeared before in the engineering literature. The earlier treatment yielded biased parameter estimates. An asymptotically unbiased estimator of the autoregressive parameters is obtained as the solution of a modified set of Yule-Walker equations. The asymptotic estimator covariance matrix behaves like a least-squares parameter estimate of an observation set with unknown error covariances. The estimators are also shown to be unbiased in the presence of additive independent observation noise of arbitrary finite correlation time. An example illustrates the performance of the estimating procedures.     

一类MIMO状态可测的非线性连续系统的激励辨识方法

研究一类 MIMO 状态可测的非线性连续系统的激励辨识问题. 输入激励信号采用高斯白噪声, 均匀采样获得输出状态数据. 根据 Girsanov 定理获得系统参数的渐近无偏估计. 数值仿真试验说明了该方法的有效性并发现耦合多变量系统辨识中的 NNR 现象. 最后给出该系统的分步激励辨识算法.     

A least-squares/penalty method for distributed optimal control problems for Stokes equations

The purpose of this paper is to construct an unconstrained optimal control problem by using a least-squares approach for the constrained distributed optimal control problem associated with incompressible Stokes equations. The constrained equations are reformulated to the equivalent first-order system by introducing vorticity, and then the least-squares functional corresponding to the system is enforced via a penalty term to the objective functional. The existence of a solution of the unconstrained optimal control problem is proved, and the convergence of this solution to that of unpenalized one is demonstrated as the penalty parameter tends to zero. Finite element approximations with error estimates are studied, and the relevant computational experiments are presented.     

A new formula for the log-likelihood gradient for continuous-timestochastic systems

Using a finitely additive white noise approach, we obtain an explicit expression for the gradient of the log-likelihood ratio for system parameter estimation for continuous-time linear stochastic systems with noisy observations. Our gradient formula includes the smoother estimates of the state vector, and derivatives of only the system matrices, and not the estimates or error covariances. A scheme to calculate the log-likelihood gradient without solving a Riccati equation is described when only one matrix and the initial covariance depend on the unknown parameter     

An innovations approach to least-squares estimation--Part III: Nonlinear estimation in white Gaussian noise

In Parts I and II of this paper, we presented the innovations approach to linear least-squares estimation in additive white noise. In the present paper, we show how to extend this technique to the nonlinear estimation (filtering and smoothing) of non-Gaussian signals in additive white Gaussian noise. The use of the innovations allows us to obtain formulas and simple derivations that are remarkably similar to those used for the linear case thereby distinguishing clearly the essential points at which the nonlinear problem differs from the linear one.     

Identification of continuous-time AR processes from unevenly sampled data

When identifying a continuous-time AR process from discrete-time data, an obvious approach is to replace the derivative operator in the continuous-time model by an approximation. In some cases, a linear regression model can then be formulated. The well-known least-squares method would be very desirable to apply, since it enjoy good numerical properties and low computational complexity, in particular for fast or nonuniform sampling. The focus of this paper is the latter, i.e., nonuniform sampling. Two consistent least-squares schemes for the case of unevenly sampled data are presented. The precise choice of derivative approximation turns out to be crucial. The obtained results are compared to a prediction error method.     

Issues in sampling and estimating continuous-time models with stochastic disturbances

The standard continuous time state space model with stochastic disturbances contains the mathematical abstraction of continuous time white noise. To work with well defined, discrete time observations, it is necessary to sample the model with care. The basic issues are well known, and have been discussed in the literature. However, the consequences have not quite penetrated the practice of estimation and identification. One example is that the standard model of an observation, being a snapshot of the current state plus noise independent of the state, cannot be reconciled with this picture. Another is that estimation and identification of time continuous models require a more careful treatment of the sampling formulas. We discuss and illustrate these issues in the current contribution. An application of particular practical importance is the estimation of models based on irregularly sampled observations.     

On the convergence of least squares estimates in white noise

The problem of convergence of least squares (LS) estimates in a stochastic linear regression model with white noise is considered. It is well known that if the parameter estimates are known to converge, the convergence analysis for many adaptive systems can be rendered considerably less arduous. For an important case where the regression vector is a measurable function of the observations and the noise is Gaussian, it has been shown, by using a Bayesian embedding argument, that the LS estimates converge almost surely for almost all true parameters in the parameter space except for a zero-measure set. However, nothing can be said about a particular given system, which is usually the objective. It has long been conjectured that such a “bad” zero measure set in the parameter space does not actually exist. A conclusive answer to this important question is provided and it is shown that the set can indeed exist. This then shows that to provide conclusive convergence results for stochastic adaptive systems, it is necessary to resort to a sample pathwise analysis instead of the Bayesian embedding approach     

The Least-Squares Error for Structure from Infinitesimal Motion

We analyze the least-squares error for structure from motion with a single infinitesimal motion (structure from optical flo). We present asymptotic approximations to the noiseless error over two, complementary regions of motion estimates: roughly forward and non-forward translations. Our approximations are powerful tools for understanding the error. Experiments show that they capture its detailed behavior over the entire range of motions. We illustrate the use of our approximations by deriving new properties of the least-squares error. We generalize the earlier results of Jepson/Heeger/Maybank on the bas-relief ambiguity and of Oliensis on the reflected minimum. We explain the error's complexity and its multiple local minima for roughly forward translation estimates (epipoles within the field of view) and identify the factors that make this complexity likely. For planar scenes, we clarify the effects of the two-fold ambiguity, show the existence of a new, double bas-relief ambiguity, and analyze the error's local minima. For nonplanar scenes, we derive simplified error approximations for reasonable assumptions on the image and scene. For example, we show that the error tends to have a simpler form when many points are tracked. We show experimentally that our analysis for zero image noise gives a good model of the error for large noise. We show theoretically and experimentally that the error for projective structure from motion is simpler but flatter than the error for calibrated images.     

Ensemble data assimilation in the presence of cloud

In numerical weather prediction (NWP) data assimilation (DA) methods are used to combine available observations with numerical model estimates. This is done by minimising measures of error on both observations and model estimates with more weight given to data that can be more trusted. For any DA method an estimate of the initial forecast error covariance matrix is required. For convective scale data assimilation, however, the properties of the error covariances are not well understood.An effective way to investigate covariance properties in the presence of convection is to use an ensemble-based method for which an estimate of the error covariance is readily available at each time step. In this work, we investigate the performance of the ensemble square root filter (EnSRF) in the presence of cloud growth applied to an idealised 1D convective column model of the atmosphere. We show that the EnSRF performs well in capturing cloud growth, but the ensemble does not cope well with discontinuities introduced into the system by parameterised rain. The state estimates lose accuracy, and more importantly the ensemble is unable to capture the spread (variance) of the estimates correctly. We also find, counter-intuitively, that by reducing the spatial frequency of observations and/or the accuracy of the observations, the ensemble is able to capture the states and their variability successfully across all regimes.     

传递函数估计误差的硬界及其渐近性质

本文讨论带遗忘因子的最小二乘法估计传递函数的误差硬界及其渐近性质.在噪声有界 等一定假定下,当样本个数趋向无限时,误差硬界收敛于复平面内的一个圆.     

A generalized autocovariance least-squares method for Kalman filter tuning

This paper discusses a method for estimating noise covariances from process data. In linear stochastic state-space representations the true noise covariances are generally unknown in practical applications. Using estimated covariances a Kalman filter can be tuned in order to increase the accuracy of the state estimates. There is a linear relationship between covariances and autocovariance. Therefore, the covariance estimation problem can be stated as a least-squares problem, which can be solved as a symmetric semidefinite least-squares problem. This problem is convex and can be solved efficiently by interior-point methods. A numerical algorithm for solving the symmetric is able to handle systems with mutually correlated process noise and measurement noise.     

Analysis, design, and performance limitations of optimal filtering in the presence of an additional input with known frequency

In this paper, the inputs are considered to be of two types. The first type of input, as in standard H₂ optimal filtering, is a zero mean wide sense stationary white noise, while the second type is a linear combination of sinusoidal signals each of which has an unknown amplitude and phase but known frequency. The generalized H₂ optimal filtering problem seeks to find a linear stable filter that estimates a desired output such that the H₂ norm of the transfer matrix from the white noise input to the estimation error is minimized subject to the constraint that the mean of the error converges to zero for all initial conditions of the given system and filter and for all possible external sinusoidal signals. The analysis, design, and performance limitations of generalized H₂ optimal filters are presented here.