Variance-error quantification for identified poles and zeros

This paper deals with quantification of noise induced errors in identified discrete-time models of causal linear time-invariant systems, where the model error is described by the asymptotic (in data length) variance of the estimated poles and zeros. The main conclusion is that there is a fundamental difference in the accuracy of the estimates depending on whether the zeros and poles lie inside or outside the unit circle. As the model order goes to infinity, the asymptotic variance approaches a finite limit for estimates of zeros and poles having magnitude larger than one, but for zeros and poles strictly inside the unit circle the asymptotic variance grows exponentially with the model order. We analyze how the variance of poles and zeros is affected by model order, model structure and input excitation. We treat general black-box model structures including ARMAX and Box–Jenkins models.     

On the accuracy of a covariance matching method for continuous-time errors-in-variables identification

An analysis of a covariance matching method for continuous-time errors-in-variables system identification from discrete-time data is made. In the covariance matching method, the noise-free input signal is not explicitly modeled and only assumed to be a stationary process. The asymptotic normalized covariance matrix, valid for a large number of data and a small sampling interval, is derived. This involves the evaluation of a covariance matrix of estimated covariance elements and estimated derivatives of such elements, and large parts of the paper are devoted to this task. The latter covariance matrix consists of two parts, where the first part contains integrals that are approximations of Riemann sums, and the second part depends on the measurement noise variances.     

An improved bias-compensation approach for errors-in-variables model identification

Parametric estimation of the dynamic errors-in-variables models is considered in this paper. In particular, a bias compensation approach is examined in a generalized framework. Sufficient conditions for uniqueness of the identified model are presented. Subsequently, a statistical accuracy analysis of the estimation algorithm is carried out. The asymptotic covariance matrix of the system parameter estimates depends on a user chosen filter and a certain weighting matrix. It is shown how these can be tuned to boost the estimation performance. The numerical simulation results suggest that the covariance matrix of the estimated parameter vector is very close to the Cramér-Rao lower bound for the estimation problem.     

A user perspective on errors-in-variables methods in system identification

The paper starts with a brief survey of errors-in-variables methods in system identification. Background and motivation are given, and it is illustrated why the identification problem can be difficult. Under general weak assumptions, the system is not identifiable, but can be parameterized using one degree of freedom. Examples where identifiability is achieved under additional assumptions are also provided. A number of approaches for parameter estimation of errors-in-variables models are reviewed. The underlying assumptions and principles for each approach are highlighted. The paper then continues by discussing from a user’s perspective on how to proceed when practical problems are handled.     

Unifying some higher-order statistic-based methods for errors-in-variables model identification

In this paper, the problem of identifying linear discrete-time systems from noisy input and output data is addressed. Several existing methods based on higher-order statistics are presented. It is shown that they stem from the same set of equations and can thus be united from the viewpoint of extended instrumental variable methods. A numerical example is presented which confirms the theoretical results. Some possible extensions of the methods are then given.     

Accuracy analysis of bias-eliminating least squares estimates for errors-in-variables systems

The bias-eliminating least squares (BELS) method is one of the consistent estimators for identifying dynamic errors-in-variables systems. In this paper, we investigate the accuracy properties of the BELS estimates. An explicit expression for the normalized asymptotic covariance matrix of the estimated parameters is derived and supported by some numerical examples.     

Third-order cumulants based methods for continuous-time errors-in-variables model identification

In this paper, the problem of identifying stochastic linear continuous-time systems from noisy input/output data is addressed. The input of the system is assumed to have a skewed probability density function, whereas the noises contaminating the data are assumed to be symmetrically distributed. The third-order cumulants of the input/output data are then (asymptotically) insensitive to the noises, that can be coloured and/or mutually correlated. Using this noise-cancellation property two computationally simple estimators are proposed. The usefulness of the proposed algorithms is assessed through a numerical simulation.     

The cost of complexity in system identification: The Output Error case

In this paper we investigate the cost of complexity, which is defined as the minimum amount of input power required to estimate the frequency response of a given linear time invariant system of order n with a prescribed degree of accuracy. In particular we require that the asymptotic (in the data length) variance is less or equal to γ over a prespecified frequency range [0,ω_B]. The models considered here are Output Error models, with an emphasis on fixed denominator and Laguerre models. Several properties of the cost are derived. For instance, we present an expression which shows how the pole of the Laguerre model affects the cost. These results quantify how the cost of the system identification experiment depends on n and on the model structure. Also, they show the relation between the cost and the amount of information we would like to extract from the system (in terms of ω_B and γ). For simplicity we assume that there is no undermodelling.     

Identification of delay dominant recycle systems

A new identification method for single-input–single-output delay dominant recycle systems is presented in this paper. Identification of recycle systems is similar to that of closed loop systems. However, identification of recycle systems poses certain challenges in that the input-equivalent signal in the closed loop identification is not available for recycle systems. Therefore, special identification routines are required to ensure consistency of recycle and forward models. It is shown that consistent estimates of delay dominant recycle systems can be obtained by treating a delayed output as one of the inputs. Asymptotic variance expressions for the estimates of forward and recycle models are provided. These are then used in designing an optimal excitation signal for recycle systems. The results are illustrated through an industrial example.     

A generalized instrumental variable estimation method for errors-in-variables identification problems

The errors-in-variables identification problem concerns dynamic systems whose input and output variables are affected by additive noise. Several estimation methods have been proposed for identifying dynamic errors-in-variables models. In this paper it is shown how a number of common methods for errors-in-variables methods can be put into a general framework, resulting into a Generalized Instrumental Variable Estimator (GIVE). Various computational aspects of GIVE are presented, and the asymptotic distribution of the parameter estimates is derived.     

On the frequency domain accuracy of closed-loop estimates

It has been argued that the frequency domain accuracy of high model-order estimates obtained on the basis of closed-loop data is largely invariant to whether direct or indirect approaches are used. The analysis underlying this conclusion has employed variance expressions that are asymptotic both in the data length and the model order, and hence are approximations when either of these are finite. However, recent work has provided variance expressions that are exact for finite (possibly low) model order, and hence can potentially deliver more accurate quantification of estimation accuracy. This paper, and a companion one, revisits the study of identification from closed-loop data in light of these new quantifications and establishes that, under certain assumptions, there can be significant differences in the accuracy of frequency response estimates. These discrepencies are established here and in the companion paper to be dependent on what type of direct, indirect or joint input-output identification strategy is pursued.     

On the lower smoothing bound in identification of time-varying systems

In certain applications of nonstationary system identification the model-based decisions can be postponed, i.e. executed with a delay. This allows one to incorporate in the identification process not only the currently available information, but also a number of “future” data points. The resulting estimation schemes, which involve smoothing, are not causal. Assuming that the infinite observation history is available, the paper establishes the lower steady-state estimation bound for any noncausal estimator applied to a linear system with randomly drifting coefficients (under Gaussian assumptions). This lower bound complements the currently available one, which is restricted to causal estimators.     

The asymptotic model quality assessment for instrumental variable identification revisited

In this paper the problem of computing uncertainty regions for models identified through an instrumental variable technique is considered. Recently, it has been pointed out that, in certain operating conditions, the asymptotic theory of system identification (the most widely used method for model quality assessment) may deliver unreliable confidence regions. The aim of this paper is to show that, in an instrumental variable setting, the asymptotic theory exhibits a certain “robustness” that makes it reliable even with a moderate number of data samples. Reasons for this are highlighted in the paper through a theoretical analysis and simulation examples.     

Enforcing stability constraints in set-membership identification of linear dynamic systems

In this paper, we consider the identification of linear systems, a priori known to be stable, from input–output data corrupted by bounded noise. By taking explicitly into account a priori information on system stability, a formal definition of the feasible parameter set for a stable linear system is provided. On the basis of a detailed analysis of the geometrical structure of the feasible set, convex relaxation techniques are presented to solve nonconvex optimization problems arising in the computation of parameter uncertainty intervals. Properties of the computed relaxed bounds are discussed. A simulated example is presented to show the effectiveness of the proposed technique.     

Variance analysis of identified linear MISO models having spatially correlated inputs,with application to parallel Hammerstein models

This contribution concerns variance analysis of linear multi-input single-output models when the inputs are temporally white but where different inputs may be correlated. An expression is provided for the variance of a linearly parametrized estimate of the frequency response function from one block, i.e. from one input to the output. In particular, this expression reveals that the variance increases in one block when the number of estimated parameters in another block is increased, but levels off when the number of parameters in the other block reaches the number of parameters in the block in question. It also quantifies exactly how correlation between inputs affects the resulting accuracy and a graphical representation is provided for this purpose. The results are applicable to parallel MISO Hammerstein models when the nonlinearities are known and generalize an existing variance expression for this type of model.     

Accuracy analysis of time domain maximum likelihood method and sample maximum likelihood method for errors-in-variables and output error identification

For identifying errors-in-variables models, the time domain maximum likelihood (TML) method and the sample maximum likelihood (SML) method are two approaches. Both methods give optimal estimation accuracy but under different assumptions. In the TML method, an important assumption is that the noise-free input signal is modelled as a stationary process with rational spectrum. For SML, the noise-free input needs to be periodic. It is interesting to know which of these assumptions contain more information to boost the estimation performance. In this paper, the estimation accuracy of the two methods is analyzed statistically for both errors-in-variables (EIV) and output error models (OEM). Numerical comparisons between these two estimates are also done under different signal-to-noise ratios (SNRs). The results suggest that TML and SML have similar estimation accuracy at moderate or high SNR for EIV. For OEM identification, these two methods have the same accuracy at any SNR.     

On noncausal weighted least squares identification of nonstationary stochastic systems

In this paper, we consider the problem of noncausal identification of nonstationary, linear stochastic systems, i.e., identification based on prerecorded input/output data. We show how several competing weighted (windowed) least squares parameter smoothers, differing in memory settings, can be combined together to yield a better and more reliable smoothing algorithm. The resulting parallel estimation scheme automatically adjusts its smoothing bandwidth to the unknown, and possibly time-varying, rate of nonstationarity of the identified system. We optimize the window shape for a certain class of parameter variations and we derive computationally attractive recursive smoothing algorithms for such an optimized case.     

Identification of dynamic errors-in-variables models: Approaches based on two-dimensional ARMA modeling of the data

In this paper we propose a parametric and a non-parametric identification algorithm for dynamic errors-in-variables model. We show that the two-dimensional process composed of the input-output data admits a finite order ARMA representation. The non-parametric method uses the ARMA structure to compute a consistent estimate of the joint spectrum of the input and the output. A Frisch scheme is then employed to extract an estimate of the joint spectrum of the noise free input-output data, which in turn is used to estimate the transfer function of the system. The parametric method exploits the ARMA structure to give estimates of the system parameters. The performances of the algorithms are illustrated using the results obtained from a numerical simulation study.     

System identification from noisy measurements of inputs and outputs

In this paper a system identification method is described for the case of measurement errors on inputs and outputs. The method gives consistent estimates of the parameters and in case of normal measurement errors maximum likelihood estimates are obtained. More specific statistical properties of the estimators are also provided. Furthermore, the sensitivity of the results with respect to the assumptions is studied.