Worst-case analysis of the least-squares method and related identification methods

We consider worst-case analysis of system identification under less restrictive assumptions on the noise than the l_∞ bounded error condition. It is shown that the least-squares method has a robust convergence property in l₂ identification, but lacks a corresponding property in l₁ identification (as well as in all other non-Hilbert space settings). The latter result is in stark contrast with typical results in asymptotic stochastic analysis of the least-squares method. Furthermore, it is shown that the Khintchine inequality is useful in the analysis of least l^p identification methods.     

A new fast cellular automata orthogonal least-squares identification method

A novel fast cellular automata orthogonal least squares (FCA-OLS) identification method is introduced by extending and developing the CA-OLS identification method presented in a previous study by Billings and Yang. Both one-dimensional CA and two-dimensional CA were identified with FCA-OLS as the simulation examples. The simulation results show that the new method significantly reduces the computational time compared with existing methods.     

A stopping rule for least-squares identification

A stopping rule for least-squares identification is developed. The stopping rule is determined by the construction of a confidence ellipsoid. For any predetermined estimation error ϵ>0, if the iterates are inside of an ellipsoidal confidence region with volume less than or equal to ϵ^r, then the recursive online algorithm will be terminated with high probability     

Newton-Raphson algorithm for the recursive generalized least-squares identification

A new variant of the Newton-Raphson algorithm for the recursive generalized least-squares identification is proposed. As compared to an earlier one [2], it uses the exactly computed Hessian and gradient. In this way, significant improvement of identification accuracy is obtained.     

A nonlinear least-squares approach for identification of the induction motor parameters

A nonlinear least-squares method is presented for the identification of the induction motor parameters. A major difficulty with the induction motor is that the rotor state variables are not available measurements so that the system identification model cannot be made linear in the parameters without overparametrizing the model. Previous work in the literature has avoided this issue by making simplifying assumptions such as a "slowly varying speed." Here, no such simplifying assumptions are made. The problem is formulated as a nonlinear least-squares identification problem and uses elimination theory (resultants) to compute the parameter vector that minimizes the residual error. The only requirement is that the system must be sufficiently excited. The method is suitable for online operation to continuously update the parameter values. Experimental results are presented.     

Choice of least-squares algorithms for the identification of ARM AX dynamic systems

A comparative analysis of three identification algorithms, which minimise the sum of the squared errors between the model output and the observations of the output, is presented. Batch least-squares and extended recursive least-squares algorithms can be used to identify the model structure goodness of ARMAX application of each algorithm are investigated.     

Auxiliary model-based least-squares identification methods for Hammerstein output-error systems

The difficulty in identification of a Hammerstein (a linear dynamical block following a memoryless nonlinear block) nonlinear output-error model is that the information vector in the identification model contains unknown variables—the noise-free (true) outputs of the system. In this paper, an auxiliary model-based least-squares identification algorithm is developed. The basic idea is to replace the unknown variables by the output of an auxiliary model. Convergence analysis of the algorithm indicates that the parameter estimation error consistently converges to zero under a generalized persistent excitation condition. The simulation results show the effectiveness of the proposed algorithms.     

Moving identification via weighted least-squares estimation

Moving identification using weighted least-squares estimation is developed. Two weighting factors are used: one for discarding old data, the other for adding new data, and these two weighting factors are chosen to ensure that the estimated parameters are convergent. The sufficient conditions for the convergent property are also presented. The proposed method is suitable for time-varying systems.     

Analysis of the identification of closed-loop systems using least-squares methods

This paper presents theory and results concerning the analysis of the identification of closed-loop systems using least-squares methods. The least-squares technique is applied in its normal form and in a modified version developed to cope with the bias problem. The analysis has been established following a mathematical investigation of the problem, and by simulation of different identification experiments applied to different structures of closed-loop systems.

The results derived from this analysis show the conditions under which the identifiability of the open-loop process can be ensured, considering different situations such as whether or not there is noise present at the system output and whether or not external signals are used to perform the identification experiments.

Practical experiments of closed-loop identification on a micromachine system in use in the Department of Electrical Engineering of the University of Manchester are also described. Results for different experimental conditions are presented through graphs showing both the plant and the identified model outputs for the same sequence of sampled input signals.     

A comparative analysis of various least-squares identification algorithms

The purpose of this paper is to clarify the relations and to provide some selection guides among several time-series identification algorithms that appear in the literature under different names but which are essentially least-squares identification algorithms where only the numerical solution of the least-squares estimation problem is different. Such algorithms are, apart from the batch and the sequential forms of direct least-squares, the PARCOR (partial correlation) algorithm, (which may be in the Durbin, the Levinson or the autocorrelation form), the lattice or the ladder algorithm, also known as the Markel-Gray algorithm, the square-root algorithm, the equation-error algorithm, and related algorithms.Further to the above, we shall discuss why certain such algorithms differ in performance from the direct least-squares forms, in terms of convergence, convergence-rate, computational effort (speed) per iteration, and in terms of robustness to computational errors, such as arise when using short word-length computers.     

Differential constraints for bounded recursive identification with multivariate splines

The ability to perform online model identification for nonlinear systems with unknown dynamics is essential to any adaptive model-based control system. In this paper, a new differential equality constrained recursive least squares estimator for multivariate simplex splines is presented that is able to perform online model identification and bounded model extrapolation in the framework of a model-based control system. A new type of linear constraints, the differential constraints, are used as differential boundary conditions within the recursive estimator which limit polynomial divergence when extrapolating data. The differential constraints are derived with a new, one-step matrix form of the de Casteljau algorithm, which reduces their formulation into a single matrix multiplication. The recursive estimator is demonstrated on a bivariate dataset, where it is shown to provide a speedup of two orders of magnitude over an ordinary least squares batch method. Additionally, it is demonstrated that inclusion of differential constraints in the least squares optimization scheme can prevent polynomial divergence close to edges of the model domain where local data coverage may be insufficient, a situation often encountered with global recursive data approximation.     

A bilinear formulation for the output feedback problem in linearsystems

This note ties together a geometric theory and previous algebraic results for the closed-loop eigenstructure assignment by output feedback in multivariable systems. Necessary and sufficient conditions for the closed-loop stabilization, eigenstructure assignment are presented in terms of a bilinear Sylvester equation, the solutions of which completely characterize the closed-loop eigenstructure problem with output feedback. Some preliminary results for the solution of this bilinear equation are presented     

Optimal asymptotic identification under bounded disturbances

The intrinsic limitation of worst-case identification of linear time-invariant systems using data corrupted by bounded disturbances, when the unknown plant is known to belong to a given model set, is studied. This is done by analyzing the optimal worst-case asymptotic error achievable by performing experiments using any bounded input and estimating the plant using any identification algorithm. It is shown that under some topological conditions on the model set, there is an identification algorithm which is asymptotically optimal for any input, and the optimal asymptotic error is characterized as a function of the inputs. These results, which hold for any error metric and disturbance norm, are applied to three specific identification problems: identification of stable systems in the l₁ norm, identification of stable rational systems in the H_∞ norm and identification of unstable rational systems in the gap metric. For each of these problems, the general characterization of optimal asymptotic error is used to find near-optimal inputs to minimize the error     

Near-optimal output feedback regulation of ill-conditioned linearsystems

A two-time scale approximation for the linear quadratic optimal output feedback regulator program is examined. Necessary conditions for optimality, as well as an algorithm for computing locally near-optimal gains are derived. If it is assumed that the slow and fast subsystem initial conditions are uniformly distributed, optimal gains for the two-time-scale problem provide a second-order approximation to optimal closed-loop performance in the unperturbed system. This is verified with a numerical example     

A new autocovariance least-squares method for estimating noise covariances

Industrial implementation of model-based control methods, such as model predictive control, is often complicated by the lack of knowledge about the disturbances entering the system. In this paper, we present a new method (constrained ALS) to estimate the variances of the disturbances entering the process using routine operating data. A variety of methods have been proposed to solve this problem. Of note, we compare ALS to the classic approach presented in Mehra [(1970). On the identification of variances and adaptive Kalman filtering. IEEE Transactions on Automatic Control, 15(12), 175-184]. This classic method, and those based on it, use a three-step procedure to compute the covariances. The method presented in this paper is a one-step procedure, which yields covariance estimates with lower variance on all examples tested. The formulation used in this paper provides necessary and sufficient conditions for uniqueness of the estimated covariances, previously not available in the literature. We show that the estimated covariances are unbiased and converge to the true values with increasing sample size. The proposed method also guarantees positive semidefinite covariance estimates by adding constraints to the ALS problem. The resulting convex program can be solved efficiently.     

A discontinuous least-squares finite-element method for second-order elliptic equations

In this paper, a discontinuous least-squares (DLS) finite-element method is introduced. The novelty of this work is twofold, to develop a DLS formulation that works for general polytopal meshes and to provide rigorous error analysis for it. This new method provides accurate approximations for both the primal and the flux variables. We obtain optimal-order error estimates for both the primal and the flux variables. Numerical examples are tested for polynomials up to degree 4 on non-triangular meshes, i.e. on rectangular and hexagonal meshes.     

A generalized autocovariance least-squares method for Kalman filter tuning

This paper discusses a method for estimating noise covariances from process data. In linear stochastic state-space representations the true noise covariances are generally unknown in practical applications. Using estimated covariances a Kalman filter can be tuned in order to increase the accuracy of the state estimates. There is a linear relationship between covariances and autocovariance. Therefore, the covariance estimation problem can be stated as a least-squares problem, which can be solved as a symmetric semidefinite least-squares problem. This problem is convex and can be solved efficiently by interior-point methods. A numerical algorithm for solving the symmetric is able to handle systems with mutually correlated process noise and measurement noise.     

Robustness and convergence of least-squares identification and adaptive tracking for stochastic systems

The adaptive conlrol for tracking a stochastic reference signal based upon the extended least-squares algorithm estimating unknown parameters of the modelled part in a stochastic syslem is recursively defined. It is shown that the closed-loop system is stable: the estimation error decreases as the unmodelled dynamics decays and the tracking error differs from the minimum plus a small value when the unmodelled dynamics is bounded in the average sense; the strong consistency of the estimates and asymptotical optimality of adaptive tracking are obtained when the unmodelled dynamics approaches zero in the average sense.     

Min-max feedback model predictive control for constrained linearsystems

Min-max feedback formulations of model predictive control are discussed, both in the fixed and variable horizon contexts. The control schemes the authors discuss introduce, in the control optimization, the notion that feedback is present in the receding-horizon implementation of the control. This leads to improved performance, compared to standard model predictive control, and resolves the feasibility difficulties that arise with the min-max techniques that are documented in the literature. The stabilizing properties of the methods are discussed as well as some practical implementation details