Outliers, inliers and the generalized least trinuned squares estimator in system identification

The least trimmed squares estimator (LTS) is a well known robust estimator in terms of protecting the estimate from the outliers. Its high computational complexity is however a problem in practice. We show that the LTS estimate can be obtained by a simple algorithm with the complexity O( N In N) for large N, where N is the number of measurements. We also show that though the LTS is robust in terms of the outliers, it is sensitive to the inliers. The concept of the inliers is introduced. Moreover, the Generalized Least Trimmed Squares estimator (GLTS) together with its solution are presented that reduces the effect of both the outliers and the inliers.     

Outliers, inliers and the generalized least trimmed squares estimator in system identification

The least trimmed squares estimator (LTS) is a well known robust estinaator in terms of protecting the estimatefrom the outliers. Its high computational complexity is however a problem in practice. We show that the LTS estimate can be obtained by a simple algorithm with the complexity O( N In N) for large N, where N is the number of measurements. We also showthat though the LTS is robust in terms of the outliers, it is sensitive to the inliers. The concept of the inliers is introduced. Moreover, the Generalized Least Trimmed Squares estimator (GLTS) together with its solution are presented that reduces the effect of both the outliers and the inliers.     

A random least-trimmed-squares identification algorithm

The Least-trimmed-squares (LTS) estimator is a well known robust estimator in terms of protecting the estimate from the outliers. Its high computational complexity is however a problem in practice. In this paper, we propose a random LTS algorithm which has a low computational complexity that can be calculated a priori as a function of the required error bound and the confidence interval. Moreover, if the number of data points goes to infinite, the algorithm becomes a deterministic one that converges to the true LTS in some probability sense.     

Combined parameter and output estimation of dual-rate systems using an auxiliary model

For a dual-rate sampled-data system, an auxiliary model based identification algorithm for combined parameter and output estimation is proposed. The basic idea is to use an auxiliary model to estimate the unknown noise-free output (true output) of the system, and directly to identify the parameters of the underlying fast single-rate model from the dual-rate input-output data. It is shown that the parameter estimation error consistently converges to zero under generalized or weak persistent excitation conditions and unbounded noise variance, and that the output estimates uniformly converge to the true outputs. An example is included.     

An efficient hierarchical identification method for general dual-rate sampled-data systems

For the lifted input–output representation of general dual-rate sampled-data systems, this paper presents a decomposition based recursive least squares (D-LS) identification algorithm using the hierarchical identification principle. Compared with the recursive least squares (RLS) algorithm, the proposed D-LS algorithm does not require computing the covariance matrices with large sizes and matrix inverses in each recursion step, and thus has a higher computational efficiency than the RLS algorithm. The performance analysis of the D-LS algorithm indicates that the parameter estimates can converge to their true values. A simulation example is given to confirm the convergence results.     

Worst-case analysis of the least-squares method and related identification methods

We consider worst-case analysis of system identification under less restrictive assumptions on the noise than the l_∞ bounded error condition. It is shown that the least-squares method has a robust convergence property in l₂ identification, but lacks a corresponding property in l₁ identification (as well as in all other non-Hilbert space settings). The latter result is in stark contrast with typical results in asymptotic stochastic analysis of the least-squares method. Furthermore, it is shown that the Khintchine inequality is useful in the analysis of least l^p identification methods.     

Identification of Hammerstein nonlinear ARMAX systems

Two identification algorithms, an iterative least-squares and a recursive least-squares, are developed for Hammerstein nonlinear systems with memoryless nonlinear blocks and linear dynamical blocks described by ARMAX/CARMA models. The basic idea is to replace unmeasurable noise terms in the information vectors by their estimates, and to compute the noise estimates based on the obtained parameter estimates. Convergence properties of the recursive algorithm in the stochastic framework show that the parameter estimation error consistently converges to zero under the generalized persistent excitation condition. The simulation results validate the algorithms proposed.     

Comments on “Identification of Hammerstein nonlinear ARMAX systems”

This note points out some errors in the proof of the main result of Ding and Chen [2005. Identification of Hammerstein nonlinear ARMAX systems. Automatica 41(9), 1479-1489] and mentions the well-known results that have solved the problem discussed in Theorem 1 of the above paper by Ding and Chen.     

Least squares based iterative identification for a class of multirate systems

This paper studies modeling and identification problems for multi-input multirate systems with colored noises. The state-space models are derived for the systems with different input updating periods and furthermore the corresponding transfer functions are obtained. To solve the difficulty of identification models with unmeasurable noises terms, the least squares based iterative algorithm is presented by replacing the unmeasurable variables with their iterative estimates. Finally, the simulation results indicate that the proposed iterative algorithm has advantages over the recursive algorithms.     

A continuous-time framework for least squares parameter estimation

This paper proposes a continuous-time framework for the least-squares parameter estimation method through evolution equations. Nonlinear systems in the standard state space representation that are linear in the unknown, constant parameters are investigated. Two estimators are studied. The first one consists of a linear evolution equation while the second one consists of an impulsive linear evolution equation. The paper discusses some theoretical aspects related to the proposed estimators: uniqueness of a solution and an attractive equilibrium point which solves for the unknown parameters. A deterministic framework for the estimation under noisy measurements is proposed using a Sobolev space with negative index to model the noise. The noise can be of large magnitude. Concrete signals issued from an electronic device are used to discuss numerical aspects.     

Instability detection of ARMA systems based on AR system identification

The ability of parametric autoregressive (AR) system identification methods to detect the instability of an autoregressive moving average (ARMA) system of an unknown order is investigated. The collection of least squares AR estimators of various orders is shown to have the capacity to detect the instability of the underlying system. Necessary information is not the order of the system but, instead, an upper bound of the number of unstable poles with the maximal magnitude outside the unit circle.     

A genetic algorithm calibration method based on convergence due to genetic drift

The selection of Genetic Algorithm (GA) parameters is a difficult problem, and if not addressed adequately, solutions of good quality are unlikely to be found. A number of approaches have been developed to assist in the calibration of GAs, however there does not exist an accepted method to determine the parameter values. In this paper, a GA calibration methodology is proposed based on the convergence of the population due to genetic drift, to allow suitable GA parameter values to be determined without requiring a trial-and-error approach. The proposed GA calibration method is compared to another GA calibration method, as well as typical parameter values, and is found to regularly lead the GA to better solutions, on a wide range of test functions. The simplicity and general applicability of the proposed approach allows suitable GA parameter values to be estimated for a wide range of situations.     

Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process

In the past decade, support vector machines (SVMs) have gained the attention of many researchers. SVMs are non-parametric supervised learning schemes that rely on statistical learning theory which enables learning machines to generalize well to unseen data. SVMs refer to kernel-based methods that have been introduced as a robust approach to classification and regression problems, lately has handled nonlinear identification problems, the so called support vector regression. In SVMs designs for nonlinear identification, a nonlinear model is represented by an expansion in terms of nonlinear mappings of the model input. The nonlinear mappings define a feature space, which may have infinite dimension. In this context, a relevant identification approach is the least squares support vector machines (LS-SVMs). Compared to the other identification method, LS-SVMs possess prominent advantages: its generalization performance (i.e. error rates on test sets) either matches or is significantly better than that of the competing methods, and more importantly, the performance does not depend on the dimensionality of the input data. Consider a constrained optimization problem of quadratic programing with a regularized cost function, the training process of LS-SVM involves the selection of kernel parameters and the regularization parameter of the objective function. A good choice of these parameters is crucial for the performance of the estimator. In this paper, the LS-SVMs design proposed is the combination of LS-SVM and a new chaotic differential evolution optimization approach based on Ikeda map (CDEK). The CDEK is adopted in tuning of regularization parameter and the radial basis function bandwith. Simulations using LS-SVMs on NARX (Nonlinear AutoRegressive with eXogenous inputs) for the identification of a thermal process show the effectiveness and practicality of the proposed CDEK algorithm when compared with the classical DE approach.     

On the convergence condition and convergence time of BGP

The Internet composes of thousands of Autonomous System (ASes). The Border Gateway Protocol (BGP) is the standard protocol for sharing inter-domain routing information. Unlike OSPF and IS-IS, BGP allows an AS to use a lot of attributes to express semantic rich routing policies that are consistent with its desired economic, business, performance, and security goals. However, the expressiveness could cause to delay convergence or even divergence in BGP. Recent work do not rigorously analyze the impact of the general routing policies on the convergence condition and convergence time of BGP, especially considering the widely used Multi-Exit Discriminator (MED) attribute. In this paper, we will fill this gap and give the rigorous analysis on the impact of the general routing policies on the convergence condition and convergence time of BGP, including MED attribute. We first introduce a timeless model to represent BGP with the general routing policies including the MED attribute. By incorporating the timeless model we derive a sufficient condition on these general routing policies for robust convergence of BGP. We then extend the timeless model to the real-time model by adding the edge delay. Finally, we find an upper bound on convergence time of BGP by incorporating the real-time model.     

A note on the theoretical convergence properties of the SIMP method

The solid isotropic material with penalization (SIMP) method is used in topology optimization to solve problems where the variables are 0 or 1. The theoretical convergence properties have not been exhaustively studied. In this paper a convergence theorem with weaker assumptions than earlier conditions is given.     

Rapid parameter estimation algorithms using the ELS principle

Two computationally efficient algorithms for estimating the parameters of linear discrete-time systems are proposed. The algorithms are based on the extended least squares (ELS) principle. They are essentially a correlation version of the off-line ELS method that eliminate all the redundant computations, do not require construction and operations of large matrices and bypass the explicit evaluation of residuals. Examples are given to illustrate their feasibility and performance.