A new class of wavelet networks for nonlinear system identification

A new class of wavelet networks (WNs) is proposed for nonlinear system identification. In the new networks, the model structure for a high-dimensional system is chosen to be a superimposition of a number of functions with fewer variables. By expanding each function using truncated wavelet decompositions, the multivariate nonlinear networks can be converted into linear-in-the-parameter regressions, which can be solved using least-squares type methods. An efficient model term selection approach based upon a forward orthogonal least squares (OLS) algorithm and the error reduction ratio (ERR) is applied to solve the linear-in-the-parameters problem in the present study. The main advantage of the new WN is that it exploits the attractive features of multiscale wavelet decompositions and the capability of traditional neural networks. By adopting the analysis of variance (ANOVA) expansion, WNs can now handle nonlinear identification problems in high dimensions.     

Wavelet based non-parametric NARX models for nonlinear input–output system identification

Wavelet based non-parametric additive NARX models are proposed for nonlinear input–output system identification. By expanding each functional component of the non-parametric NARX model into wavelet multiresolution expansions, the non-parametric estimation problem becomes a linear-in-the-parameters problem, and least-squares-based methods such as the orthogonal forward regression (OFR) approach, coupled with model size determination criteria, can be used to select the model terms and estimate the parameters. Wavelet based additive models, combined with model order determination and variable selection approaches, are capable of handling problems of high dimensionality.     

Discrete time subharmonic modelling and analysis

Traditionally the Volterra time and frequency domain analysis tools cannot be applied to severely non-linear systems. In this paper, a new method of building a time-domain NARX MISO model for a class of severely SISO non-linear systems that exhibit subharmonics is introduced and it is shown how this allows the Volterra time and frequency domain analysis to be extended to this class of non-linear systems. The new approach is based on decomposing the original single input based on a Fourier analysis to provide a set of modified input signals which have the same period as the output signal. A MISO NARX model can then be constructed from the decomposed multiple inputs and the single output signal. The resulting MISO model is shown to meet the basic requirement for the existence of a Volterra series representation from which important frequency domain properties can be derived, explained and discussed. This is done by first introducing the derivation of generalized frequency response functions (GFRFs) from time domain MISO NARX models. The steady state response synthesis problem using the input spectrum and the MISO GFRFs is then investigated in order to verify the effectiveness and accuracy of the MISO modelling approach for severely non-linear systems. Finally a new frequency domain analysis method is introduced for systems that exhibit subharmonic oscillations.     

Simple recursive algorithm for linear-in-theparameters nonlinear model identification

This paper introduces a simple recursive algorithm for nonlinear dynamic system identification using linear-in-the-parameters models for NARX or RBF network where both the structure and parameters can be obtained simultaneously and recursively. The main objective is to improve the numerical stability when the model terms are highly correlated. This is based on the "innovation" idea and net contribution criteria. Using the recursive formulae for the computation of the Moore-Penrose inverse of matrices and th...     

一种基于小波分解的非线性系统辨识的新方法

提出了一种结合小波理论和NARX模型的新辨识算法．该算法利用小波(多维小波)函数有效的逼近能力避免了通常确定NARX模型结构时的复杂过程，构成了一个相当通用且不依赖于系统先验信息的辨识框架．应用递推最小二乘算法估计模型参数时，该算法可实现系统的在线辨识．两仿真算例说明了这种算法的有效性．     

Identification and reconstruction of chaotic systems using multiresolution wavelet decompositions

A new modelling framework for identifying and reconstructing chaotic systems is developed based on multiresolution wavelet decompositions. Qualitative model validation is used to compare the multiresolution wavelet models and it is shown that the dynamical features of chaotic systems can be captured by the identified models providing the wavelet basis functions are properly selected. Two basis selection algorithms, orthogonal least squares and a new matching pursuit orthogonal least squares, are considered and compared. Several examples are included to illustrate the results.     

Term and variable selection for non-linear system identification

The purpose of variable selection is to pre-select a subset consisting of the significant variables or to eliminate the redundant variables from all the candidate variables of a system under study prior to model term detection. It is required that the selected significant variables alone should sufficiently represent the system. Generally, not all the model terms, which are produced by combining different variables, make an equal contribution to the system output and terms, which make little contribution, can be omitted. A parsimonious representation, which contains only the significant terms, can often be obtained without the loss of representational accuracy by eliminating the redundant terms. Based on these observations, a new variable and term selection algorithm is proposed in this paper. The term detection algorithm can be applied to the general class of non-linear modelling problems which can be expressed as a linear-in-the-parameters form. The variable selection procedure is based on locally linear and cross-bilinear models, which are used together with the forward orthogonal least squares (OLS) and error reduction ratio (ERR) approach to determine the significant terms and to pre-select the important variables for both time series and input–output systems. Several numerical examples are provided to illustrate the applicability and effectiveness of the new approach.     

Nonlinear system identification using wavelet networks

A novel identification scheme using wavelet networks is presented for nonlinear dynamical systems. Based on fixed wavelet networks, parameter adaptation laws are developed using a Lyapunov synthesis approach. This guarantees the stability of the overall identification scheme and the convergence of both the parameters and the state errors, even in the presence of modelling errors. Using the decomposition and reconstruction techniques of multiresolution decompositions, variable wavelet networks are introduced to achieve a desired estimation accuracy and a suitable sized network, and to adapt to variations of the characteristics and operating points in nonlinear systems. B-spline wavelets are used to form the wavelet networks and the identification scheme is illustrated using a simulated example.     

Detrended fluctuation thresholding for empirical mode decomposition based denoising

Signal decompositions such as wavelet and Gabor transforms have successfully been applied in denoising problems. Empirical mode decomposition (EMD) is a recently proposed method to analyze non-linear and non-stationary time series and may be used for noise elimination. Similar to other decomposition based denoising approaches, EMD based denoising requires a reliable threshold to determine which oscillations called intrinsic mode functions (IMFs) are noise components or noise free signal components. Here, we propose a metric based on detrended fluctuation analysis (DFA) to define a robust threshold. The scaling exponent of DFA is an indicator of statistical self-affinity. In our study, it is used to determine a threshold region to eliminate the noisy IMFs. The proposed DFA threshold and denoising by DFA–EMD are tested on different synthetic and real signals at various signal to noise ratios (SNR). The results are promising especially at 0 dB when signal is corrupted by white Gaussian noise (WGN). The proposed method outperforms soft and hard wavelet threshold method.     

Neural model input selection for a MIMO chemical process

In modelling non-linear systems using neural networks (NN), a commonly used method for the selection of network inputs, or to determine system order and time-delay, is to try different combinations of the system input–output data and choose the best one, giving minimum prediction error. The method is increasingly difficult to apply to industrial systems, due to their multivariable nature and complexity. A systematic method for the selection of model order and time-delay is developed in this paper, and applied to the neural modelling of a multivariable chemical process rig. The method is much simpler compared to the structure identification of the Non-linear Auto-Regressive with eXogenous inputs model (NARX), since the latter also needs to determine the significant terms from a linear-in-parameters polynomial. The orders and delays for system input and output are determined by identifying linearised models of the system. The method can also be applied to other approximations of a MIMO non-linear system, such as fuzzy logic models, etc. The application example demonstrates the selection procedure. Finally, the process rig is modelled using NNs according to the chosen structure, and the modelling error is compared with that of models with different structures to show the effectiveness of the method.     

New bound characteristics of NARX model in the frequency domain

New results about the bound characteristics of both the generalized frequency response functions (GFRFs) and the output frequency response for the NARX (Non-linear AutoRegressive model with eXogenous input) model are established. It is shown that the magnitudes of the GFRFs and the system output spectrum can all be bounded by a polynomial function of the magnitude bound of the first order GFRF, and the coefficients of the polynomial are functions of the NARX model parameters. These new bound characteristics of the NARX model provide an important insight into the relationship between the model parameters and the magnitudes of the system frequency response functions, reveal the effect of the model parameters on the stability of the NARX model to a certain extent, and provide a useful technique for the magnitude based analysis of nonlinear systems in the frequency domain, for example, evaluation of the truncation error in a volterra series expression of non-linear systems and the highest order needed in the volterra series approximation. A numerical example is given to demonstrate the effectiveness of the theoretical results.     

Identification of non-linear parametrically varying models using separable least squares

In this paper a novel identification algorithm for a class of non-linear, possibly parameter varying models is proposed. The algorithm is based on separable least squares ideas. These models are given in the form of a linear fractional transformation (LFT) where the ‘forward’ part is represented by a conventional linear regression and the ‘feedback’ part is given by a non-linear map which can take into account scheduling variables available for measurement. The non-linear part of the model can be parameterized according to various paradigms, like, e.g. neural network (NN) or general nonlinear autoregressive exogenous (NARX) models. The estimation algorithm exploits the separability of the criterion used to estimate the parameters. When using a NN, it is possible the explicit computation of the Frechet derivative needed to implement a separable least square algorithm.     

NARX model based nonlinear dynamic system identification using low complexity neural networks and robust H∞ filter

This paper proposes NARX (nonlinear autoregressive model with exogenous input) model structures with functional expansion of input patterns by using low complexity ANN (artificial neural network) for nonlinear system identification. Chebyshev polynomials, Legendre polynomials, trigonometric expansions using sine and cosine functions as well as wavelet basis functions are used for the functional expansion of input patterns. The past input and output samples are modeled as a nonlinear NARX process and robust H_∞ filter is proposed as the learning algorithm for the neural network to identify the unknown plants. H_∞ filtering approach is based on the state space modeling of model parameters and evaluation of Jacobian matrices. This approach is the robustification of Kalman filter which exhibits robust characteristics and fast convergence properties. Comparison results for different nonlinear dynamic plants with forgetting factor recursive least square (FFRLS) and extended Kalman filter (EKF) algorithms demonstrate the effectiveness of the proposed approach.     

High-resolution time–frequency representation of EEG data using multi-scale wavelets

An efficient time-varying autoregressive (TVAR) modelling scheme that expands the time-varying parameters onto the multi-scale wavelet basis functions is presented for modelling nonstationary signals and with applications to time–frequency analysis (TFA) of electroencephalogram (EEG) signals. In the new parametric modelling framework, the time-dependent parameters of the TVAR model are locally represented by using a novel multi-scale wavelet decomposition scheme, which can allow the capability to capture the smooth trends as well as track the abrupt changes of time-varying parameters simultaneously. A forward orthogonal least square (FOLS) algorithm aided by mutual information criteria are then applied for sparse model term selection and parameter estimation. Two simulation examples illustrate that the performance of the proposed multi-scale wavelet basis functions outperforms the only single-scale wavelet basis functions or Kalman filter algorithm for many nonstationary processes. Furthermore, an application of the proposed method to a real EEG signal demonstrates the new approach can provide highly time-dependent spectral resolution capability.     

多自由度非线性系统动态参数化模型建模方法研究

针对多自由度非线性系统的动态模型辨识问题,基于NARX(Non-linear Autoregressive with Exogenous inputs)模型的建模方法,考虑系统的物理设计参数,建立非线性系统动态参数化模型.首先,根据系统输入、输出数据建立系统不同参数下的NARX模型,并通过EFOR(Extended Forward Orthogonal Regression)算法对不同参数下NARX模型进行修正,以统一辨识得到的系统模型结构.随后,建立NARX模型系数与物理设计参数间的函数关系,得到多自由度非线性系统的动态参数化模型.以单输入、单输出两自由度非线性系统为例,根据数值仿真结果,对系统的动态参数化模型建模过程进行说明.最后,以带非线性涂层阻尼的悬臂梁作为试验对象,建立其动态参数化模型以反映其动力学特性.试验结果表明,非线性系统动态参数化模型能准确预测多自由度非线性系统的输出响应,为非线性系统的分析与优化设计提供了理论基础.     

Set-membership estimation from poor quality data sets: Modelling ammonia volatilisation in flooded rice systems

A set-membership (bounded-error) estimation approach can handle small and poor quality data sets as it does not require testing of statistical assumptions which is possible only with large informative data sets. Thus, set-membership estimation can be a good tool in the modelling of agri-environmental systems, which typically suffers from limited and poor quality observational data sets. The objectives of the paper are (i) to demonstrate how six parameters in an agri-environmental model, developed to estimate NH₃ volatilisation in flooded rice systems, were estimated based on two data sets using a set-membership approach, and (ii) to compare the set-membership approach with conventional non-linear least-squares methods. Results showed that the set-membership approach is efficient in retrieving feasible parameter-vectors compared with non-linear least-squares methods. The set of feasible parameter-vectors allows the formation of a dispersion matrix of which the eigenvalue decomposition reflects the parameter sensitivity in a region.     

Regressor and structure selection in NARX models using a structured ANOVA approach

Regressor selection can be viewed as the first step in the system identification process. The benefits of finding good regressors before estimating complex models are especially clear for nonlinear systems, where the class of possible models is huge. In this article, a structured way of using the tool analysis of variance (ANOVA) is presented and used for NARX model (nonlinear autoregressive model with exogenous input) identification with many candidate regressors.     

Minimality of an Automaton Cascade Decomposition for Learning System Environments

We already know how to decompose any finite automaton with a strongly connected state diagram into a strongly connected version of what we call a synchronizable cascade decomposition. This is a two component cascade decomposition whose first component has a synchronizer and whose second component is a permutation automaton. Here, we give a simpler procedure for constructing such a decomposition and show that the constructed decomposition is a homomorphic image of a subautomaton of any other synchronizable cascade decomposition. The constructed decomposition is then of minimal size, and all minimal synchronizable cascade decompositions are isomorphic, including all decompositions constructed by the old procedure. This means that their first components are isomorphic, but their second components need not be. In analyzing learning systems, we can use a synchronizable cascade decomposition to model a finite automaton environment, and in these analytical applications, the second component has equiprobable states and can often be ignored in analysis. There are many ways of constructing synchronizable cascade decompositions and we will want to use the construction that is easiest for the analytical application. The isomorphism result says that two different construction methods produce isomorphic decompositions provided the decompositions produced have the minimal number of states, and it is often easy to show this by a simple counting argument. This paper confines itself to giving the simpler procedure and proving the homomorphism result. It does not discuss analytical applications.     

Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks

This study deals with the neuro-fuzzy (NF) modelling of a real industrial winding process in which the acquired NF model can be exploited to improve control performance and achieve a robust fault-tolerant system. A new simulator model is proposed for a winding process using non-linear identification based on a recurrent local linear neuro-fuzzy (RLLNF) network trained by local linear model tree (LOLIMOT), which is an incremental tree-based learning algorithm. The proposed NF models are compared with other known intelligent identifiers, namely multilayer perceptron (MLP) and radial basis function (RBF). Comparison of our proposed non-linear models and associated models obtained through the least square error (LSE) technique (the optimal modelling method for linear systems) confirms that the winding process is a non-linear system. Experimental results show the effectiveness of our proposed NF modelling approach.