Obtaining initial parameter estimates for nonlinear systems using multicriteria associative memories

Parameter estimation problems for nonlinear systems are typically formulated as nonlinear optimization problems. For such problems, one has the usual difficulty that standard successive approximation schemes require good initial estimates for the parameter vector. This paper develops a simple multicriteria associative memory (MAM) procedure for obtaining useful initial parameter estimates for nonlinear systems. An easily calculated one-parameter family of associative memory matrices is developed; see Equation (25). Each memory matrix is efficient with respect to two criteria: accurate recovery of parameter-output training case associations; and small matrix norm to guard against noise arising from imprecise calculations and observations. For illustration, the MAM procedure is used to obtain initial parameter estimates for a well-known nonlinear economic model, the Solow-Swan growth model. Surprisingly accurate initial parameter estimates are obtained over broad ranges of the family of MAM memory matrices, even when observations are corrupted by i.i.d. or correlated noise.     

Stochastic stability of the discrete-time extended Kalman filter

The authors analyze the error behavior for the discrete-time extended Kalman filter for general nonlinear systems in a stochastic framework. In particular, it is shown that the estimation error remains bounded if the system satisfies the nonlinear observability rank condition and the initial estimation error as well as the disturbing noise terms are small enough. This result is verified by numerical simulations for an example system     

Identification of block-oriented nonlinear systems using orthonormal bases

In this paper, new noniterative algorithms for the identification of (multivariable) block-oriented nonlinear models consisting of the interconnection of linear time invariant systems and static nonlinearities are presented. The proposed algorithms are numerically robust, since they are based only on least squares estimation and singular value decomposition. Two different block-oriented nonlinear models are considered in this paper, viz., the Hammerstein model, and the Wiener model. For the Hammerstein model, the proposed algorithm provides consistent estimates even in the presence of colored output noise, under weak assumptions on the persistency of excitation of the inputs. For the Wiener model, consistency of the estimates can only be guaranteed in the noise free case. Key in the derivation of the results is the use of basis functions for the representation of the linear and nonlinear parts of the models. The performance of the proposed identification algorithms is illustrated through simulation examples of two benchmark problems drawn from the process control literature, viz., a binary distillation column and a pH neutralization process.     

Robust numerical approach to steady-state calibration of mean-value models

A numerically robust approach to steady-state calibration of nonlinear dynamic models is presented. The approach is based on explicit formulation of the constraints on validity of internal model signals by set of inequalities. The constrained optimization with feasible iterates guarantees that the model will never be evaluated with invalid internal signals. This overcomes numerical difficulties often encountered when dealing with highly nonlinear models. Because the approach uses a large number of slack variables, distributed least squares algorithm is proposed. The robustness of this approach is demonstrated on a steady-state calibration of turbocharged diesel engine model starting from grossly inaccurate initial estimates.     

Finite model order accuracy in Hammerstein model estimation

Hammerstein models is one of the most commonly used model classes used for identifying nonlinear systems. A static input nonlinearity followed by a linear dynamical part is an adequate way to model many real-life systems. This paper investigates the asymptotic (in terms of sample size) variance of Hammerstein model estimates. The work extends earlier results by Ninness and Gibson (2002) in the following ways. Not only frequency function estimation but estimation of general quantities is considered. The expressions are not restricted to be valid asymptotically in the model order. In addition, the results cover model structures having noise models and allow for data generated under feedback. The increase in variance due to the estimation of the input nonlinearity is characterized. In particular, under open loop operation, white additive noise and the assumption of a separable process, it is shown that the variance increase is exactly a term that was observed in Ninness and Gibson (2002) to result in good agreement with simulations. This term vanishes in the formal asymptotic in model order analysis in Ninness and Gibson (2002).     

Nonlinear filtering structure for image smoothing in mixed-noise environments

This paper introduces a new nonlinear filtering structure for filtering image data that have been corrupted by both impulsive and nonimpulsive additive noise. Like other nonlinear filters, the proposed filtering structure uses order-statistic operations to remove the effects of the impulsive noise. Unlike other filters, however, nonimpulsive noise is smoothed by using a maximum a posteriori estimation criterion. The prior model for the image is a novel Markov random-field model that models image edges so that they are accurately estimated while additive Gaussian noise is smoothed. The Markov random-field-based prior is chosen such that the filter has desirable analytical and computational properties. The estimate of the signal value is obtained at the unique minimum of the a posteriori log likelihood function. This function is convex so that the output of the filter can be easily computed by using either digital or analog computational methods. The effects of the various parameters of the model will be discussed, and the choice of the predetection order statistic filter will also be examined. Example outputs under various noise conditions will be given.     

A new real-time suboptimum filtering and prediction scheme for general nonlinear discrete dynamic systems with Gaussian or non-Gaussian noise

A new suboptimum state filtering and prediction scheme is proposed for nonlinear discrete dynamic systems with Gaussian or non-Gaussian disturbance and observation noises. This scheme is an online estimation scheme for real-time applications. Furthermore, this scheme is very suitable for state estimation under either constraints imposed on estimates or missing observations. State and observation models can be any nonlinear functions of the states, disturbance and observation noises as long as noise samples are independent, and the density functions of noise samples and conditional density functions of the observations given the states are available. State models are used to calculate transition probabilities from gates to gates. If these transition probabilities are known or can be estimated, state models are not needed for estimation. The proposed scheme (PR) is based upon state quantisation and multiple hypothesis testing. Monte Carlo simulations have shown that the performance of the PR, sampling importance resampling (SIR) particle filter and extended Kalman (EK) filter are all model-dependent, and that the performance of the PR is better than both the SIR particle filter and EK filter for some nonlinear models, simulation results of three of which are given in this article.     

Identification of Hammerstein nonlinear ARMAX systems

Two identification algorithms, an iterative least-squares and a recursive least-squares, are developed for Hammerstein nonlinear systems with memoryless nonlinear blocks and linear dynamical blocks described by ARMAX/CARMA models. The basic idea is to replace unmeasurable noise terms in the information vectors by their estimates, and to compute the noise estimates based on the obtained parameter estimates. Convergence properties of the recursive algorithm in the stochastic framework show that the parameter estimation error consistently converges to zero under the generalized persistent excitation condition. The simulation results validate the algorithms proposed.     

Generalised observer design for dissipative Lipschitz nonlinear systems in the presence of measurement noise

This paper presents two novel observer concepts. First, it develops a globally exponentially stable nonlinear observer for noise-free dissipative nonlinear systems. Second, for a dissipative nonlinear system with measurement noise, the paper develops an observer to guarantee a desired performance, namely an upper limit on the ratio of the square of the weighted L2 norm of the error to the square of the weighted L2 norm of the measurement noise. The necessary and sufficient conditions for both observers are reformulated as algebraic Riccati equations (AREs) so that standard solvers can be utilised. In addition, the paper presents necessary and sufficient conditions to be satisfied by the nonlinear system in order to ensure that the ARE (and hence the observer design problem) has a solution. The use of the methodology developed in this paper is demonstrated through illustrative examples. In literature, there is no previous observer for dissipative system that provides both necessary and sufficient conditions. Results for noisy system either rely on linearising the system about state trajectory (requiring initial estimates to be close to the actual states) or are for specialised systems that cannot be extended to dissipative systems.     

On optimal experiment design for identifying premise and conclusion parameters of Takagi-Sugeno models: Nonlinear regression case

Optimal Experiment Design (OED) is a well-developed concept for regression problems that are linear-in-the-parameters. In case of experiment design to identify nonlinear Takagi-Sugeno (TS) models, non-model-based approaches or OED restricted to the local model parameters (assuming the partitioning to be given) have been proposed. In this article, a Fisher Information Matrix (FIM) based OED method is proposed that considers local model and partition parameters. Due to the nonlinear model, the FIM depends on the model parameters that are subject of the subsequent identification. To resolve this paradoxical situation, at first a model-free space filling design (such as Latin Hypercube Sampling) is carried out. The collected data permits making design decisions such as determining the number of local models and identifying the parameters of an initial TS model. This initial TS model permits a FIM-based OED, such that data is collected which is optimal for a TS model. The estimates of this first stage will in general not be ideal. To become robust against parameter mismatch, a sequential optimal design is applied. In this work the focus is on D-optimal designs. The proposed method is demonstrated for three nonlinear regression problems: an industrial axial compressor and two test functions.     

On the Consistency of Instantaneous Rigid Motion Estimation

Instantaneous camera motion estimation is an important research topic in computer vision. Although in theory more than five points uniquely determine the solution in an ideal situation, in practice one can usually obtain better estimates by using more image velocity measurements because of the noise present in the velocity measurements. However, the usefulness of using a large number of observations has never been analyzed in detail. In this paper, we formulate this problem in the statistical estimation framework. We show that under certain noise models, consistency of motion estimation can be established: that is, arbitrarily accurate estimates of motion parameters are possible with more and more observations. This claim does not simply follow from the general consistency result for maximum likelihood estimates. Some experiments will be provided to verify our theory. Our analysis and experiments also indicate that many previously proposed algorithms are inconsistent under even very simple noise models.     

A fractional approach to identify Wiener–Hammerstein systems

Block-oriented nonlinear models are appealing due to their simplicity and parsimony. Existing methods to identify the Wiener–Hammerstein model suffer from one or several drawbacks. This paper shows that it is possible to generate initial estimates in an alternative way. A fractional model parameterization is the key to the success of this approach. Advantages are that no more than two iterative optimizations are needed and that large model orders can be handled. As illustrated through a simulation example and experimental benchmark data, it gives superior initial estimates and comparable optimized results.     

Adaptive,high precision,satellite attitude control for microprocessor implementation

A control system is presented for three-axis, gas jet, satellite attitude control having application to any spacecraft where precise pointing is required within stringent mass and power limitations. Several novel features are incorporated as follows: parabolic switching boundaries are employed with parameters which adapt to a disturbing acceleration estimate in order to achieve a zero offset steady-state limit cycle of preset amplitude in the arcsecond region which minimizes both fuel consumption and thruster operation frequency. The disturbing acceleration estimate is obtained from a third-order state estimator, together with angle error and rate estimates using an angle error measurement from a rate integrating gyro and a jet drive input. Time optimal recovery from large initial angle errors and rapid response to step changes in disturbing acceleration are achieved. In addition, stable control is obtained with disturbing acceleration approaching the control jet acceleration. A slew control algorithm is incorporated which enables the same control law to be utilized for fuel optimal slewing through unlimited angles, one axis at a time. Simulation results are presented, including demonstration of stochastic performance with gyro and jet noise.     

Hammerstein-Wiener system estimator initialization

In nonlinear system identification, the system is often represented as a series of blocks linked together. Such block-oriented models are built with static nonlinear subsystems and linear dynamic systems. This paper deals with the identification of the Hammerstein-Wiener model, which is a block-oriented model where a linear dynamic system is surrounded by two static nonlinearities at its input and output. The proposed identification scheme is iterative and will be demonstrated on measurements. It will be proven that on noiseless data and in absence of modeling errors, the optimization procedure converges to the true system locally.     

基于遗传规划的哈默斯坦模型的辨识

该文基于遗传规划提出了一种辨识哈默斯坦模型的新方法。哈默斯坦模型由静态非线性模块和动态线性模块串联而成,因此系统辨识的目标是要找到非线性和线性模块的最优数学模型。该文通过遗传规划确定非线性模块的函数结构,并结合遗传算法确定模型的未知参数,适应度值的计算采用了最小信息量准则(A IC),以平衡模型的复杂度和精确度。该方法不需要对模型的先验知识有详细了解,就能达到较好的辨识效果,并且能够克服观测噪声的污染,获得参数的无偏估计。仿真结果说明了该方法的有效性。     

Static and dynamic preprocessing methods in neural networks

Preprocessing is recognized as an important tool in modeling, particularly when the data or underlying physical process involves complex nonlinear dynamical interactions. This paper will give a review of preprocessing methods used in linear and nonlinear models. The problem of static preprocessing will be considered first, where no dependence on time between the input vectors is assumed. Then, dynamic preprocessing methods which involve the modification of time-dependent input values before they are used in the linear or nonlinear models will be considered. Furthermore, the problem of an insufficient number of input vectors is considered. It is shown that one way in which this problem can be overcome is by expanding the weight vector in terms of the available input vectors. Finally, a new problem which involves both cases of: (1) transformation of input vectors; and (2) insufficient number of input vectors is considered. It is shown how a combination of the techniques used to solve the individual problems can be combined to solve this composite problem. Some open issues in this type of preprocessing methods are discussed.     

声呐位置误差情况下的运动目标多基地定位方法

在多基地声呐定位系统中,声呐的位置信息往往含有随机误差,这些误差会严重影响目标的定位精度。针对这一问题,提出了一种基于时间和多普勒频率的运动目标定位方法。首先,将基于时间和多普勒频率定位机制的非线性量测方程组转化为关于目标位置、速度及中间变量的伪线性方程组,利用加权最小二乘估计法对运动目标的位置、速度进行初始求解;然后,利用位置、速度及中间变量之间的相关性对位置和速度的估计偏差进行求解;最后,对位置和速度的初始解进行误差修正。分析了所提算法在量测误差较小情况下的统计有效性,并通过蒙特卡洛模拟进行了数值验证。     

Estimation of nonlinear errors-in-variables models for computer vision applications

In an errors-in-variables (EIV) model, all the measurements are corrupted by noise. The class of EIV models with constraints separable into the product of two nonlinear functions, one solely in the variables and one solely in the parameters, is general enough to represent most computer vision problems. We show that the estimation of such nonlinear EIV models can be reduced to iteratively estimating a linear model having point dependent, i.e., heteroscedastic, noise process. Particular cases of the proposed heteroscedastic errors-in-variables (HEIV) estimator are related to other techniques described in the vision literature: the Sampson method, renormalization, and the fundamental numerical scheme. In a wide variety of tasks, the HEIV estimator exhibits the same, or superior, performance as these techniques and has a weaker dependence on the quality of the initial solution than the Levenberg-Marquardt method, the standard approach toward estimating nonlinear models.     

A new measurement microphone based on MEMS technology

This paper presents a new type of measurement microphone that is based on MEMS technology. The silicon chip design and fabrication are discussed, as well as the specially developed packaging technology. The microphones are tested on a number of key parameters for measurement microphones: sensitivity, noise level, frequency response, and immunity to disturbing environmental parameters, such as temperature changes, humidity, static pressure variations, and vibration. A sensitivity of 22 mV/Pa (-33 dB re. 1 V/Pa), and a noise level of 23 dB(A) were measured. The noise level is 7 dB lower than state-of-the-art 1/4-inch measurement microphones. A good uniformity on sensitivity and frequency response has been measured. The sensitivity to temperature changes, humidity, static pressure variations and vibrations is fully comparable to the traditional measurement microphones. This paper shows that high-quality measurement microphones can be made using MEMS technology, with a superior noise performance.     

Recursive Identification for Hammerstein System With ARX Subsystem

Identification is considered for the Hammerstein system consisting of a static nonlinear block f(middot) followed by an ARX subsystem, when the system output is observed with noise. No assumption is made on the structure of f(middot). Recursive estimates are given for coefficients of the ARX subsystem and for the value of f(u) at any u. All estimates are proved to converge to the true values with probability one. Numerical examples are provided justifying the theoretical analysis