Nonlinear black-box modeling in system identification: a unified overview

A nonlinear black-box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics. There has been considerable recent interest in this area, with structures based on neural networks, radial basis networks, wavelet networks and hinging hyperplanes, as well as wavelet-transform-based methods and models based on fuzzy sets and fuzzy rules. This paper describes all these approaches in a common framework, from a user's perspective. It focuses on what are the common features in the different approaches, the choices that have to be made and what considerations are relevant for a successful system-identification application of these techniques. It is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space. These mappings are discussed separately. The latter mapping is usually formed as a basis function expansion. The basis functions are typically formed from one simple scalar function, which is modified in terms of scale and location. The expansion from the scalar argument to the regressor space is achieved by a radial- or a ridge-type approach. Basic techniques for estimating the parameters in the structures are criterion minimization, as well as two-step procedures, where first the relevant basis functions are determined, using data, and then a linear least-squares step to determine the coordinates of the function approximation. A particular problem is to deal with the large number of potentially necessary parameters. This is handled by making the number of ‘used’ parameters considerably less than the number of ‘offered’ parameters, by regularization, shrinking, pruning or regressor selection.     

Variance analysis of identified linear MISO models having spatially correlated inputs,with application to parallel Hammerstein models

This contribution concerns variance analysis of linear multi-input single-output models when the inputs are temporally white but where different inputs may be correlated. An expression is provided for the variance of a linearly parametrized estimate of the frequency response function from one block, i.e. from one input to the output. In particular, this expression reveals that the variance increases in one block when the number of estimated parameters in another block is increased, but levels off when the number of parameters in the other block reaches the number of parameters in the block in question. It also quantifies exactly how correlation between inputs affects the resulting accuracy and a graphical representation is provided for this purpose. The results are applicable to parallel MISO Hammerstein models when the nonlinearities are known and generalize an existing variance expression for this type of model.     

Optimally robust system identification of systems subject toamplitude-bounded stochastic disturbances

In this paper it is shown that log cos(πx/(2C)) is the optimally robust criterion function for prediction error methods with respect to amplitude-bounded stochastic disturbances. This criterion function minimizes the maximum asymptotic covariance matrix of the parameter estimates for the family of innovations of the systems which are amplitude bounded by the constant C. Furthermore, the stochastic worst case performance of the estimate corresponding to the criterion function log cos(πx/(2C)) is better than the worst case performance of the least squares estimate even if the constant C is chosen larger than the actual amplitude bound on the innovations. In addition to its favorable properties in a stochastic setting, this criterion function also generates estimates which are unfalsified in a deterministic framework     

Seasonal effects on the physicochemical characteristics of fish sauce made from capelin (Mallotus villosus)

Fresh capelin (Mallotus villosus) was harvested from the North Atlantic during both summer and winter fishing seasons. Reaction conditions for fish sauce processing were optimized with respect to temperature, salt concentration and reaction time, using a response surface methodology (RSM) experimental design. Whole capelin was minced and samples were ground with increasing salt concentrations. RSM optimizations were conducted, ranging from 5% to 30% salt, and incubating at 5° intervals from 0 to 65 °C. Autolytic activity was estimated by extracting the liquid formed by the mixture with trichloroacetic acid and estimating protein content by the Lowry method. Samples for fish sauce production were then prepared under optimized conditions by mixing ground capelin with 10% salt and incubating at 50 °C for up to 270 days for the summer capelin and up to 360 days for the winter capelin. Samples were collected at regular intervals and analyzed for liquid yield, moisture, protein, soluble solids, specific gravity, pH, colour and amino acid content. Kjeldahl protein content in the fish sauce from summer capelin was 2.03% after 250 days of fermentation and twice as high as that in winter capelin fish sauce. Moisture content and pH were lower in the summer capelin fish sauce, but Brix and density were higher than those in fish sauce from winter capelin. Brown colour formation was very rapid in the summer capelin fish sauce but slow in the winter capelin fish sauce. Summer capelin may be successfully utilized for the production of fish sauce without added enzymes.     

Composite modeling of transfer functions

The problem under consideration is how to estimate the frequency function of a system and the associated estimation error when a set of possible model structures is given and then one of them is known to contain the true system. The “classical” solution to this problem is to, first, use a consistent model structure selection criterion to discard all but one single structure, second, estimate a model in this structure and, third, conditioned on the assumption that the chosen structure contains the true system, compute an estimate of the estimation error. For a finite data set, however, one cannot guarantee that the correct structure is chosen, and this “structural” uncertainty is lost in the previously mentioned approach. In this contribution a method is developed that combines the frequency function estimates and the estimation errors from all possible structures into a joint estimate and estimation error. Hence, this approach bypasses the structure selection problem. This is accomplished by employing a Bayesian setting. Special attention is given to the choice of priors. With this approach it is possible to benefit from a priori information about the frequency function even though the model structure is unknown     

Nonlinear black-box models in system identification: Mathematical foundations

We discuss several aspects of the mathematical foundations of the nonlinear black-box identification problem. We shall see that the quality of the identification procedure is always a result of a certain trade-off between the expressive power of the model we try to identify (the larger the number of parameters used to describe the model, the more flexible is the approximation), and the stochastic error (which is proportional to the number of parameters). A consequence of this trade-off is the simple fact that a good approximation technique can be the basis of a good identification algorithm. From this point of view, we consider different approximation methods, and pay special attention to spatially adaptive approximants. We introduce wavelet and ‘neuron’ approximations, and show that they are spatially adaptive. Then we apply the acquired approximation experience to estimation problems. Finally, we consider some implications of these theoretical developments for the practically implemented versions of the ‘spatially adaptive’ algorithms.     

The fundamental role of general orthonormal bases in systemidentification

The purpose of this paper is threefold. Firstly, it is to establish that contrary to what might be expected, the accuracy of well-known and frequently used asymptotic variance results can depend on choices of fixed poles or zeros in the model structure. Secondly, it is to derive new variance expressions that can provide greatly improved accuracy while also making explicit the influence of any fixed poles or zeros. This is achieved by employing certain new results on generalized Fourier series and the asymptotic properties of Toeplitz-like matrices in such a way that the new variance expressions presented here encompass pre-existing ones as special cases. Via this latter analysis a new perspective emerges on recent work pertaining to the use of orthonormal basis structures in system identification. Namely, that orthonormal bases are much more than an implementational option offering improved numerical properties. In fact, they are an intrinsic part of estimation since, as shown here, orthonormal bases quantify the asymptotic variability of the estimates whether or not they are actually employed in calculating them     

An adaptive method for consistent estimation of real-valued non-minimum phase zeros in stable LTI systems

An adaptive algorithm, consisting of a recursive estimator for a finite impulse response model having two non-zero lags only, and an adaptive input are presented. The model is parametrized in terms of the first impulse response coefficient and the model zero. For linear time-invariant single-input single-output systems with real rational transfer functions possessing at least one real-valued non-minimum phase zero of multiplicity one, it is shown that the model zero converges to such a zero of the true system. In the case of multiple non-minimum phase zeros, the algorithm can be tailored to converge to a particular zero. The result is shown to hold for systems and noise spectra of arbitrary degree. The algorithm requires prior knowledge of the sign of the high frequency gain of the system as well as an interval to which the non-minimum phase zero of interest belongs.     

Towards incremental speech generation in conversational systems

This paper presents a model of incremental speech generation in practical conversational systems. The model allows a conversational system to incrementally interpret spoken input, while simultaneously planning, realising and self-monitoring the system response. If these processes are time consuming and result in a response delay, the system can automatically produce hesitations to retain the floor. While speaking, the system utilises hidden and overt self-corrections to accommodate revisions in the system. The model has been implemented in a general dialogue system framework. Using this framework, we have implemented a conversational game application. A Wizard-of-Oz experiment is presented, where the automatic speech recognizer is replaced by a Wizard who transcribes the spoken input. In this setting, the incremental model allows the system to start speaking while the user's utterance is being transcribed. In comparison to a non-incremental version of the same system, the incremental version has a shorter response time and is perceived as more efficient by the users.     

Variance results for identification of cascade systems

The objective of this contribution is to analyze statistical properties of estimated models of cascade systems. Models of such systems are important in for example cascade control applications. The aim is to present and analyze some fundamental limitations in the quality of an identified model of a cascade system under the condition that the true subsystems have certain common dynamics. The model quality is analyzed by studying the asymptotic (large data) covariance matrix of the Prediction Error Method parameter estimate. The analysis will focus on cascade systems with three subsystems. The main result is that if the true transfer functions of the first and second subsystem are identical, the output signal information from the second and third subsystems will not affect the asymptotic variance of the estimated model of the first subsystem. This result implies that for a cascade system with two subsystems, where the dynamics of the first subsystem is a factor of the dynamics of the second one, the output signal information from the second subsystem will not improve the asymptotic quality of the estimate of the first subsystem. The results are illustrated by some simple FIR examples.