Nonlinear systems analysis with non-Gaussian white stimuli; General basis functionals and kernels (Corresp.)

The Wiener-Lee-Schetzen scheme of using Gaussian white noise to test a nonlinear dynamical system is extended in two ways. 1) An arbitrary non-Ganssian white noise stationary signal can be used as the test stimulus. 2) An arbitrary function of this stimulus can then be used as the analyzing function for cross correlating with the response to obtain the kernels characterizing the system. Closed form expressions are given for the generalized orthogonal basis functions. The generalized kernels are expanded in terms of Volterra kernels and Wiener kernels. The expansion coefficients are closely related to the cumulants of the stimulus probability distribution. These results are applied to the special case of a Gaussian stimulus and a three-level analysis function. For this case a detailed analysis is Lade of the magnitude of the deviation of the kernels obtained with the ternary truncation as compared to the Wiener kernels obtained by cross correlating with the same Gaussian as was used for the stimulus. The deviations are found to be quite small.     

Second-order analysis of the output of a discrete-time Volterra system driven by white noise

The second-order analysis of the output of a discrete-time nonlinear system described by a truncated Volterra series whose input consists of a sequence of independent random variables (white noise)is considered. The main result consists of an explicit formula for the mean value of the output process in terms of the cumulants of the input and of Volterra kernels. This formula, with suitable modifications, allows the calculation of the output correlation as well as of the continuous and discrete components of the spectral distribution. Several applications are considered, and in particular the Gaussian white noise case is worked out in detail. Finally, the computational aspects of the analysis are discussed with the aim of showing that in several situations closed-form results can be obtained.     

Application of the Volterra Functional Expansion in the Detection of Nonlinear Functions of Gaussian Processes

Detection of a memoryless nonlinear functional of a Gaussian process in additive Gaussian white noise is considered. The Volterra functional expansion for the likelihood ratio, and two examples of calculating the kernels are presented. It is shown that kernels up to third order can be obtained for a hard-limited Gaussian process and for the absolute value of a Gaussian process. For the case of hard limiting, the kernels are nonlinear functions of the autocorrelation of the Gaussian process. For the absolute value case, the kernels are nonlinear functions of the kernel derived for the linear problem. A Monte Carlo simulation of receiver performance is presented for the case of detection of the absolute value of a first-order Butterworth process in additive Gaussian white noise. The suboptimal detector is obtained by truncating the log likelihood ratio to second order.     

Second-order adaptive Volterra system identification based ondiscrete nonlinear Wiener model

The authors present the nonlinear LMS adaptive filtering algorithm based on the discrete nonlinear Wiener (1942) model for second-order Volterra system identification application. The main approach is to perform a complete orthogonalisation procedure on the truncated Volterra series. This allows the use of the LMS adaptive linear filtering algorithm for calculating all the coefficients with efficiency. This orthogonalisation method is based on the nonlinear discrete Wiener model. It contains three sections: a single-input multi-output linear with memory section, a multi-input, multi-output nonlinear no-memory section and a multi-input, single-output amplification and summary section. For a white Gaussian noise input signal, the autocorrelation matrix of the adaptive filter input vector can be diagonalised unlike when using the Volterra model. This dramatically reduces the eigenvalue spread and results in more rapid convergence. Also, the discrete nonlinear Wiener model adaptive system allows us to represent a complicated Volterra system with only few coefficient terms. In general, it can also identify the nonlinear system without over-parameterisation. A theoretical performance analysis of steady-state behaviour is presented. Computer simulations are also included to verify the theory     

A digital technique to estimate second-order distortion usinghigher order coherence spectra

A digital spectral method for evaluating second-order distortion of a nonlinear system, which can be represented by Volterra kernels up to second order and which is subjected to a random noise input, is discussed. The importance of departures from the commonly assumed Gaussian excitation is investigated. The Hinich test is shown to be an appropriate test for orthogonality in the system identification. Tests for Gaussianity of two important sources, which are commonly used for Gaussian inputs in nonlinear system identification, are presented: (1) commercial software routines for simulation experiments, and (2) noise generators for practical experiments. The deleterious effects of assuming a Gaussian input when it is not are demonstrated. The random input method for evaluating the second-order distortion of a nonlinear system is compared with the sine-wave input method using both simulation and experimental data. The approach is applied to a loudspeaker in the low-frequency band     

Nonlinear system modeling based on the Wiener theory

This paper is a tutorial of nonlinear system modeling methods which are based on the Wiener theory of nonlinear systems. The basic concepts that underlie the Wiener theory are discussed and illustrated. Various modeling methods are presented by which a non-linear system can be modeled using either white Gaussian, nonwhite Gaussian, or certain non-Gaussian inputs. The experimental error in determining the Wiener model is discussed in terms of a new concept called measurement stability. Since attempts are being made to apply these modeling methods to diverse areas of study, this paper is written to be comprehensible by nonspecialists in system theory     

Estimating second-order Volterra system parameters from noisy measurements based on an LMS variant or an errors-in-variables method

This paper deals with the identification of a nonlinear SISO system modelled by a second-order Volterra series expansion when both the input and the output are disturbed by additive white Gaussian noises. Two methods are proposed. Firstly, we present an unbiased on-line approach based on the LMS. It includes a bias correction scheme which requires the variance of the input additive noise. Secondly, we suggest solving the identification problem as an errors-in-variables issue, by means of the so-called Frisch scheme. Although its computational cost is high, this approach has the advantage of estimating the Volterra kernels and the variances of both the additive noises and the input signal, even if the signal-to-noise ratios at the input and the output are low.     

Blind identification of quadratic nonlinear models using neuralnetworks with higher order cumulants

A novel approach to blindly estimate kernels of any discrete- and finite-extent quadratic models in higher order cumulants domain based on artificial neural networks is proposed in this paper. The input signal is assumed an unobservable independently identically, distributed random sequence which is viable for engineering practice. Because of the properties of the third-order cumulant functions, identifiability of the nonlinear model holds, even when the model output measurement is corrupted by a Gaussian random disturbance. The proposed approach enables a nonlinear relationship between model kernels and model output cumulants to be established by means of neural networks. The approximation ability of the neural network with the weights-decoupled extended Kalman filter training algorithm is then used to estimate the model parameters. Theoretical statements and simulation examples together with practical application to the train vibration signals modeling corroborate that the developed methodology is capable of providing a very promising way to identify truncated Volterra models blindly     

Parameter estimation and spectral analysis of the discrete nonlinear secondorder Wiener filter (NSWF)

In many problems of digital signal processing, it is required to determine a model matching the statistics of a given observation of a generally non-Gaussian random process. Because of the wide range of systems that can be represented by Volterra series and Wiener expansions, the discrete nonlinear second-order Wiener filter (NSWF) driven by white Gaussian noise has been used in this study to match the statistics of a discrete zero-mean stationary non-Gaussian random process. First, we derive the autocorrelation function and show that it does not provide sufficient information necessary for estimating the parameters of the proposed model. Next, we derive the third-order moment sequence and show that it provides additional information that can be used in conjunction with the autocorrelation function to solve the problem. The power spectrum and bispectrum of the discrete NSWF have been also derived.     

Sigma-delta modulation with i.i.d. Gaussian inputs

The response of a single-loop sigma-delta modulator to an independent identically distributed (i.i.d.) Gaussian input signal is analyzed. A continuous-time stochastic model is developed and the connection of the model to the system is described. A condition is given so that the difference between the behavior of the model and that of the true system can be made arbitrarily small. Theories from renewal and Wiener processes are applied to show the convergence and mixing properties of the output sequence. Also derived is the power spectrum of the quantization noise. Compared to the spectrum when the input is DC, the i.i.d. Gaussian random process smears the discrete spectrum into band structures     

Nonlinear Evolution of Gaussian ASE Noise in ZMNL Fiber

This paper investigates the evolution of kurtosis of the input Gaussian amplified spontaneous emission (ASE) noise in a nonlinear fiber with negligible dispersion. The nonlinear Schrodinger equation (NLSE) describing propagation in optical fibers is simplified such that the fiber represents a zero memory nonlinear (ZMNL) system, and this approximation allows the development of analytical formulas for the statistical moments of the output noise. It is possible to calculate moments of all integer orders and the explicit expressions for the first four moments are given. The investigations show that the ASE noise does not preserve its Gaussian character when Kerr nonlinearity is significant. This observation proves that the common assumption of the Gaussian output ASE is not necessarily valid. Numerical simulations are provided to support the derivation. Kurtosis deviating significantly from the value typical for Gaussian noise is also an indicator that BER calculation in the coherent systems based on the assumption that ASE is Gaussian is likely to be inaccurate.     

Nonlinear Influence of T-Channels in an in silico Relay Neuron

Thalamic relay cells express distinctive response modes based on the state of a low-threshold calcium channel (T-channel). When the channel is fully active (burst mode), the cell responds to inputs with a high-frequency burst of spikes; with the channel inactive ( tonic mode), the cell responds at a rate proportional to the input. Due to the T-channel's dynamics, we expect the cell's response to become more nonlinear as the channel becomes more active. To test this hypothesis, we study the response of an in silico relay cell to Poisson spike trains. We first validate our model cell by comparing its responses with in vitro responses. To characterize the model cell's nonlinearity, we calculate Poisson kernels, an approach akin to white noise analysis but using the randomness of Poisson input spikes instead of Gaussian white noise. We find that a relay cell with active T-channels requires at least a third-order system to achieve a characterization as good as a second-order system for a relay cell without T-channels.      

Stochastic gradient based third-order Volterra system identification by using nonlinear Wiener adaptive algorithm

The nonlinear Wiener stochastic gradient adaptive algorithm for third-order Volterra system identification application with Gaussian input signals is presented. The complete self-orthogonalisation procedure is based on the delay-line structure of the nonlinear discrete Wiener model. The approach diagonalises the autocorrelation matrix of an adaptive filter input vector which dramatically reduces the eigenvalue spread and results in more rapid convergence speed. The relationship between the autocorrelation matrix and cross-correlation matrix of filter input vectors of both nonlinear Wiener and Volterra models is derived. The algorithm has a computational complexity of O(M/sup 3/) multiplications per sample input where M represents the length of memory for the system model, which is comparable to the existing algorithms. It is also worth noting that the proposed algorithm provides a general solution for the Volterra system identification application. Computer simulations are included to verify the theory.     

On identification of certain nonlinear systems (Corresp.)

In the fields of communication and control there sometimes arises the problem of determining the characteristics of a time-invariant system from discrete records of its input and output during a limited interval of time, where the output data are contaminated with random noise. When this system is linear, we can use the convolution sum to obtain a characterization of the output as a linear combination of past inputs. Hill and McMurtry [4] showed that if we choose a Legendre binary noise sequence as input to a linear system, then the least squares approximation to the characterizing coefficients is expressible in a computationally feasible form, even when a large number of coefficients is involved. In this correspondence we characterize a nonlinear system by a generalization of the convolution sum and show that if we choose Golomb's maximal linear recurring binary-noise sequence [2] as input, then the least squares approximation to the characterizing coefficients is expressible in a computationally feasible form. Thus, the maximal linear recurring sequence occupies the same role in investigating certain nonlinear systems that the Legendre sequence occupies in investigating linear systems.     

Blind Maximum-Likelihood Identification of Wiener Systems

This paper is about the identification of discrete-time Wiener systems from output measurements only (blind identification). Assuming that the unobserved input is white Gaussian noise, that the static nonlinearity is invertible, and that the output is observed without errors, a Gaussian maximum-likelihood estimator is constructed. Its asymptotic properties are analyzed and the Cramer-Rao lower bound is calculated. A two-step procedure for generating high-quality initial estimates is presented as well. The paper includes the illustration of the method on a simulation example.     

Adaptive AR modeling in white Gaussian noise

Autoregressive (AR) modeling is widely used in signal processing. The coefficients of an AR model can be easily obtained with a least mean square (LMS) prediction error filter. However, it is known that this filter gives a biased solution when the input signal is corrupted by white Gaussian noise. Treichler (1979) suggested the γ-LMS algorithm to remedy this problem and proved that the mean weight vector can converge to the Wiener solution. In this paper, we develop a new algorithm that extends works of Vijayan et al. (1990), for adaptive AR modeling in the presence of white Gaussian noise. By theoretical analysis, we show that the performance of the new algorithm is superior to the γ-LMS filter. Simulations are also provided to support our theoretical results     

Nonlinear system identification using Gaussian inputs

The paper is concerned with the identification of nonlinear systems represented by Volterra expansions and driven by stationary, zero mean Gaussian inputs, with arbitrary spectra that are not necessarily white. Procedures for the computation of the Volterra kernels both in the time as well as in the frequency domain are developed based on cross-cumulant information. The derived kernels are optimal in the mean squared error sense for noncausal systems. Order recursive procedures based on minimum mean squared error reduction are derived. More general input output representations that result when the Volterra kernels are expanded in a given orthogonal base are also considered     

On information transfer by envelope-constrained signals over theAWGN channel

Using T.E. Duncan's theorem (1970) on the relation between mutual information and the mean-square error of the optimum causal estimator of a random signal in additive white Gaussian noise (AWGN), the maximum achievable information transfer over the AWGN channel is derived with the random telegraph wave input. The information transfer is bounded and symptotically determined for the Wiener phase-modulated process input at large signal-to-noise ratio (SNR). Both results are compared to the information transfer for the capacity-achieving Gauss-Markov input process. For both the Wiener phase-modulated and the Gauss-Markov processes the information transfer increases asymptotically as the square root of SNR, but for the random telegraph wave it increases only as its logarithm     

A general likelihood-ratio formula for random signals in Gaussian noise

It is shown that the likelihood ratio for the detection of a random, not necessarily Gaussian, signal in additive white Gaussian noise has the same form as that for a known signal in white Gaussian noise. The role of the known signal is played by the casual least-squares estimate of the signal from the observations. However, the "correlation" integral has to be interpreted in a special sense as an Itô stochastic integral. It will be shown that the formula includes all known explicit formulas for signals in white Gaussian noise. However, and more important, the formula suggests an "estimator-correlator" philosophy for engineering approximation of the optimum receiver. Some extensions of the above result are also discussed, e.g., additive finite-variance, not necessarily Gaussian, noise plus a white Gaussian noise component. Purely colored Gaussian noise can be treated if whitening filters can be specified. The analog implementation of Itô integrals is briefly discussed. The proofs of the formulas are based on the concept of an innovation process, which has been useful in certain related problems of linear and nonlinear least-squares estimation, and on the concept of covariance factorization.     

The Gaussian Nonlinearity's Output Power Spectral Density Due to a Sinusoid Plus Band-Limited Noise

A sinusoid accompanied by stationary, additive, nonzero-mean, band-limited, white Gaussian noise is passed through a Gaussian nonlinear characteristic, and the output power spectral density is evaluated by using known results due to Rice and Atherton. Several characteristics of the Gaussian nonlinearity are revealed in the process, and input tuning is shown to contribute to output noise reduction. The results are applicable in the analysis of tracking systems employing sinusoidal dithers.