Linear modeling of multidimensional non-gaussian processes using cumulants

Extending the notion of second-order correlations, we define thecumulants of stationary non-Gaussian random fields, and demonstrate their potential for modeling and reconstruction of multidimensional signals and systems. Cumulants and their Fourier transforms calledpolyspectra preserve complete amplitude and phase information of a multidimensional linear process, even when it is corrupted by additive colored Gaussian noise of unknown covariance function. Relying on this property, phase reconstruction algorithms are developed using polyspectra, which can be computed via a 2-D FFT-based algorithm. Additionally, consistent ARMA parameter estimators are derived for identification of linear space-invariant multidimensional models which are driven by unobservable, i.i.d., non-Gaussian random fields. Contrary to autocorrelation based multidimensional modeling approaches, when cumulants are employed, the ARMA model is allowed to be non-minimum phase, asymmetric non-causal or non-separable.This work was performed at the University of Southern California, Los Angeles, under National Science Foundation Grant ECS-8602531 and Naval Ocean Systems Center Contract N6601-85-D-0203. The second author was partly supported by Univ. of Virginia Engr. Research Initiation Grant 6-42410, and HDL Contract 5-25227. Parts of this paper were presented at ICASSP-88, New York, NY, April 1988, and at the IV IEEE ASSP Workshop on Spectrum Estimation and Modeling, Minneapolis, MN, August 1988.     

Interpolation of complex stationary processes

The problem of minimum mean-square infinite extent interpolation for discrete-time stationary complex stochastic processes is studied. The interpolator consists of linear combinations of samples of the process and of their complex conjugate. The expressions of the interpolator and of the approximation error are derived and various consequences are examined. It is shown in particular that the approximation error may be zero while the interpolation error obtained when using only linear combinations of the samples is maximum     

On the asymptotic capacity of stationary Gaussian fading channels

We consider a peak-power-limited single-antenna flat complex-Gaussian fading channel where the receiver and transmitter, while fully cognizant of the distribution of the fading process, have no knowledge of its realization. Upper and lower bounds on channel capacity are derived, with special emphasis on tightness in the high signal-to-noise ratio (SNR) regime. Necessary and sufficient conditions (in terms of the autocorrelation of the fading process) are derived for capacity to grow double-logarithmically in the SNR. For cases in which capacity increases logarithmically in the SNR, we provide an expression for the "pre-log", i.e., for the asymptotic ratio between channel capacity and the logarithm of the SNR. This ratio is given by the Lebesgue measure of the set of harmonics where the spectral density of the fading process is zero. We finally demonstrate that the asymptotic dependence of channel capacity on the SNR need not be limited to logarithmic or double-logarithmic behaviors. We exhibit power spectra for which capacity grows as a fractional power of the logarithm of the SNR     

Memory-universal prediction of stationary random processes

We consider the problem of one-step-ahead prediction of a real-valued, stationary, strongly mixing random process (Xi)_i=-∞^∞. The best mean-square predictor of X₀ is its conditional mean given the entire infinite past (X_i)_i=-∞^-1. Given a sequence of observations X₁, X₂, X_N, we propose estimators for the conditional mean based on sequences of parametric models of increasing memory and of increasing dimension, for example, neural networks and Legendre polynomials. The proposed estimators select both the model memory and the model dimension, in a data-driven fashion, by minimizing certain complexity regularized least squares criteria. When the underlying predictor function has a finite memory, we establish that the proposed estimators are memory-universal: the proposed estimators, which do not know the true memory, deliver the same statistical performance (rates of integrated mean-squared error) as that delivered by estimators that know the true memory. Furthermore, when the underlying predictor function does not have a finite memory, we establish that the estimator based on Legendre polynomials is consistent     

Nonlinear infinite extent interpolation of stationary processes

The problem of minimum mean-square interpolation of discrete time stationary stochastic processes using a system consisting of a finite number p of nonlinear instantaneous transformations followed by a p input-one output infinite extent linear filter is studied. The expressions of the optimal interpolator and of the approximation error are derived and generalize the corresponding relations known in linear interpolation theory. Some extensions, including the estimation problem with several missing observations, are also investigated     

Series expansion of wide-sense stationary random processes

This paper presents a general approach to the derivation of series expansions of second-order wide-sense stationary mean-square continuous random process valid over an infinite-time interval. The coefficients of the expansion are orthogonal and convergence is in the mean-square sense. The method of derivation is based on the integral representation of such processes. It covers both the periodic and the aperiodic cases. A constructive procedure is presented to obtain an explicit expansion for a given spectral distribution.     

On Walsh differentiable dyadically stationary random processes

Some basic properties of dyadically stationary (DS) processes are introduced, including continuity and spectral representation. A sampling theorem based on the Walsh functions is investigated for random signals that are not necessarily sequency-limited. By using the concept of a dyadic derivative, the resulting aliasing error is calculated together with the speed of convergence. An example gives a glimpse into the possibilities of applying the sampling theorem as well as the dyadic derivative.     

Adaptive linear estimation for stationary M-dependent processes

In recent years adaptive linear estimation based upon the gradient-following algorithm has been proposed in a wide range of applications. However, little analysis on the convergence of the estimation has appeared when the elements of the data sequence are dependent. This paper presents such an analysis under the assumptions of stationarity andM-dependence (all data sets separated by more than a constantM < inftyare statistically independent). It is shown that for a sufficiently small adaptation constant, the mean error in the estimator weights converges to a finite limit, generally nonzero. In addition, hounds on the norm of the mean weight-deviation and on the mean norm-square of the weight-deviation are found and shown to converge to asymptotic bounds, which can be made arbitrarily small by decreasing the adaptation constant and increasing the data block length over which gradient estimates are made.     

A positive systems model of TCP-like congestion control: asymptotic results

We study communication networks that employ drop-tail queueing and Additive-Increase Multiplicative-Decrease (AIMD) congestion control algorithms. It is shown that the theory of nonnegative matrices may be employed to model such networks. In particular, important network properties, such as: 1) fairness; 2) rate of convergence; and 3) throughput, can be characterized by certain nonnegative matrices. We demonstrate that these results can be used to develop tools for analyzing the behavior of AIMD communication networks. The accuracy of the models is demonstrated by several NS studies.     

New asymptotic results in parallel distributed detection

The performance of a parallel distributed detection system is investigated as the number of sensors tends to infinity. It is assumed that the i.i.d. sensor data are quantized locally into m-ary messages and transmitted to the fusion center for binary hypothesis testing. The boundedness of the second moment of the postquantization log-likelihood ratio is examined in relation to the asymptotic error exponent. It is found that, when that second moment is unbounded, the Neyman-Pearson error exponent can become a function of the test level, whereas the Bayes error exponent remains, as previously conjectured by J.N. Tsitsiklis, (1986), unaffected. Large deviations techniques are also used to show that in Bayes testing the equivalence of absolutely optimal and best identical-quantizer systems is not limited to error exponents, but extends to the actual Bayes error probabilities up to a multiplicative constant     

The asymptotic CRLB for the spectrum of ARMA processes

This paper addresses the issue of quantifying the frequency domain accuracy of autoregressive moving average (ARMA) spectral estimates as dictated by the Cramer-Rao lower bound (CRLB). Classical work in this area has led to expressions that are asymptotically exact as both data length and model order tend to infinity, although they are commonly used in finite model order and finite data length settings as approximations. More recent work has established quantifications that, for AR models, are exact for finite model order. By employing new analysis methods based on rational orthonormal parameterizations, together with the ideas of reproducing kernel Hilbert spaces, this paper develops quantifications that extend this previous work by being exact for finite model order in all of the AR, MA, and ARMA system cases. These quantifications, via their explicit dependence on poles and zeros of the underlying spectral factor, reveal certain fundamental aspects of the accuracy achievable by spectral estimates of ARMA processes.     

New results on coding of stationary nonergodic sources

Two results on the coding of stationary nonergodic sources are presented. The first is a source coding theorem stating that there exist variable-rate codes with performance arbitrarily close to the rate-distortion function of the stationary nonergodic source. The second is a converse information transmission theorem. It is shown that the distortion which results when the source is transmitted across a channel with capacityCis no less than the least distortion achievable by fixed-rate codes with rateC.     

The first-order asymptotic of waiting times with distortion betweenstationary processes

Let X and Y be two independent stationary processes on general metric spaces, with distributions P and Q, respectively. The first-order asymptotic of the waiting time W_n(D) between X and Y, allowing distortion, is established in the presence of one-sided ψ-mixing conditions for Y. With probability one, n^-1log W _n(D) has the same limit as -n^-1logQ(B(X₁ⁿ, D)), where Q(B(X₁ ⁿ, D)) is the Q-measure of the D-ball around (X₁ ,...,X_n), with respect to a given distortion measure. Large deviations techniques are used to get the convergence of -n^-1 log Q(B(X₁ⁿ, D)). First, a sequence of functions R_n in terms of the marginal distributions of X₁ⁿ and Y₁ⁿ as well as D are constructed and demonstrated to converge to a function R(P, Q, D). The functions R_n and R(P, Q, D) are different from rate distortion functions. Then -n^-1logQ(B(X₁ⁿ , D)) is shown to converge to R(P, Q, D) with probability one     

Spectral analysis of stationary components of periodically correlated random processes

The properties of stationary components of periodically correlated random processes separated by using zonal filtering have been considered. The properties of estimates of their correlation and spectral characteristics were investigated. The latter were built by using the Blackman-Tukey method.     

Recursive probability density estimation for weakly dependent stationary processes

Recursive estimation of the univariate probability density functionf(x)for stationary processes{X_{j}}is considered. Quadratic-mean convergence and asymptotic normality for density estimatorsf_{n}(x)are established for strong mixing and for asymptotically uncorrelated processes{X_{j}}. Recent results for nonrecursive density estimators are extended to the recursive case.     

Exact likelihood for stationary vector autoregressive moving average processes

Autoregressive moving average (ARMA) models are useful approximants to the kinds of random processes commonly encountered in discrete-time signal processing applications. Such models may be used to compress data in low bit-rate information transmission, to improve frequency resolution in spectrum analysis, and to forecast in economic, meteorological, and other time series.In this paper we discuss several aspects of the maximum likelihood theory of parameter identification in ARMA models. We highlight the role of ‘innovations’ representations in exact likelihood theory and show how internal model structure may be used to speed up calculation of likelihood in either fast Kalman predictor or fast lattice implementations.     

Parameter estimation of continuous-time stationary Gaussian processes with rational spectra

This paper considers the problem of estimating the parameters of continuous-time stationary Gaussian processes with rational spectra, from uniformly sampled measurements. The sampled process is shown to be an autoregressive moving-average process, and explicit relationships between the parameters of the continuous-time and the sampled processes are derived. These relationships are then used to derive a lower bound on the variances of biased estimates of the continuous-time parameters, and on the generalized variance of such estimates. It is shown by some examples that the bound on the generalized variance depends on the sampling interval in a nonmonotonic manner. In particular, for each specific set of parameters there exists a sampling interval for which the lower bound is minimized.This work was supported by the Army Research Office under Contract Number DAAG29-83-C-0027.     

A matrix-valued wavelet KL-like expansion for wide-sense stationary random processes

Matrix-valued wavelet series expansions for wide-sense stationary processes are studied in this paper. The expansion coefficients a are uncorrelated matrix random process, which is a property similar to that of a matrix Karhunen-Loe/spl grave/ve (MKL) expansion. Unlike the MKL expansion, however, the matrix wavelet expansion does not require the solution of the eigen equation. This expansion also has advantages over the Fourier series, which is often used as an approximation to the MKL expansion in that it completely eliminates correlation. The basis functions of this expansion can be obtained easily from wavelets of the Matrix-valued Lemarie/spl acute/-Meyer type and the power-spectral density of the process.