Adaptive linear estimation for stationary M-dependent processes

In recent years adaptive linear estimation based upon the gradient-following algorithm has been proposed in a wide range of applications. However, little analysis on the convergence of the estimation has appeared when the elements of the data sequence are dependent. This paper presents such an analysis under the assumptions of stationarity andM-dependence (all data sets separated by more than a constantM < inftyare statistically independent). It is shown that for a sufficiently small adaptation constant, the mean error in the estimator weights converges to a finite limit, generally nonzero. In addition, hounds on the norm of the mean weight-deviation and on the mean norm-square of the weight-deviation are found and shown to converge to asymptotic bounds, which can be made arbitrarily small by decreasing the adaptation constant and increasing the data block length over which gradient estimates are made.     

Recursive probability density estimation for weakly dependent stationary processes

Recursive estimation of the univariate probability density functionf(x)for stationary processes{X_{j}}is considered. Quadratic-mean convergence and asymptotic normality for density estimatorsf_{n}(x)are established for strong mixing and for asymptotically uncorrelated processes{X_{j}}. Recent results for nonrecursive density estimators are extended to the recursive case.     

Convergence analysis of adaptive linear estimation for dependentstationary processes

The convergence properties of an adaptive linear mean-square estimator that uses a modified LMS algorithm are established for generally dependent processes. Bounds on the mean-square error of the estimates of the filter coefficients and on the excess error of the estimate of the signal are derived for input processes which are either strong mixing or asymptotically uncorrelated. It is shown that the mean-square deviation is bounded by a constant multiple of the adaptation step size and that the same holds for the excess error of the signal estimation. The present findings extend earlier results in the literature obtained for independent and M-dependent input data     

An asymptotically least-favorable Chernoff bound for a large class of dependent data processes

It is desired to determine the worst-case asymptotic error probability performance of a given detector operating in an environment of uncertain data dependency. A class of Markov data process distributions is considered which satisfy a one-shift dependency bound and agree with a specified univariate distribution. Within this dependency contamination class the distribution structure which minimizes the exponential rate of decrease of detection error probabilities is identified. This is a uniform least-favorability principle, because the least-favorable dependency structure is the same for all bounded memoryless detectors. The error probability exponential rate criterion used is a device of large deviations theory. The results agree well with previous results obtained using Pitman's asymptotic relative efficiency (ARE), which is a more tractable small-signal performance criterion. In contrast to ARE, large deviations theory is closely related to finite-sample error probabilities via the finite-sample Chernoff bounds and other exponentially tight bounds and other approximations.     

An adaptive run-to-run optimizing controller for linear andnonlinear semiconductor processes

This paper presents a new run-to-run (R2R) multiple-input-multiple-output controller for semiconductor manufacturing processes. The controller, termed optimizing adaptive quality controller (OAQC), can act both as an optimizer-in case equipment models are not available-or as a controller for given models. The main components of the OAQC are shown and a study of its performance is presented. The controller allows one to specify input and output constraints and weights, and input resolutions. A multivariate control chart can be applied either as a deadband on the controller or simply to provide out of control alarms. Experimental designs can be utilized for on-line (recursive) model identification in the optimization phase. For testing purposes, two chemical mechanical planarization processes were simulated based on real equipment models. It is shown that the OAQC allows one to keep adequate control even if the input-output transfer function is severely nonlinear. Software implementation including the integration of the OAQC with the University of Michigan's Generic Cell Controller (GCC) is briefly discussed     

Restricted-structure linear estimators for multiple-model systems

A new class of discrete-time optimal linear estimators is introduced for multiple-model systems that minimises a minimum-variance criterion but where the structure is prespecified to have a simple low-order form. The restricted-structure estimator can be of much lower order than a Kalman (1961) or Wiener (1949) estimator and it minimises the estimation-error variance, subject to the constraint referred to. The numerical optimisation algorithm is simple to implement and full-order optimal solutions are available as a by-product of the analysis. The algorithm enables low-order optimal estimators to be computed that directly minimise the cost index across a set of possible linear signal or noise source models. The main technical advances lie in the theoretical analysis that enables the expanded cost expression to be simplified before the numerical solution is obtained, and the extension of the restricted-structure optimisation technique to multiple-model systems     

Evaluation of the transient response of linear stationary systems to a class of excitations

A procedure is presented to evaluate the transient response of linear stationary systems to a class of excitations with Laplace transforms that are not rational functions. The procedure extends Liou's state-space approach which is limited to time functions whose Laplace transforms are rational functions.     

On the error matrix in optimal linear filtering of stationary processes

The error covariance matrix corresponding to optimal linear causal filtering of second-order stationary processes in additive noise is considered. Formulas expressing this error matrix in terms of the optimal transfer function are established, and in the nonsingular case the optimal transfer function is expressed in terms of the spectral densities. These are straightforward generalizations of previously published scalar results, and the derivation is similarly based on Hardy space theory. Explicit bounds on the minimal error (i.e., the trace of the optimal error covariance matrix) are obtained for filtering in white noise. Furthermore, an explicit expression for the error covariance matrix is derived for the case of transmitting the same signal over several white-noise channels.     

Analysis of adaptive differential PCM of a stationary Gauss - Markov input

An adaptive matched differential pulse-code modulator (AMDPCM) is analyzed. The adaptation of the symmetric uniform quantizer parameterDelta_{n}is performed by fixed multipliers assigned to the quantizer output levels. The input is stationary first-order Gauss-Markov. The correlation of the samples is used as the leakage parameter in the matched integrator, with the predictive reconstruction similarly matched. For a4-level quantizer and multipliers(gamma^{-1}, gamma)the limiting joint distribution of the prediction error andDelta_{n}is derived and the asymptotic sample-point and time-averaged mean-square error (rose) and mean and variance ofDelta_{n}as functions ofgamma in (1,2]are computed and plotted. It is found that the asymptotic performance of AMDPCM does not depend on the choice ofDelta_{0}, that the increase in mse incurred by using A(M)DPCM instead of (M)DPCM withDelta_{opt}is small, with mse(A(M)DPCM)downarrow min_{Delta}mse ((M)DPCM) asgamma downarrow 1, and that the signal-to-noise ratio of AMDPCM does not depend on the input power.     

NAR estimators of spatial covariance matrices for adaptive arraydetection

The theory of noise-alone-reference (NAR) power estimation is extended to the estimation of spatial covariance matrices. A NAR covariance estimator insensitive to signal presence is derived. The SNR (signal-to-noise ratio) loss incurred by using this estimator is independent of the input SNR and is less than that encountered with the maximum likelihood covariance estimator given that the same number of uncorrelated snapshots is available to both estimators. The analysis assumes first a deterministic signal. The results are extended and generalized to signals with unknown parameters or random signals. For the random signal case, generalized and quasi-NAR covariance estimators are presented     

A class of shrinkage estimators for the variance of a normal population

The crux of this paper is to propose a class of shrinkage estimators for the variance of a normal population and study its properties. Some estimators are generated from a proposed class and compared with the usual unbiased estimator, minimum mean squared error (MMSE) estimator and Pandey and Singh, South African Statistical Journal (1976) and J. Indian Statistical Assoc. 15, 141–150 (1977) estimator.     

Harmonic analysis for a class of multiplicative processes

The harmonic analysis of certain multiplicative processes of the formg(t)X(t)is considered, wheregis a deterministic function, and the stochastic processX(t)is of the formX(t)=sum X_{n}l_{[n alpha , (n+l) alpha]}(t), where a is a positive constant and theX_{n}, n=0, pm 1,pm 2, cdotsare independent and identically distributed random variables with zero means and finite variances. In particular, we show that if g is Riemann integrable and periodic, with period incommensurate withalpha, theng(t)X(t)has an autocovariance in the Wiener sense equal to the product of the Wiener autocovariances of its factors,C_{gx} = C_{g}C_{x}. Some important cases are examined where the autocovariance of the multiplicative process exists but cannot be obtained multiplicatively.     

Fast recursive basis function estimators for identification oftime-varying processes

When system parameters vary rapidly with time, the weighted least squares filters are not capable of following the changes satisfactorily; some more elaborate estimation schemes, based on the method of basis functions, have to be used instead. The basis function estimators have increased tracking capabilities but are computationally very demanding. The paper introduces a new class of adaptive filters, based on the concept of postfiltering, which have improved parameter tracking capabilities that are typical of the basis function algorithms but, at the same time, have pretty low computational requirements, which is typical of the weighted least squares algorithms     

Interpolation of complex stationary processes

The problem of minimum mean-square infinite extent interpolation for discrete-time stationary complex stochastic processes is studied. The interpolator consists of linear combinations of samples of the process and of their complex conjugate. The expressions of the interpolator and of the approximation error are derived and various consequences are examined. It is shown in particular that the approximation error may be zero while the interpolation error obtained when using only linear combinations of the samples is maximum     

A tight upper bound on the gain of linear and nonlinear predictorsfor stationary stochastic processes

One of the striking questions in prediction theory is this: is there a chance to predict future values of a given signal? Usually, we design a predictor for a special signal or problem and then measure the resulting prediction quality. If there is no a priori knowledge on the optimal predictor, the achieved prediction gain will depend strongly of the prediction model used. To cope with this lack of knowledge, a theorem on the maximum achievable prediction gain of stationary signals is presented. This theorem provides the foundation for estimating a quality goal for the predictor design, independent of a special predictor implementation (linear or nonlinear). As usual, the prediction gain is based on the mean square error (MSE) of the predicted signal. The achievable maximum of the prediction gain is calculated using an information theoretic quantity known as the mutual information. In order to obtain the gain, we use a nonparametric approach to estimate the maximum prediction gain based on the observation of one specific signal. We illustrate this by means of well-known example signals and show an application to load forecasting. An estimation algorithm for the prediction gain has been implemented and used in the experimental part of the paper     

Two-sided linear prediction and reconstruction of a stationary timeseries

The problem of two-sided linear prediction and reconstruction of a stationary process y on a finite interval is solved     

Nonparametric polyspectral estimators for kth-order (almost)cyclostationary processes

Second- and higher-order almost cyclostationary processes are random signals with almost periodically time-varying statistics. The class includes stationary and cyclostationary processes as well as many real-life signals of interest. Cyclic and time-varying cumulants and polyspectra are defined for discrete-time real kth-order cyclostationary processes, and their interrelationships are explored. Smoothed polyperiodograms are proposed for cyclic polyspectral estimation and are shown to be consistent and asymptotically normal. Asymptotic covariance expressions are derived along with their computable forms. Higher than second-order cyclic cumulants and polyspectra convey time-varying phase information and are theoretically insensitive to any stationary (for nonzero cycles) as well as additive cyclostationary Gaussian noise (for all cycles)     

An efficient Newton-type method for the computation of ML estimators in a uniform linear array

In the problem of estimating the angles of arrival to a uniform linear array, we present an efficient method to compute Maximum Likelihood (ML) estimations, based on the Modified Variable Projection (MVP) algorithm. In contrast to methods like Iterative Quadratical Maximum Likelihood (IQML) or the Iterative Method of Direction Estimation (IMODE), it is not based on a polynomial parameterization but on directly exploiting the Vandermonde structure through analytical tools like the Fast Fourier Transform (FFT), the geometric series summation formula, and Horner's synthetic division. The computational burden of the proposed method is significantly smaller than the burden of IMODE and of the Relaxation (RELAX) algorithm. Besides, it is shown that the computation of the ML estimation can be divided in a preliminary step in which a few FFTs are computed and an iterative step with a complexity that is independent of the array size.     

Memory-universal prediction of stationary random processes

We consider the problem of one-step-ahead prediction of a real-valued, stationary, strongly mixing random process (Xi)_i=-∞^∞. The best mean-square predictor of X₀ is its conditional mean given the entire infinite past (X_i)_i=-∞^-1. Given a sequence of observations X₁, X₂, X_N, we propose estimators for the conditional mean based on sequences of parametric models of increasing memory and of increasing dimension, for example, neural networks and Legendre polynomials. The proposed estimators select both the model memory and the model dimension, in a data-driven fashion, by minimizing certain complexity regularized least squares criteria. When the underlying predictor function has a finite memory, we establish that the proposed estimators are memory-universal: the proposed estimators, which do not know the true memory, deliver the same statistical performance (rates of integrated mean-squared error) as that delivered by estimators that know the true memory. Furthermore, when the underlying predictor function does not have a finite memory, we establish that the estimator based on Legendre polynomials is consistent