Wiener design of adaptation algorithms with time-invariant gains

A design method is presented that extends least mean squared (LMS) adaptation of time-varying parameters by including general linear time-invariant filters that operate on the instantaneous gradient vector. The aim is to track time-varying parameters of linear regression models in situations where the regressors are stationary or have slowly time-varying properties. The adaptation law is optimized with respect to the steady-state parameter error covariance matrix for time-variations modeled as vector-ARIMA processes. The design method systematically uses prior information about time-varying parameters to provide filtering, prediction, or fixed lag smoothing estimates for arbitrary lags. The method is based on a transformation of the adaptation problem into a Wiener filter design problem. The filter works in open loop for slow parameter variations, whereas a time-varying closed loop has to be considered for fast variations. In the latter case, the filter design is performed iteratively. The general form of the solution at each iteration is obtained by a bilateral Diophantine polynomial matrix equation and a spectral factorization. For white gradient noise, the Diophantine equation has a closed-form solution. Further structural constraints result in very simple design equations. Under certain model assumptions, the Wiener designed adaptation laws reduce to LMS adaptation. Compared with Kalman estimators, the channel tracking performance becomes nearly the same in mobile radio applications, whereas the complexity is, in general, much lower     

Adaptive polynomial filters

Adaptive nonlinear filters equipped with polynomial models of nonlinearity are explained. The polynomial systems considered are those nonlinear systems whose output signals can be related to the input signals through a truncated Volterra series expansion or a recursive nonlinear difference equation. The Volterra series expansion can model a large class of nonlinear systems and is attractive in adaptive filtering applications because the expansion is a linear combination of nonlinear functions of the input signal. The basic ideas behind the development of gradient and recursive least-squares adaptive Volterra filters are first discussed. Adaptive algorithms using system models involving recursive nonlinear difference equations are then treated. Such systems may be able to approximate many nonlinear systems with great parsimony in the use of coefficients. Also discussed are current research trends and new results and problem areas associated with these nonlinear filters. A lattice structure for polynomial models is described     

A polynomial approach to optimal and adaptive filtering withapplication to speech

Explicit polynomial solutions to the Wiener filtering problem are given. They rely on the identification of innovations models for the disturbance and for the noisy signal. The Wiener filter is found from the solution of a diophantine equation. Results illustrating the attenuation of background interference in a speech signal are presented. The explicit approach presented does not rely on minimizing a prediction error over a performance surface and can be applied where two input techniques are impracticable or impossible     

The structure and design of realizable decision feedback equalizers for IIR channels with colored noise

A simple algorithm for optimizing decision feedback equalizers (DFEs) by minimizing the mean-square error (MSE) is presented. A complex baseband channel and correct past decisions are assumed. The dispersive channel may have infinite impulse response, and the noise may be colored. Consideration is given to optimal realizable (stable and finite-lag smoothing) forward and feedback filters in discrete time. They are parameterized as recursive filters. In the special case of transmission channels with finite impulse response and autoregressive noise, the minimum MSE can be attained with transversal feedback and forward filters. In general, the forward part should include a noise-whitening filter (the inverse noise model). The finite realizations of the filters are calculated using a polynomial equation approach to the linear quadratic optimization problem. The equalizer is optimized essentially by solving a system of linear equations Ax=B, where A contains transfer function coefficients from the channel and noise model. No calculation of correlations is required with this method. A simple expression for the minimal MSE is presented. The DFE is compared to MSE-optimal linear recursive equalizers. Expressions for the equalizer in the limiting case of infinite smoothing lags are also discussed.<>     

Optimal Filtering for Incompletely Measured Polynomial Systems with Multiplicative Noise

In this paper, the optimal filtering problem for polynomial system states with polynomial multiplicative noise over linear observations with an arbitrary, not necessarily invertible, observation matrix is treated proceeding from the general expression for the stochastic Ito differential of the optimal estimate and the error variance. Thus, the Ito differentials for the optimal estimate and error variance corresponding to the stated filtering problem are first derived. A transformation of the observation equation is introduced to reduce the original problem to the previously solved one with an invertible observation matrix. The procedure for obtaining a closed system of the filtering equations for any polynomial state with polynomial multiplicative noise over linear observations is then established, which yields the explicit closed form of the filtering equations in the particular cases of linear and bilinear state equations. In an example, the performance of the designed optimal filter is verified against those of the optimal filter for a quadratic state with a state-independent noise and a conventional extended Kalman–Bucy filter. The authors thank the Mexican National Science and Technology Council (CONACyT) for financial support under Grants No. 55584 and 52953.     

Tracking of time-varying mobile radio channels .1. The Wiener LMSalgorithm

Adaptation algorithms with constant gains are designed for tracking smoothly time-varying parameters of linear regression models, in particular channel models occurring in mobile radio communications. In a companion paper, an application to channel tracking in the IS-136 TDMA system is discussed. The proposed algorithms are based on two key concepts. First, the design is transformed into a Wiener filtering problem. Second, the parameters are modeled as correlated ARIMA processes with known dynamics. This leads to a new framework for systematic and optimal design of simple adaptation laws based on prior information. The algorithms can be realized as Wiener filters, called learning filters, or as "LMS/Newton" updates complemented by filters that provide predictions or smoothing estimates. The simplest algorithm, named the Wiener LMS, is presented. All parameters are here assumed governed by the same dynamics and the covariance matrix of the regressors is assumed known. The computational complexity is of the same order of magnitude as that of LMS for regressors which are either white or have autoregressive statistics. The tracking performance is, however, substantially improved     

A delay-dependent approach to robust H/sub /spl infin// filtering for uncertain discrete-time state-delayed systems

A delay-dependent approach to robust H/sub /spl infin// filtering is proposed for linear discrete-time uncertain systems with multiple delays in the state. The uncertain parameters are supposed to reside in a polytope and the attention is focused on the design of robust filters guaranteeing a prescribed H/sub /spl infin// noise attenuation level. The proposed filter design methodology incorporates some recently appeared results, such as Moon's new version of the upper bound for the inner product of two vectors and de Oliveira's idea of parameter-dependent stability, which greatly reduce the overdesign introduced in the derivation process. In addition to the full-order filtering problem, the challenging reduced-order case is also addressed by using different linearization procedures. Both full- and reduced-order filters can be obtained from the solution of convex optimization problems in terms of linear matrix inequalities, which can be solved via efficient interior-point algorithms. Numerical examples have been presented to illustrate the feasibility and advantages of the proposed methodologies.     

Two-channel constrained least squares problems: solutions using power methods and connections with canonical coordinates

The problem of two-channel constrained least squares (CLS) filtering under various sets of constraints is considered, and a general set of solutions is derived. For each set of constraints, the solution is determined by a coupled (asymmetric) generalized eigenvalue problem. This eigenvalue problem establishes a connection between two-channel CLS filtering and transform methods for resolving channel measurements into canonical or half-canonical coordinates. Based on this connection, a unified framework for reduced-rank Wiener filtering is presented. Then, various representations of reduced-rank Wiener filters in canonical and half-canonical coordinates are introduced. An alternating power method is proposed to recursively compute the canonical coordinate and half-canonical coordinate mappings. A deflation process is introduced to extract the mappings associated with the dominant coordinates. The correctness of the alternating power method is demonstrated on a synthesized data set, and conclusions are drawn.     

多通道ARMA信号的三种多传感器信息融合Wiener滤波器

应用现代时间序列分析方法,基于ARMA新息模型、白噪声估值器和观测预报器,对带白色观测噪声的多通道ARMA信号,在线性最小方差最优信息融合准则下,提出了统一的和通用的按矩阵加权、按标量加权和按对角阵加权的多传感器信息融合Wiener滤波器,可统一处理滤波、平滑和预报问题．提出了计算局部估计误差方差和协方差的公式,它们被用于计算最优加权．同单传感器情形相比,可提高滤波精度．一个目标跟踪仿真例子说明了其有效性,且说明了三种加权融合滤波器的精度无显著差异,因而利用按标量加权融合滤波器以轻微的精度损失提供一种快速融合估计算法,便于实时应用．     

A digital signal processing approach to interpolation

In many digital signal precessing systems, e.g., vacoders, modulation systems, and digital waveform coding systems, it is necessary to alter the sampling rate of a digital signal Thus it is of considerable interest to examine the problem of interpolation of bandlimited signals from the viewpoint of digital signal processing. A frequency dmnain interpretation of the interpolation process, through which it is clear that interpolation is fundamentally a linear filtering process, is presented, An examination of the relative merits of finite duration impulse response (FIR) and infinite duration impulse response (IIR) digital filters as interpolation filters indicates that FIR filters are generally to be preferred for interpolation. It is shown that linear interpolation and classical polynomial interpolation correspond to the use of the FIR interpolation filter. The use of classical interpolation methods in signal processing applications is illustrated by a discussion of FIR interpolation filters derived from the Lagrange interpolation formula. The limitations of these filters lead us to a consideration of optimum FIR filters for interpolation that can be designed using linear programming techniques. Examples are presented to illustrate the significant improvements that are obtained using the optimum filters.     

Cyclic Wiener filtering: theory and method

Conventional time and space filtering of stationary random signals, which amounts to forming linear combinations of time translates and space translates, exploits the temporal and spatial coherence of the signals. By including frequency translates as well, the spectral coherence that is characteristic of cyclostationary signals can also be exploited. Some of the theoretical concepts underlying this generalized type of filtering, called frequency-shift (FRESH) filtering, are developed. The theory of optimum FRESH filtering, which is a generalization of Wiener filtering called cyclic Wiener filtering, is summarized, and the theory is illustrated with specific examples of separating temporally and spectrally overlapping communications signals, including AM, BPSK, and QPSK. The structures and performances of optimum FRESH filters are presented, and adaptive adjustment of the weights in these structures is discussed. Also, specific results on the number of digital QAM signals that can be separated, as a function of excess bandwidth, are obtained     

Iterative methods for image deblurring

The authors discuss the use of iterative restoration algorithms for the removal of linear blurs from photographic images that may also be assumed to be degraded by pointwise nonlinearities such as film saturation and additive noise. Iterative algorithms allow for the incorporation of various types of prior knowledge about the class of feasible solutions, can be used to remove nonstationary blurs, and are fairly robust with respect to errors in the approximation of the blurring operator. Special attention is given to the problem of convergence of the algorithms, and classical solutions such as inverse filters, Wiener filters, and constrained least-squares filters are shown to be limiting solutions of variations of the iterations. Regularization is introduced as a means for preventing the excessive noise magnification that is typically associated with ill-conditioned inverse problems such as the deblurring problem, and it is shown that noise effects can be minimized by terminating the algorithms after a finite number of iterations. The role and choice of constraints on the class of feasible solutions are also discussed     

H/sub 2/ optimal linear robust sampled-data filtering design using polynomial approach

A new frequency domain approach to robust multi-input-multi-output (MIMO) linear filter design for sampled-data systems is presented. The system and noise models are assumed to be represented by polynomial forms that are not perfectly known except that they belong to a certain set. The optimal design guarantees that the error variance is kept below an upper bound that is minimized for all admissible uncertainties. The design problem is cast in the context of H/sub 2/ via the polynomial matrix representation of systems with norm bounded unstructured uncertainties. The sampled-data mix of continuous and discrete time systems is handled by means of a lifting technique; however, it does not increase the dimensionality or alter the computational cost of the solution. The setup adopted allows dealing with several filtering problems. A simple deconvolution example illustrates the procedure.     

Noise reduction in recursive digital filters using high-order errorfeedback

The problem of solving the optimal (minimum-noise) error feedback coefficients for recursive digital filters is addressed in the general high-order case. It is shown that when minimum noise variance at the filter output is required, the optimization problem leads to set of familiar Wiener-Hopf or Yule-Walker equations, demonstrating that the optimal error feedback can be interpreted as a special case of Wiener filtering. As an alternative to the optimal solution, the formulas for suboptimal error feedback with symmetric or antisymmetric coefficients are derived. In addition, the design of error feedback using power-of-two coefficients is discussed. The efficiency of high order error feedback is examined by test implementations of the set of standard filters. It is concluded that error feedback is a very powerful and versatile method for cutting down the quantization noise in any classical infinite impulse response (IIR) filter implemented as a cascade of second-order direct form sections. The high-order schemes are attractive for use with high-order direct form sections     

Information fusion Wiener filter for the multisensor multichannel ARMA signals with time-delayed measurements

For the multisensor multichannel autoregressive moving average (ARMA) signals with time-delayed measurements, a measurement transformation approach is presented, which transforms the equivalent state space model with measurement delays into the state space model without measurement delays, and then using the Kalman filtering method, under the linear minimum variance optimal weighted fusion rules, three distributed optimal fusion Wiener filters weighted by matrices, diagonal matrices and scalars are presented, respectively, which can handle the fused filtering, prediction and smoothing problems. They are locally optimal and globally suboptimal. The accuracy of the fuser is higher than that of each local signal estimator. In order to compute the optimal weights, the formulae of computing the cross-covariances among local signal estimation errors are given. A Monte Carlo simulation example for the three-sensor target tracking system with time-delayed measurements shows their effectiveness.     

Split Wiener filtering with application in adaptive systems

This paper proposes a new structure for split transversal filtering and introduces the optimum split Wiener filter. The approach consists of combining the idea of split filtering with a linearly constrained optimization scheme. Furthermore, a continued split procedure, which leads to a multisplit filter structure, is considered. It is shown that the multisplit transform is not an input whitening transformation. Instead, it increases the diagonalization factor of the input signal correlation matrix without affecting its eigenvalue spread. A power normalized, time-varying step-size least mean square (LMS) algorithm, which exploits the nature of the transformed input correlation matrix, is proposed for updating the adaptive filter coefficients. The multisplit approach is extended to linear-phase adaptive filtering and linear prediction. The optimum symmetric and antisymmetric linear-phase Wiener filters are presented. Simulation results enable us to evaluate the performance of the multisplit LMS algorithm.     

Polynomial weighted median filtering

This paper extends weighted median (WM) filters to the class of polynomial weighted median (PWM) filters. Traditional polynomial filtering theory, based on linear combinations of polynomial terms, is able to approximate important classes of nonlinear systems. The linear combination of polynomial terms, however, yields poor performance in environments characterized by heavy tailed distributions. Weighted median filters, in contrast, are well known for their outlier suppression and detail preservation properties. The weighted median sample selection methodology is naturally extended to the polynomial sample case, yielding a filter structure that exploits the higher order statistics of the observed samples while simultaneously being robust to outliers. Moreover, the PWM filter class is well motivated by an analysis of cross and square term statistics. A presented probability density function analysis shows that these terms have heavier tails than the observed samples, indicating that robust combination methods should be utilized to avoid undue influence of outliers. Further analysis shows weighted median processing of polynomial terms is justified from a maximum likelihood perspective. The established PWM filter class is statistically analyzed through the determination of the filter output distribution and breakdown probability. Filter parameter optimization procedures are also presented. Finally, the effectiveness of PWM filters is demonstrated through simulations that include temporal, spectrum, and bispectrum analysis.     

Robust Filtering for Linear Time-Invariant Continuous Systems

The problem of robust filtering for linear time-invariant (LTI) continuous systems subject to parametric uncertainties is treated in this paper through transfer function and polynomial representations, and then in the state-space domain. The basic idea consists of introducing the gradient of the estimation error with respect to the uncertain parameters in the optimization scheme via a epsiv-contaminated model. The general solution to the problem is given in the transfer function representation while, in the polynomial framework, the causal estimator is obtained by means of a spectral factorization and a Diophantine equation. The state-space realization of the causal estimator is discussed. Examples show the ability of the proposed technique to provide a reliable estimation in presence of model uncertainty.     

A summary, by illustrations, of least-squares filters with constraints

Several methods of combining a number of time series into a single series are discussed. They are all individual filtering followed by summation and are somewhat like Wiener filtering in that a least-squares criterion is used to design the filter coefficients. They differ from Wiener filtering in that signal information is given in the form of various constraints on the filter coefficients rather than being given as a signal correlation function. The equations are worked out explicitly for the case of two time series and three filter points and presented in such a way as to make generalization clear.     

Joint state filtering and parameter estimation for linear stochastic time-delay systems

This paper presents the joint state filtering and parameter estimation problem for linear stochastic time-delay systems with unknown parameters. The original problem is reduced to the mean-square filtering problem for incompletely measured bilinear time-delay system states over linear observations. The unknown parameters are considered standard Wiener processes and incorporated as additional states in the extended state vector. To deal with the new filtering problem, the paper designs the mean-square finite-dimensional filter for incompletely measured bilinear time-delay system states over linear observations. A closed system of the filtering equations is then derived for a bilinear time-delay state over linear observations. Finally, the paper solves the original joint estimation problem. The obtained solution is based on the designed mean-square filter for incompletely measured bilinear time-delay states over linear observations, taking into account that the filter for the extended state vector also serves as the identifier for the unknown parameters. In the example, performance of the designed state filter and parameter identifier is verified for a linear time-delay system with an unknown multiplicative parameter over linear observations.