Consistent normalized least mean square filtering with noisy data matrix

When the ordinary least squares method is applied to the parameter estimation problem with noisy data matrix, it is well-known that the estimates turn out to be biased. While this bias term can be somewhat reduced by the use of models of higher order, or by requiring a high signal-to-noise ratio (SNR), it can never be completely removed. Consistent estimates can be obtained by means of the instrumental variable method (IVM),or the total/data least squares method (TLS/DLS). In the adaptive setting for the such problem, a variety of least-mean-squares (LMS)-type algorithms have been researched rather than their recursive versions of IVM or TLS/DLS that cost considerable computations. Motivated by these observations, we propose a consistent LMS-type algorithm for the data least square estimation problem. This novel approach is based on the geometry of the mean squared error (MSE) function, rendering the step-size normalization and the heuristic filtered estimation of the noise variance, respectively, for fast convergence and robustness to stochastic noise. Monte Carlo simulations of a zero-forcing adaptive finite-impulse-response (FIR) channel equalizer demonstrate the efficacy of our algorithm.     

Modified LMS algorithm for unbiased impulse response estimation innonstationary noise

In the presence of input interference, the Wiener solution for impulse response estimation is biased. It is proved that bias removal can be achieved by proper scaling of the optimal filter coefficients and a modified least mean squares (LMS) algorithm is then developed for accurate system identification in noise. Simulation results show that the proposed method outperforms two total least squares (TLS) based adaptive algorithms under nonstationary interference conditions     

Adaptive reduced-rank LCMV beamforming algorithms based on joint iterative optimization of filters: Design and analysis

This paper presents reduced-rank linearly constrained minimum variance (LCMV) beamforming algorithms based on joint iterative optimization of filters. The proposed reduced-rank scheme is based on a constrained joint iterative optimization of filters according to the minimum variance criterion. The proposed optimization procedure adjusts the parameters of a projection matrix and an adaptive reduced-rank filter that operates at the output of the bank of filters. We describe LCMV expressions for the design of the projection matrix and the reduced-rank filter. We then describe stochastic gradient and develop recursive least-squares adaptive algorithms for their efficient implementation along with automatic rank selection techniques. An analysis of the stability and the convergence properties of the proposed algorithms is presented and semi-analytical expressions are derived for predicting their mean squared error (MSE) performance. Simulations for a beamforming application show that the proposed scheme and algorithms outperform in convergence and tracking the existing full-rank and reduced-rank algorithms while requiring comparable complexity.     

自适应加权FIR-Myriad混合滤波算法

针对FIR滤波器滤除脉冲噪声以及加权Myriad滤波器滤除高斯噪声的不足,提出基于FIR滤波器和加权WMy滤波器有效组合的一类新的非线性滤波器FIR-WMyH滤波器.利用神经网络中的反向传播算法,在均方误差准则下,推导了一个基于统计梯度的自适应算法.基于稳定α分布脉冲噪声模型下的仿真结果说明了该算法的良好的性能.     

Optimal Regularization Parameter of the Multichannel Filtered- x Affine Projection Algorithm

We discuss the optimal regularization parameter of the Filtered- Affine Projection (FX-AP) algorithm suitable for feedforward active noise control. While the original FX-AP algorithm always provides a biased estimate of the minimum-mean-square solution, we show that the optimal regularized FX-AP algorithm is capable to eliminate the bias of the asymptotic solution and thus that the regularization parameter can optimize both the convergence speed and the residual MSE of the algorithm. We derive some expressions for the optimal regularization parameter, and we discuss some heuristic estimations of the optimal regularization parameter in practical conditions.     

An efficient recursive total least squares algorithm for FIRadaptive filtering

An algorithm for recursively computing the total least squares (TLS) solution to the adaptive filtering problem is described. This algorithm requires O(N) multiplications per iteration to effectively track the N-dimensional eigenvector associated with the minimum eigenvalue of an augmented sample covariance matrix. It is shown that the recursive least squares (RLS) algorithm generates biased adaptive filter coefficients when the filter input vector contains additive noise. The TLS solution on the other hand, is seen to produce unbiased solutions. Examples of standard adaptive filtering applications that result in noise being added to the adaptive filter input vector are cited. Computer simulations comparing the relative performance of RLS and recursive TLS are described     

A Novel Adaptive Antenna Array for DS/CDMA Code Acquisition

Equipped with an adaptive beamformer, existing adaptive array code acquisition still relies on the correlator structure. Due to the inherent property of the associated serial-search scheme, its mean acquisition time is large, especially in strong interference environments. In this paper, we propose a novel adaptive filtering scheme to solve the problem. The proposed scheme comprises two adaptive filters, an adaptive spatial and an adaptive temporal filter. With a specially designed structure, the spatial filter can act as a beamformer suppressing interference, while the temporal filter can act as a code-delay estimator. A mean squared error (MSE) criterion is proposed such that these filters can be simultaneously adjusted by a stochastic gradient descent method. The performance as well as the convergence behavior of the proposed algorithm are analyzed in detail. Closed-form expressions for optimum filter weights, optimum beamformer signal-to-interference-plus-noise ratio (SINR), steady-state MSE, and mean acquisition time are derived for the additive white Gaussian noise (AWGN) channel. Computer simulations show that the mean acquisition time of the proposed algorithm is much shorter than that of the correlator-based approach, and the derived theoretical expressions are accurate.     

Array interpolation and DOA MSE reduction

Interpolation or mapping of data from a given real array to data from a virtual array of more suitable geometry is well known in array signal processing. This operation allows arrays of any geometry to be used with fast direction-of-arrival (DOA) estimators designed for linear arrays. In an earlier companion paper , a first-order condition for zero DOA bias under such mapping was derived and was also used to construct a design algorithm for the mapping matrix that minimized the DOA estimate bias. This bias-minimizing theory is now extended to minimize not only bias, but also to consider finite sample effects due to noise and reduce the DOA mean-square error (MSE). An analytical first-order expression for mapped DOA MSE is derived, and a design algorithm for the transformation matrix that minimizes this MSE is proposed. Generally, DOA MSE is not reduced by minimizing the size of the mapping errors but instead by rotating these errors and the associated noise subspace into optimal directions relative to a certain gradient of the DOA estimator criterion function. The analytical MSE expression and the design algorithm are supported by simulations that show not only conspicuous MSE improvements in relevant scenarios, but also a more robust preprocessing for low signal-to-noise ratios (SNRs) as compared with the pure bias-minimizing design developed in the previous paper.     

A recursive least M-estimate algorithm for robust adaptive filtering in impulsive noise: fast algorithm and convergence performance analysis

This paper studies the problem of robust adaptive filtering in impulsive noise environment using a recursive least M-estimate algorithm (RLM). The RLM algorithm minimizes a robust M-estimator-based cost function instead of the conventional mean square error function (MSE). Previous work has showed that the RLM algorithm offers improved robustness to impulses over conventional recursive least squares (RLS) algorithm. In this paper, the mean and mean square convergence behaviors of the RLM algorithm under the contaminated Gaussian impulsive noise model is analyzed. A lattice structure-based fast RLM algorithm, called the Huber Prior Error Feedback-Least Squares Lattice (H-PEF-LSL) algorithm is derived. Part of the H-PEF-LSL algorithm was presented in ICASSP 2001. It has an order O(N) arithmetic complexity, where N is the length of the adaptive filter, and can be viewed as a fast implementation of the RLM algorithm based on the modified Huber M-estimate function and the conventional PEF-LSL adaptive filtering algorithm. Simulation results show that the transversal RLM and the H-PEF-LSL algorithms have better performance than the conventional RLS and other RLS-like robust adaptive algorithms tested when the desired and input signals are corrupted by impulsive noise. Furthermore, the theoretical and simulation results on the convergence behaviors agree very well with each other.     

The extended least squares criterion: minimization algorithms andapplications

The least squares (LS) estimation criterion on one hand, and the total LS (TLS), constrained TLS (CTLS) and structured TLS (STLS) criteria on the other hand, can be viewed as opposite limiting cases of a more general criterion, which we term “extended LS” (XLS). The XLS criterion distinguishes measurement errors from modeling errors by properly weighting and balancing the two error sources. In the context of certain models (termed “pseudo-linear”), we derive two iterative algorithms for minimizing the XLS criterion: One is a straightforward “alternating coordinates” minimization, and the other is an extension of an existing CTLS algorithm. The algorithms exhibit different tradeoffs between convergence rate, computational load, and accuracy. The XLS criterion can be applied to popular estimation problems, such as identifying an autoregressive (AR) with exogenous noise (ARX) system from noisy input/output measurements or estimating the parameters of an AR process from noisy measurements. We demonstrate the convergence properties and performance of the algorithms with examples of the latter     

Universal FIR MMSE Filtering

We consider the problem of causal estimation, i.e., filtering, of a real-valued signal corrupted by zero mean, time-independent, real-valued additive noise, under the mean-squared error (MSE) criterion. We build a universal filter whose per-symbol squared error, for every bounded underlying signal, is essentially as small as that of the best finite-duration impulse response (FIR) filter of a given order. We do not assume a stochastic mechanism generating the underlying signal, and assume only that the variance of the noise is known to the filter. The regret of the expected MSE of our scheme is shown to decay as $O(log n/n)$, where $n$ is the length of the signal. Moreover, we present a stronger concentration result which guarantees the performance of our scheme not only in expectation, but also with high probability. Our result implies a conventional stochastic setting result, i.e., when the underlying signal is a stationary process, our filter achieves the performance of the optimal FIR filter. We back our theoretical findings with several experiments showcasing the potential merits of our universal filter in practice. Our analysis combines tools from the problems of universal filtering and competitive on-line regression.      

偏差补偿符号子带自适应滤波器

自适应滤波器在系统辨识、回声消除、信道均衡等领域获得了广泛应用.符号子带自适应滤波器（Sign Subband Adaptive Filter,SSAF）具有较强的抗脉冲干扰能力,但当输入信号受到噪声干扰时,其对未知系统系数向量的估计会产生偏差.为了解决上述问题,本文基于无偏估计准则,提出了一种偏差补偿符号子带自适应滤波器（Bias-Compensated Sign Subband Adaptive Filter,BC-SSAF）.为了解决定步长自适应滤波器需要在收敛速度和稳态失调之间进行折中的问题,本文采用随机梯度法来更新正则化参数,提出了变正则化参数偏差补偿符号子带自适应滤波器（Variable Regularization Bias-Compensated Sign Subband Adaptive Filter,VR-BC-SSAF）.仿真结果验证了BC-SSAF和VR-BC-SSAF性能的优越性.     

Stochastic gradient adaptation under general error criteria

Examines a family of adaptive filter algorithms of the form W_k+1=W_k+μf(d_k-W_k^t X_k)X_k in which f(·) is a memoryless odd-symmetric nonlinearity acting upon the error. Such algorithms are a generalization of the least-mean-square (LMS) adaptive filtering algorithm for even-symmetric error criteria. For this algorithm family, the authors derive general expressions for the mean and mean-square convergence of the filter coefficients For both arbitrary stochastic input data and Gaussian input data. They then provide methods for optimizing the nonlinearity to minimize the algorithm misadjustment for a given convergence rate. Using the calculus of variations, it is shown that the optimum nonlinearity to minimize misadjustment near convergence under slow adaptation conditions is independent of the statistics of the input data and can be expressed as -p'(x)/p(x), where p(x) is the probability density function of the uncorrelated plant noise. For faster adaptation under the white Gaussian input and noise assumptions, the nonlinearity is shown to be x/{1+μλx²/σ_k ²}, where λ is the input signal power and σ_k² is the conditional error power. Thus, the optimum stochastic gradient error criterion for Gaussian noise is not mean-square. It is shown that the equations governing the convergence of the nonlinear algorithm are exactly those which describe the behavior of the optimum scalar data nonlinear adaptive algorithm for white Gaussian input. Simulations verify the results for a host of noise interferences and indicate the improvement using non-mean-square error criteria     

H∞ optimality of the LMS algorithm

We show that the celebrated least-mean squares (LMS) adaptive algorithm is H^∞ optimal. The LMS algorithm has been long regarded as an approximate solution to either a stochastic or a deterministic least-squares problem, and it essentially amounts to updating the weight vector estimates along the direction of the instantaneous gradient of a quadratic cost function. We show that the LMS can be regarded as the exact solution to a minimization problem in its own right. Namely, we establish that it is a minimax filter: it minimizes the maximum energy gain from the disturbances to the predicted errors, whereas the closely related so-called normalized LMS algorithm minimizes the maximum energy gain from the disturbances to the filtered errors. Moreover, since these algorithms are central H^∞ filters, they minimize a certain exponential cost function and are thus also risk-sensitive optimal. We discuss the various implications of these results and show how they provide theoretical justification for the widely observed excellent robustness properties of the LMS filter     

Entropy minimization for supervised digital communications channelequalization

This paper investigates the application of error-entropy minimization algorithms to digital communications channel equalization. The pdf of the error between the training sequence and the output of the equalizer is estimated using the Parzen windowing method with a Gaussian kernel, and then, the Renyi's quadratic entropy is minimized using a gradient descent algorithm. By estimating Renyi's entropy over a short sliding window, an online training algorithm is also introduced. Moreover, for a linear equalizer, an orthogonality condition for the minimum entropy solution that leads to an alternative fixed-point iterative minimization method is derived. The performance of linear and nonlinear equalizers trained with entropy and mean square error (MSE) is compared. As expected, the results of training a linear equalizer are very similar for both criteria since, even if the input noise is non-Gaussian, the output filtered noise tends to be Gaussian. On the other hand, for nonlinear channels and using a multilayer perceptron (MLP) as the equalizer, differences between both criteria appear. Specifically, it is shown that the additional information used by the entropy criterion yields a faster convergence in comparison with the MSE     

Convergence of exponentiated gradient algorithms

This paper studies three related algorithms: the (traditional) gradient descent (GD) algorithm, the exponentiated gradient algorithm with positive and negative weights (EG± algorithm), and the exponentiated gradient algorithm with unnormalized positive and negative weights (EGU± algorithm). These algorithms have been previously analyzed using the “mistake-bound framework” in the computational learning theory community. We perform a traditional signal processing analysis in terms of the mean square error. A relationship between the learning rate and the mean squared error (MSE) of predictions is found for the family of algorithms. This is used to compare the performance of the algorithms by choosing learning rates such that they converge to the same steady-state MSE. We demonstrate that if the target weight vector is sparse, the EG± algorithm typically converges more quickly than the GD or EGU± algorithms that perform very similarly. A side effect of our analysis is a reparametrization of the algorithms that provides insights into their behavior. The general form of the results we obtain are consistent with those obtained in the mistake-bound framework. The application of the algorithms to acoustic echo cancellation is then studied, and it is shown in some circumstances that the EG± algorithm will converge faster than the other two algorithms     

A blind adaptive decorrelating detector for CDMA systems

The decorrelating detector is known to eliminate multiaccess interference when the signature sequences of the users are linearly independent, at the cost of enhancing the Gaussian receiver noise. We present a blind adaptive decorrelating detector which is based on the observation of readily available statistics. The algorithm recursively updates the filter coefficients of a desired user by using the output of the current filter. Due to the randomness of the information bits transmitted and the ambient Gaussian channel noise, the filter coefficients evolve stochastically. We prove the convergence of the filter coefficients to a decorrelating detector in the mean squared error (MSE) sense. We develop lower and upper bounds on the MSE of the receiver filter from the convergence point and show that with a fixed step size sequence, the MSE can be made arbitrarily small by choosing a small enough step size. With a time-varying step size sequence, the MSE converges to zero implying an exact convergence. The proposed algorithm is distributed, in the sense that no information about the interfering users such as their signature sequences or power levels is needed. The algorithm requires the knowledge of only two parameters for the construction of the receiver filter of a desired user: the desired user's signature sequence and the variance of the additive white Gaussian (AWG) receiver noise. This detector, for an asynchronous code division multiple access (CDMA) channel, converges to the one-shot decorrelating detector     

Adaptive blind equalization using second- and higher orderstatistics

This paper presents two classes of adaptive blind algorithms based on second- and higher order statistics. The first class contains fast recursive algorithms whose cost functions involve second and third- or fourth-order cumulants. These algorithms are stochastic gradient-based but have structures similar to the fast transversal filters (FTF) algorithms. The second class is composed of two stages: the first stage uses a gradient adaptive lattice (GAL) while the second stage employs a higher order-cumulant (HOC) based least mean squares (LMS) filter. The computational loads for these algorithms are all linearly proportional to the number of taps used. Furthermore, the second class, as various numerical examples indicate, yields very fast convergence rates and low steady state mean square errors (MSE) and intersymbol interference (ISI). MSE convergence analyses for the proposed algorithms are also provided and compared with simulation results     

Stochastic power control for cellular radio systems

For wireless communication systems, iterative power control algorithms have been proposed to minimize the transmitter power while maintaining reliable communication between mobiles and base stations. To derive deterministic convergence results, these algorithms require perfect measurements of one or more of the following parameters: (1) the mobile's signal-to-interference ratio (SIR) at the receiver; (2) the interference experienced by the mobile; and (3) the bit-error rate. However, these quantities are often difficult to measure and deterministic convergence results neglect the effect of stochastic measurements. We develop distributed iterative power control algorithms that use readily available measurements. Two classes of power control algorithms are proposed. Since the measurements are random, the proposed algorithms evolve stochastically and we define the convergence in terms of the mean-squared error (MSE) of the power vector from the optimal power vector that is the solution of a feasible deterministic power control problem. For the first class of power control algorithms using fixed step size sequences, we obtain finite lower and upper bounds for the MSE by appropriate selection of the step size. We also show that these bounds go to zero, implying convergence in the MSE sense, as the step size goes to zero. For the second class of power control algorithms, which are based on the stochastic approximations method and use time-varying step size sequences, we prove that the MSE goes to zero. Both classes of algorithms are distributed in the sense that each user needs only to know its own channel gain to its assigned base station and its own matched filter output at its assigned base station to update its power     

一种抗冲击噪声的对数总体最小二乘自适应滤波算法

在未知系统输入信号和输出信号均含有噪声的环境中,传统的自适应滤波算法,如最小均方(LMS)算法,会产生有偏估计.总体最小二乘(TLS)算法能够同时最小化输入信号与输出信号的噪声干扰,是解决此类问题的重要方法.然而,在许多实际应用中,干扰噪声可能具有冲击特性,这使得传统基于2阶统计量的自适应滤波算法,包括总体最小二乘算法...