Comparison of RLS, LMS, and sign algorithms for tracking randomlytime-varying channels

The performance of adaptive FIR filters governed by the recursive least-squares (RLS) algorithm, the least mean square (LMS) algorithm, and the sign algorithm (SA), are compared when the optimal filtering vector is randomly time-varying. The comparison is done in terms of the steady-state excess mean-square estimation error ξ and the steady-state mean-square weight deviation, η. It is shown that ξ does not depend on the spread of eigenvalues of the input covariance matrix, R, in the cases of the LMS algorithm and the SA, while it does in the case of the RLS algorithm. In the three algorithms, η is found to be increasing with the eigenvalue spread. The value of the adaptation parameter that minimizes ξ is different from the one that minimizes η. It is shown that the minimum values of ξ and η attained by the RLS algorithm are equal to the ones attained by the LMS algorithm in any one of the three following cases: (1) if R has equal eigenvalues, (2) if the fluctuations of the individual elements of the optimal vector are mutually uncorrelated and have the same mean-square value, or (3) if R is diagonal and the fluctuations of the individual elements of the optimal vector have the same mean-square value. Conditions that make the values of ξ and η of the LMS algorithm smaller (or greater) than the ones of the RLS algorithm are derived. For Gaussian input data, the minimum values of ξ and η attained by the SA are found to exceed the ones attained by the LMS algorithm by 1 dB independently of R and the mutual correlation between the elements of the optimal vector     

ULV and generalized ULV subspace tracking adaptive algorithms

Traditional adaptive filters assume that the effective rank of the input signal is the same as the input covariance matrix or the filter length N. Therefore, if the input signal lives in a subspace of dimension less than N, these filters fail to perform satisfactorily. In this paper, we present two new algorithms for adapting only in the dominant signal subspace. The first of these is a low-rank recursive-least-squares (RLS) algorithm that uses a ULV decomposition (Stewart 1992) to track and adapt in the signal subspace. The second adaptive algorithm is a subspace tracking least-mean-squares (LMS) algorithm that uses a generalized ULV (GULV) decomposition, developed in this paper, to track and adapt in subspaces corresponding to several well-conditioned singular value clusters. The algorithm also has an improved convergence speed compared with that of the LMS algorithm. Bounds on the quality of subspaces isolated using the GULV decomposition are derived, and the performance of the adaptive algorithms are analyzed     

Comparative tracking performance of the LMS and RLS algorithms forchirped narrowband signal recovery

This paper studies the comparative tracking performance of the recursive least squares (RLS) and least mean square (LMS) algorithms for time-varying inputs, specifically for linearly chirped narrowband input signals in additive white Gaussian noise. It is shown that the structural differences in the implementation of the LMS and RLS weight updates produce regions where the LMS performance exceeds that of the RLS and other regions where the converse occurs. These regions are shown to be a function of the signal bandwidth and signal-to-noise ratio (SNR). LMS is shown to place a notch in the signal band of the mean lag filter, thus reducing the lag error and improving the tracking performance. For the chirped signal, it is shown that this produces smaller tracking error for small SNR. For high SNR, there is a region of signal bandwidth for which RLS will provide lower error than LMS, but even for these high SNR inputs, LMS always provides superior performance for very narrowband signals     

An Approximate Statistical Analysis of the Widrow LMS Algorithm with Application to Narrow-Band Interference Rejection

The Widrow LMS algorithm is considered for the implementation of an adaptive prewhitening filter in a direct-sequence (DS) spread-spectrum receiver. Exact expressions for the steady-state tapweight covariance matrix and resulting average excess mean square error are developed for the real LMS algorithm when the input contains a random binary sequence (used to model a pseudonoise spreading sequence). It is shown here that the output samples of the adaptive filter possess approximately Gaussian statistics under the conditions of slow convergence and a large number of filter taps. Using this approximation, expressions for the resulting bit error rate (BER) when the adaptive algorithm is used to suppress a fading gone jammer are developed, and numerical results obtained from these expressions are compared to simulation results for the DS receiver.     

Coherent LMS algorithms

Pilot symbol-assisted adaptive algorithms provide coherent detection for communication systems when the filtering coefficients, such as beamforming weights or equalizer coefficients, are converged. This property can be exploited to speed up the convergence of adaptive algorithms used. In this letter, two new adaptive algorithms, coherent least mean square (C-LMS) and coherent normalized LMS (CN-LMS), are proposed by constraining the desired signal component of the filtering output to be always coherent in phase with the reference signal at each iteration. For same adaption step size, these new algorithms provide faster convergence, while yield same steady-state excess mean-square error as compared with their standard LMS counterparts     

Line search algorithms for adaptive filtering

Line search algorithms for adaptive filtering that choose the convergence parameter so that the updated filter vector minimizes the sum of squared errors on a linear manifold are described. A shift invariant property of the sample covariance matrix is exploited to produce an adaptive filter stochastic line search algorithm for exponentially weighted adaptive equalization requiring 3N+5 multiplications and divisions per iteration. This algorithm is found to have better numerical stability than fast transversal filter algorithms for an application requiring steady-state tracking capability similar to that of least-mean square (LMS) algorithms. The algorithm is shown to have faster initial convergence than the LMS algorithm and a well-known variable step size algorithm having similar computational complexity in an adaptive equalization experiment     

一种约束稳定性最小均方噪声对消算法

研究了最小均方误差(LMS)算法、归一化的最小均方(NLMS)算法及变步长NLMS算法在自适应噪声干扰抵消器中的应用,针对目前这些算法在噪声对消器应用中的缺点,将约束稳定性最小均方(CS-LMS)算法应用到噪声处理中,并进一步结合变步长的思想提出来一种新的变步长CS-LMS算法。通过MATLAB进行仿真分析,结果证实提出的算法与其他算法相比,能很好地滤除掉噪声从而得到期望信号,明显的降低了稳态误差,并拥有好的收敛速度。     

Stochastic analysis of the LMS algorithm with a saturationnonlinearity following the adaptive filter output

This paper presents a statistical analysis of the least mean square (LMS) algorithm with a zero-memory scaled error function nonlinearity following the adaptive filter output. This structure models saturation effects in active noise and active vibration control systems when the acoustic transducers are driven by large amplitude signals. The problem is first defined as a nonlinear signal estimation problem and the mean-square error (MSE) performance surface is studied. Analytical expressions are obtained for the optimum weight vector and the minimum achievable MSE as functions of the saturation. These results are useful for adaptive algorithm design and evaluation. The LMS algorithm behavior with saturation is analyzed for Gaussian inputs and slow adaptation. Deterministic nonlinear recursions are obtained for the time-varying mean weight and MSE behavior. Simplified results are derived for white inputs and small step sizes. Monte Carlo simulations display excellent agreement with the theoretical predictions, even for relatively large step sizes. The new analytical results accurately predict the effect of saturation on the LMS adaptive filter behavior     

Statistical properties of the LMS fourier analyzer in the presence of frequency mismatch

The statistical performances of the conventional adaptive Fourier analyzers, such as the least mean square (LMS), the recursive least square (RLS) algorithms, and so forth, may degenerate significantly, if the signal frequencies given to the analyzers are different from the true signal frequencies. This difference is referred to as frequency mismatch (FM). We analyze extensively the performance of the conventional LMS Fourier analyzer in the presence of FM. Difference equations governing the dynamics and closed-form steady-state expression for the estimation mean square error (MSE) of the algorithm are derived in detail. It is revealed that the discrete Fourier coefficient (DFC) estimation problem in the LMS eventually reduces to a DFC tracking one due to the FM, and an additional term derived from DFC tracking appears in the closed-form MSE expression, which essentially deteriorates the performance of the algorithm. How to derive the optimum step size parameters that minimize or mitigate the influence of the FM is also presented, which can be used to perform robust design of step size parameters for the LMS algorithm in the presence of FM. Extensive simulations are conducted to reveal the validity of the analytical results.     

A New Normalized Subband Adaptive Filter Algorithm with Individual Variable Step Sizes

Proposed is a novel variable step size normalized subband adaptive filter algorithm, which assigns an individual step size for each subband by minimizing the mean square of the noise-free a posterior subband error. Furthermore, a noniterative shrinkage method is used to recover the noise-free priori subband error from the noisy subband error signal. Simulation results using the colored input signals have demonstrated that the proposed algorithm not only has better tracking capability than the existing subband adaptive filter algorithms, but also exhibits lower steady-state error.     

A variable step size LMS algorithm

A least-mean-square (LMS) adaptive filter with a variable step size is introduced. The step size increases or decreases as the mean-square error increases or decreases, allowing the adaptive filter to track changes in the system as well as produce a small steady state error. The convergence and steady-state behavior of the algorithm are analyzed. The results reduce to well-known results when specialized to the constant-step-size case. Simulation results are presented to support the analysis and to compare the performance of the algorithm with the usual LMS algorithm and another variable-step-size algorithm. They show that its performance compares favorably with these existing algorithms     

LMS与RLS算法仿真消噪对比研究

讨论了RLS（递归最小二乘）和LMS（最小均方）自适应滤波算法及原理,对两种算法进行了系统全面的分析,对比研究了各自的优势及不足,提出了两种算法在语音消噪仿真中的算法实现,对实际语音信号进行了仿真消噪,研究表明选用算法对语音消噪是明显有效的,RLS自适应消噪算法及LMS自适应噪声抵消算法具有很强的实际应用价值。     

Asymptotic analysis of stochastic gradient-based adaptive filteringalgorithms with general cost functions

This paper presents an analysis of stochastic gradient-based adaptive algorithms with general cost functions. The analysis holds under mild assumptions on the inputs and the cost function. The method of analysis is based on an asymptotic analysis of fixed stepsize adaptive algorithms and gives almost sure results regarding the behavior of the parameter estimates, whereas previous stochastic analyses typically considered mean and mean square behavior. The parameter estimates are shown to enter a small neighborhood about the optimum value and remain there for a finite length of time. Furthermore, almost sure exponential bounds are given for the rate of convergence of the parameter estimates. The asymptotic distribution of the parameter estimates is shown to be Gaussian with mean equal to the optimum value and covariance matrix that depends on the input statistics. Specific adaptive algorithms that fall under the framework of this paper are signed error least mean square (LMS), dual sign LMS, quantized state LMS, least mean fourth, dead zone algorithms, momentum algorithms, and leaky LMS     

Adaptive recovery of a chirped sinusoid in noise. II. Performanceof the LMS algorithm

For pt.I see ibid., vol.39, no. 3, p.583-94 (1991). The authors present a methodology for evaluating the tracking behavior of the least-mean square (LMS) algorithm for the nontrivial case of recovering a chirped sinusoid in additive noise. A complete closed-form analysis of the LMS tracking properties for a nonstationary inverse system modeling problem is also presented. The mean-square error (MSE) performance of the LMS algorithm is calculated as a function of the various system parameters. The misadjustment or residual of the adaptive filter output is the excess MSE as compared to the optimal filter for the problem. It is caused by three errors in the adaptive weight vector: the mean lag error between the (time-varying mean) weight and the time-varying optimal weight; the fluctuations of the lag error; and the noise misadjustment which is due to the output noise. These results are important because they represent a precise analysis of a nonstationary deterministic inverse modeling system problem with the input being a colored signal. The results are in agreement with the form of the upper bounds for the misadjustment provided by E. Eweda and O. Macchi (1985) for the deterministic nonstationarity     

自适应滤波器的修正的时域正交性算法及其应用

本文绘出了一个自适应滤波器的修正的时域正交性算法.计算机模拟结果表明,该算法有两个特性:一、相同的收敛因子c可以在比较宽范围的输入信号信噪比内使用。二、应用这种方法的噪声消除器的一个实用特性是,对于相同的期待信号,输入的参考信号的功率变化,不影响收敛因子c值的选择,这就使这种噪声消除器的应用更灵活。文中给出了这种算法用于谱估值时的特性,并与LMS算法进行了比较。     

Acquisition performance of an adaptive receiver for DS-CDMA

This paper presents a single-user code timing estimation algorithm for direct-sequence code-division multiple access that is based on processing the weight vector of an adaptive filter. The filter weight vector can be shown to adapt in the mean to a scaled time-shifted version of the spreading code of the desired user. Therefore, our algorithm requires very little side information in order to form its estimate. The acquisition performance of the algorithm is investigated when the filter is adapted using the least mean square (LMS) or the recursive least square (RLS) algorithm. The proposed algorithm is shown through experimental results to be resistant to the near-far problem when the RLS adaptation algorithm is used, but not when the LMS algorithm is used. However, the performance of this code-acquisition technique is still substantially better than the traditional correlator-based approach, even when the computationally simple LMS algorithm is used. As an extension to the basic timing estimator algorithm, we consider the effect of frequency synchronization error on the performance of the timing estimate. As expected, frequency-offset error degrades the performance of the timing estimate. However, a modified version of the adaptive filter is presented to combat this effect     

Adaptive joint beamforming and B‐MMSE detection for CDMA signal reception under multipath interference

The combination of antenna array beamforming with multiuser detection can effectively improve the detection efficiency of a wireless system under multipath interference, especially in a fast‐fading channel. This paper studies the performance of an adaptive beamformer incorporated with a block‐wise minimum mean square error(B‐MMSE) detector, which works on a unique signal frame characterized by training sequence preamble and data blocks segmented by zero‐bits. Both beam‐former weights updating and B‐MMSE detection are carried out by either least mean square (LMS) or recursive least square (RLS) algorithm. The comparison of the two adaptive algorithms applied to both beamformer and B‐MMSE detector will be made in terms of convergence behaviour and estimation mean square error. Various multipath patterns are considered to test the receiver's responding rapidity to changing multipath interference. The performance of the adaptive B‐MMSE detector is also compared with that of non‐adaptive version (i.e. through direct matrix inversion). The final performance in error probability simulation reveals that the RLS/B‐MMSE scheme outperforms non‐adaptive B‐MMSE by 1–5 dB, depending on the multipath channel delay profiles of concern. The obtained results also suggest that adaptive beamformer should use RLS algorithm for its fast and robust convergence property; while the B‐MMSE filter can choose either LMS or RLS algorithm depending on antenna array size, multipath severity and implementation complexity. Copyright © 2004 John Wiley & Sons, Ltd.     

Stochastic analysis of the filtered-X LMS algorithm in systems withnonlinear secondary paths

This paper presents a statistical analysis of the filtered-X LMS algorithm behavior when the secondary path (output of the adaptive filter) includes a nonlinear element. This system is of special interest for active acoustic noise and vibration control, where a saturation nonlinearity models the nonlinear distortion introduced by the power amplifiers and transducers. Deterministic nonlinear recursions are derived for Gaussian inputs for the transient mean weight, mean square error, and cross-covariance matrix of the adaptive weight vector at different times. The cross-covariance results provide improved steady-state predictions (as compared with previous results) for moderate to large step sizes. Monte Carlo simulations show excellent agreement with the behavior predicted by the theoretical models. The analytical and simulation results show that a small nonlinearity can have a significant impact on the adaptive filter behavior     

A fast recursive least squares adaptive second order Volterrafilter and its performance analysis

A fast, recursive least squares (RLS) adaptive nonlinear filter modeled using a second-order Volterra series expansion is presented. The structure uses the ideas of fast RLS multichannel filters, and has a computational complexity of O(N³) multiplications, where N-1 represents the memory span in number of samples of the nonlinear system model. A theoretical performance analysis of its steady-state behaviour in both stationary and nonstationary environments is presented. The analysis shows that, when the input is zero mean and Gaussian distributed, and the adaptive filter is operating in a stationary environment, the steady-state excess mean-squared error due to the coefficient noise vector is independent of the statistics of the input signal. The results of several simulation experiments show that the filter performs well in a variety of situations. The steady-state behaviour predicted by the analysis is in very good agreement with the experimental results     

On the convergence behavior of the LMS and the normalized LMSalgorithms

It is shown that the normalized least mean square (NLMS) algorithm is a potentially faster converging algorithm compared to the LMS algorithm where the design of the adaptive filter is based on the usually quite limited knowledge of its input signal statistics. A very simple model for the input signal vectors that greatly simplifies analysis of the convergence behavior of the LMS and NLMS algorithms is proposed. Using this model, answers can be obtained to questions for which no answers are currently available using other (perhaps more realistic) models. Examples are given to illustrate that even quantitatively, the answers obtained can be good approximations. It is emphasized that the convergence of the NLMS algorithm can be speeded up significantly by employing a time-varying step size. The optimal step-size sequence can be specified a priori for the case of a white input signal with arbitrary distribution