Analysis of LMS and NLMS algorithms with delayed coefficient updateunder the presence of spherically invariant processes

Certain conditions require a delay in the coefficient update of the least mean square (LMS) and normalized least mean square (NLMS) algorithms. This paper presents an in-depth analysis of these modificated versions for the important case of spherically invariant random processes (SIRPs), which are known as an excellent model for speech signals. Some derived bounds and the predicted dynamic behavior of the algorithms are found to correspond very well to simulation results and a real time implementation on a fixed-point signal processor. A modification of the algorithm is proposed to assure the well known properties of the LMS and NLMS algorithms     

Use of adaptive RLS, LMS, and NLMS algorithms for nonlinearity modeling in a modified laser interferometer

Laser heterodyne interferometer is one kind of nano-metrology systems which has been widely used in industry for high-accuracy displacement measurements. The accuracy of the nano-metrology systems based on the laser heterodyne interferometers can be effectively limited by the periodic nonlinearity. In this paper, we present the nonlinearity modeling of the nano-metrology interferometric system using some adaptive filters. The adaptive algorithms consist of the least mean squares (LMS), normalized least mean squares (NLMS), and recursive least squares (RLS). It is shown that the RLS algorithm can obtain optimal modeling parameters of nonlinearity.     

A set of algorithms linking NLMS and block RLS algorithms

This paper describes a set of block processing algorithms which contains as extremal cases the normalized least mean squares (NLMS) and the block recursive least squares (BRLS) algorithms. All these algorithms use small block lengths, thus allowing easy implementation and small input-output delay. It is shown that these algorithms require a lower number of arithmetic operations than the classical least mean squares (LMS) algorithm, while converging much faster. A precise evaluation of the arithmetic complexity is provided, and the adaptive behavior of the algorithm is analyzed. Simulations illustrate that the tracking characteristics of the new algorithm are also improved compared to those of the NLMS algorithm. The conclusions of the theoretical analysis are checked by simulations, illustrating that, even in the case where noise is added to the reference signal, the proposed algorithm allows altogether a faster convergence and a lower residual error than the NLMS algorithm. Finally, a sample-by-sample version of this algorithm is outlined, which is the link between the NLMS and recursive least squares (RLS) algorithms     

The stability of variable step-size LMS algorithms

Variable step-site LMS (VSLMS) algorithms are a popular approach to adaptive filtering, which can provide improved performance while maintaining the simplicity and robustness of conventional fixed step-size LMS. Here, we examine the stability of VSLMS with uncorrelated stationary Gaussian data. Most VSLMS described in the literature use a data-dependent step-size, where the step-size either depends on the data before the current time (prior step-size rule) or through the current time (posterior step-size rule). It has often been assumed that VSLMS algorithms are stable (in the sense of mean-square bounded weights), provided that the step-size is constrained to lie within the corresponding stability region for the LMS algorithm. For a single tap fitter, we find exact expressions for the stability region of VSLMS over the classes of prior and posterior step-sizes and show that the stability region for prior step size coincides with that of fixed step-size, but the region for posterior step-size is strictly smaller than for fixed step-size. For the multiple tap case, we obtain bounds on the stability regions with similar properties. The approach taken here is a generalization of the classical method of analyzing, the exponential stability of the weight covariance equation for LMS. Although it is not possible to derive a weight covariance equation for general data-dependent VSLMS, the weight variances can be upper bounded by the solution of a linear time-invariant difference equation, after appropriately dealing with certain nonlinear terms. For prior step-size (like fixed step-size), the state matrix is symmetric, whereas for posterior step-size, the symmetry is lost, requiring a more detailed analysis. The results are verified by computer simulations     

Conditioning of LMS algorithms with fast sampling

The LMS algorithm is very commonly used in signal processing. Its convergence properties depend primarily on the step size chosen and the condition number of an information matrix associated with the system. In most applications today, the LMS uses a regression vector based on the shift operator (including the ubiquitous tapped delay line). We demonstrate that generically, when fast sampling is employed, these regression vectors lead to poorly conditioned LMS. By comparison, delta operator based regression vectors lend with rapid sampling to improved condition numbers, hence, to better performance     

A class of order statistic LMS algorithms

Conventional gradient-based adaptive filters, as typified by the well-known LMS algorithm, use an instantaneous estimate of the error-surface gradient to update the filter coefficients. Such a strategy leaves the algorithm extremely vulnerable to impulsive interference. A class of adaptive algorithms employing order statistic filtering of the sampled gradient estimates is presented. These algorithms, dubbed order statistic least mean squares (OSLMS), are designed to facilitate adaptive filter performance close to the least squares optimum across a wide range of input environments from Gaussian to highly impulsive. Three specific OSLMS filters are defined: the median LMS, the average LMS, and the trimmed-mean LMS. The properties of these algorithms are investigated and the potential for improvement demonstrated. Finally, a general adaptive OSLMS scheme in which the nature of the order-statistic operator is also adapted in response to the statistics of the input signal is presented. It is shown that this can facilitate performance gains over a wide range of input data types     

Second-order H∞ optimal LMS and NLMS algorithmsbased on a second-order Markov model

It is shown that two algorithms obtained by simplifying a Kalman filter considered for a second-order Markov model are H^∞ suboptimal. Similar to least mean squares (LMS) and normalised LMS (NLMS) algorithms, these second order algorithms can be thought of as approximate solutions to stochastic or deterministic least squares minimisation. It is proved that second-order LMS and NLMS are exact solutions causing the maximum energy gain from the disturbances to the predicted and filtered errors to be less than one, respectively. These algorithms are implemented in two steps. Operation of the first step is like conventional LMS/NLMS algorithms and the second step consists of the estimation of the weight increment vector and prediction of weights for the next iteration. This step applies simple smoothing on the increment of the estimated weights to estimate the speed of the weights. Also they are cost-effective, robust and attractive for improving the tracking performance of smoothly time-varying models     

基于LMS及RLS的自适应均衡算法仿真分析

在通信系统中采用均衡技术是改善信道特性行之有效的方法,为此从时域均衡原理出发,讨论了基于LMS和基于RLS的自适应均衡算法,并利用Matlab对两类算法进行了仿真,从均衡前后信号的星座图、算法收敛特性以及均衡前后系统的误码特性这三个方面对两类算法的性能进行了比较.     

Mean weight behavior of the filtered-X LMS algorithm

A new stochastic analysis is presented for the filtered-X LMS (FXLMS) algorithm. The analysis does not use independence theory. An analytical model is derived for the mean behavior of the adaptive weights. The model is valid for white or colored reference inputs and accurately predicts the mean weight behavior even for large step sizes. The constrained Wiener solution is determined as a function of the input statistics and the impulse responses of the adaptation loop filters. Effects of secondary path estimation error are studied. Monte Carlo simulations demonstrate the accuracy of the theoretical model     

A new class of gradient adaptive step-size LMS algorithms

The gradient adaptive step-size least-mean-square (LMS) algorithms [an important family of variable step-size LMS (VSLMS) algorithms] are revisited. We propose a simplification to a class of the studied algorithms and show that this leads to a new class of VSLMS algorithms with reduced complexity but with no observable loss in performance     

On the convergence behavior of the LMS and the normalized LMSalgorithms

It is shown that the normalized least mean square (NLMS) algorithm is a potentially faster converging algorithm compared to the LMS algorithm where the design of the adaptive filter is based on the usually quite limited knowledge of its input signal statistics. A very simple model for the input signal vectors that greatly simplifies analysis of the convergence behavior of the LMS and NLMS algorithms is proposed. Using this model, answers can be obtained to questions for which no answers are currently available using other (perhaps more realistic) models. Examples are given to illustrate that even quantitatively, the answers obtained can be good approximations. It is emphasized that the convergence of the NLMS algorithm can be speeded up significantly by employing a time-varying step size. The optimal step-size sequence can be specified a priori for the case of a white input signal with arbitrary distribution     

Comparative tracking performance of the LMS and RLS algorithms forchirped narrowband signal recovery

This paper studies the comparative tracking performance of the recursive least squares (RLS) and least mean square (LMS) algorithms for time-varying inputs, specifically for linearly chirped narrowband input signals in additive white Gaussian noise. It is shown that the structural differences in the implementation of the LMS and RLS weight updates produce regions where the LMS performance exceeds that of the RLS and other regions where the converse occurs. These regions are shown to be a function of the signal bandwidth and signal-to-noise ratio (SNR). LMS is shown to place a notch in the signal band of the mean lag filter, thus reducing the lag error and improving the tracking performance. For the chirped signal, it is shown that this produces smaller tracking error for small SNR. For high SNR, there is a region of signal bandwidth for which RLS will provide lower error than LMS, but even for these high SNR inputs, LMS always provides superior performance for very narrowband signals     

Statistical properties of the LMS fourier analyzer in the presence of frequency mismatch

The statistical performances of the conventional adaptive Fourier analyzers, such as the least mean square (LMS), the recursive least square (RLS) algorithms, and so forth, may degenerate significantly, if the signal frequencies given to the analyzers are different from the true signal frequencies. This difference is referred to as frequency mismatch (FM). We analyze extensively the performance of the conventional LMS Fourier analyzer in the presence of FM. Difference equations governing the dynamics and closed-form steady-state expression for the estimation mean square error (MSE) of the algorithm are derived in detail. It is revealed that the discrete Fourier coefficient (DFC) estimation problem in the LMS eventually reduces to a DFC tracking one due to the FM, and an additional term derived from DFC tracking appears in the closed-form MSE expression, which essentially deteriorates the performance of the algorithm. How to derive the optimum step size parameters that minimize or mitigate the influence of the FM is also presented, which can be used to perform robust design of step size parameters for the LMS algorithm in the presence of FM. Extensive simulations are conducted to reveal the validity of the analytical results.     

Comparison of RLS, LMS, and sign algorithms for tracking randomlytime-varying channels

The performance of adaptive FIR filters governed by the recursive least-squares (RLS) algorithm, the least mean square (LMS) algorithm, and the sign algorithm (SA), are compared when the optimal filtering vector is randomly time-varying. The comparison is done in terms of the steady-state excess mean-square estimation error ξ and the steady-state mean-square weight deviation, η. It is shown that ξ does not depend on the spread of eigenvalues of the input covariance matrix, R, in the cases of the LMS algorithm and the SA, while it does in the case of the RLS algorithm. In the three algorithms, η is found to be increasing with the eigenvalue spread. The value of the adaptation parameter that minimizes ξ is different from the one that minimizes η. It is shown that the minimum values of ξ and η attained by the RLS algorithm are equal to the ones attained by the LMS algorithm in any one of the three following cases: (1) if R has equal eigenvalues, (2) if the fluctuations of the individual elements of the optimal vector are mutually uncorrelated and have the same mean-square value, or (3) if R is diagonal and the fluctuations of the individual elements of the optimal vector have the same mean-square value. Conditions that make the values of ξ and η of the LMS algorithm smaller (or greater) than the ones of the RLS algorithm are derived. For Gaussian input data, the minimum values of ξ and η attained by the SA are found to exceed the ones attained by the LMS algorithm by 1 dB independently of R and the mutual correlation between the elements of the optimal vector     

A framework for analyzing and designing scale invariant remotesensing algorithms

The land surface exhibits heterogeneity across a range of spatial scales. Remote sensors provide integrated information at the pixel scale, however, there is important spatial variability at scales smaller than the scale of the sensor. On the other hand, large scale models that use remotely sensed data do not require them at the same spatial resolution at which remote sensors are required to operate. In this paper, a framework for testing aggregation-disaggregation properties of remote sensing algorithms is presented. The proposed framework provides a systematic approach for parameterizing the land surface heterogeneity effects. For the estimation of the pixel scale response, the lumped response should be modified by the variance and covariance terms. This representation of land surface heterogeneity could lead to substantial savings in remote sensing data storage and management. Using simulated land and vegetation scenarios, the authors have successfully parameterized subpixel scale heterogeneity effects for the estimation of vegetation index, by modeling the variances and covariance terms with the pixel scale values     

Analysis of LMS algorithm with inputs from cyclostationary random processes

The least mean square (LMS) algorithm is investigated for inputs from nonstationary random processes having periodic (period P) statistics. By defining P different coefficient vectors, each vector can be shown to converge in the mean' to a biased solution which is dependent upon the algorithm step size mu .<>     

LMS算法的二次稳定性及鲁棒LMS算法

本文在时域内研究LMS算法(least mean square algorithm)的稳定性及鲁棒LMS算法的构造.首先将LMS算法表达式转化为标准的离散时间系统状态方程形式,之后运用线性矩阵不等式(LMI)技术对其二次稳定性进行了分析.针对滤波过程中会出现的输入和测量噪声干扰,本文提出了一种兼顾收敛性、鲁棒稳定性以及鲁棒性能的鲁棒LMS算法,最后给出了仿真算例,通过和一般的LMS算法的比较,体现了这种鲁棒LMS算法的优越性.     

改进遗传算法在人群行为仿真中的应用

针对目前人群行为仿真中多目标,有约束的组合优化问题,提出了一种改进遗传算法在人群行为仿真的应用方案。通过设定特殊的数据结构、仿真过程中的各种约束规则、遗传算法中的基因编码、适应度评价函数实现了人群行为仿真。仿真实验验证了该算法可以大大减少搜索空间,并能使结果达到最优。     

The stability of LMS

New and weak conditions are given under which the LMS algorithm is exponentially convergent with probability one in a stochastic setting. These results show that LMS works under very broad conditions: a small gain, input signal with finite fourth moments; time varying persistence of excitation; and weak assumptions on the correlation structure of the input signal. Previous results at this level of generality linked convergence to peak signal amplitude rather than average amplitude. Under the same conditions stochastic boundedness of a forced LMS system is also established     

The behavior of a Costas loop in the presence of space telemetrysignals

When the Costas loop is operated in the presence of a residual carrier space telemetry signal, the loop reconstructs its carrier reference from an input signal whose carrier component is not completely suppressed. The telemetry signal investigated in this work is generated by phase shift-keying the data onto a subcarrier and then phase modulating onto the sinusoidal carrier. The telemetry modulation index, telemetry bit rate, subcarrier waveform, and subcarrier frequency are shown to be the key system parameters that contribute to the performance degradation of a Costas loop. Furthermore, the effect of Doppler shift on the loop is also investigated in this work