Consistent normalized least mean square filtering with noisy data matrix

When the ordinary least squares method is applied to the parameter estimation problem with noisy data matrix, it is well-known that the estimates turn out to be biased. While this bias term can be somewhat reduced by the use of models of higher order, or by requiring a high signal-to-noise ratio (SNR), it can never be completely removed. Consistent estimates can be obtained by means of the instrumental variable method (IVM),or the total/data least squares method (TLS/DLS). In the adaptive setting for the such problem, a variety of least-mean-squares (LMS)-type algorithms have been researched rather than their recursive versions of IVM or TLS/DLS that cost considerable computations. Motivated by these observations, we propose a consistent LMS-type algorithm for the data least square estimation problem. This novel approach is based on the geometry of the mean squared error (MSE) function, rendering the step-size normalization and the heuristic filtered estimation of the noise variance, respectively, for fast convergence and robustness to stochastic noise. Monte Carlo simulations of a zero-forcing adaptive finite-impulse-response (FIR) channel equalizer demonstrate the efficacy of our algorithm.     

Application of Least Squares Lattice Algorithms to Adaptive Equalization

In many applications of adaptive data equalization, rapid initial convergence of the adaptive equalizer is of paramount importance. Apparently, the fastest known equalizer adaptation algorithm is based on a recursive least squares estimation algorithm. In this paper we show how the least squares lattice algorithms, recently introduced by Morf and Lee, can be adapted to the equalizer adjustment algorithm. The resulting algorithm, although computationally more complex than certain other equalizer algorithms (including the fast Kalman algorithm), has a number of desirable features which should prove useful in many applications.     

A recursive least M-estimate algorithm for robust adaptive filtering in impulsive noise: fast algorithm and convergence performance analysis

This paper studies the problem of robust adaptive filtering in impulsive noise environment using a recursive least M-estimate algorithm (RLM). The RLM algorithm minimizes a robust M-estimator-based cost function instead of the conventional mean square error function (MSE). Previous work has showed that the RLM algorithm offers improved robustness to impulses over conventional recursive least squares (RLS) algorithm. In this paper, the mean and mean square convergence behaviors of the RLM algorithm under the contaminated Gaussian impulsive noise model is analyzed. A lattice structure-based fast RLM algorithm, called the Huber Prior Error Feedback-Least Squares Lattice (H-PEF-LSL) algorithm is derived. Part of the H-PEF-LSL algorithm was presented in ICASSP 2001. It has an order O(N) arithmetic complexity, where N is the length of the adaptive filter, and can be viewed as a fast implementation of the RLM algorithm based on the modified Huber M-estimate function and the conventional PEF-LSL adaptive filtering algorithm. Simulation results show that the transversal RLM and the H-PEF-LSL algorithms have better performance than the conventional RLS and other RLS-like robust adaptive algorithms tested when the desired and input signals are corrupted by impulsive noise. Furthermore, the theoretical and simulation results on the convergence behaviors agree very well with each other.     

A set of algorithms linking NLMS and block RLS algorithms

This paper describes a set of block processing algorithms which contains as extremal cases the normalized least mean squares (NLMS) and the block recursive least squares (BRLS) algorithms. All these algorithms use small block lengths, thus allowing easy implementation and small input-output delay. It is shown that these algorithms require a lower number of arithmetic operations than the classical least mean squares (LMS) algorithm, while converging much faster. A precise evaluation of the arithmetic complexity is provided, and the adaptive behavior of the algorithm is analyzed. Simulations illustrate that the tracking characteristics of the new algorithm are also improved compared to those of the NLMS algorithm. The conclusions of the theoretical analysis are checked by simulations, illustrating that, even in the case where noise is added to the reference signal, the proposed algorithm allows altogether a faster convergence and a lower residual error than the NLMS algorithm. Finally, a sample-by-sample version of this algorithm is outlined, which is the link between the NLMS and recursive least squares (RLS) algorithms     

Recursive Extended Least Squares Parameter Estimation for Wiener Nonlinear Systems with Moving Average Noises

Many control algorithms are based on the mathematical models of dynamic systems. System identification is used to determine the structures and parameters of dynamic systems. Some identification algorithms (e.g., the least squares algorithm) can be applied to estimate the parameters of linear regressive systems or linear-parameter systems with white noise disturbances. This paper derives two recursive extended least squares parameter estimation algorithms for Wiener nonlinear systems with moving average noises based on over-parameterization models. The simulation results indicate that the proposed algorithms are effective.     

A fast recursive total least squares algorithm for adaptive IIR filtering

This work develops a new fast recursive total least squares (N-RTLS) algorithm to recursively compute the total least squares (TLS) solution for adaptive infinite-impulse-response (IIR) filtering. The new algorithm is based on the minimization of the constraint Rayleigh quotient in which the first entry of the parameter vector is fixed to the negative one. The highly computational efficiency of the proposed algorithm depends on the efficient computation of the gain vector and the adaptation of the Rayleigh quotient. Using the shift structure of the input data vectors, a fast algorithm for computing the gain vector is established, which is referred to as the fast gain vector (FGV) algorithm. The computational load of the FGV algorithm is smaller than that of the fast Kalman algorithm. Moreover, the new algorithm is numerically stable since it does not use the well-known matrix inversion lemma. The computational complexity of the new algorithm per iteration is also O(L). The global convergence of the new algorithm is studied. The performances of the relevant algorithms are compared via simulations.     

基于NLMS的CDMA盲自适应多用户检测算法研究

多用户检测是抑制DS-CDMA系统多址干扰最有效的技术之一。由于所需的先验知识仪有期望用户的地址码,盲多用户检测技术的研究尤受重视。最小输出能量(MOE)准则被广泛用于盲线性多用户检测。目前已提出的该类检测器多采用LMS或RLS算法。本文则研究基于NLMS算法的盲自适应检测技术,并进一步提出盲自适应变步长NLMS检测器和参数可变的盲自适应变步长NLMS检测器。它们具备很好的收敛速度和跟踪能力,以及较高的输出信干比,同时计算复杂度仅为O(3N)或O(4N),非常适合硬件实现。     

A novel variable step size adjustment method based on channel output autocorrelation for the LMS training algorithm

The least mean squares (LMS) algorithm, the most commonly used channel estimation and equalization technique, converges very slowly. The convergence rate of the LMS algorithm is quite sensitive to the adjustment of the step‐size parameter used in the update equation. Therefore, many studies have concentrated on adjusting the step‐size parameter in order to improve the training speed and accuracy of the LMS algorithm. A novel approach in adjusting the step size of the LMS algorithm using the channel output autocorrelation (COA) has been proposed for application to unknown channel estimation or equalization in low‐SNR in this paper. Computer simulations have been performed to illustrate the performance of the proposed method in frequency selective Rayleigh fading channels. The obtained simulation results using HIPERLAN/1 standard have demonstrated that the proposed variable step size LMS (VSS‐LMS) algorithm has considerably better performance than conventional LMS, recursive least squares (RLS), normalized LMS (N‐LMS) and the other VSS‐LMS algorithms. Copyright © 2011 John Wiley & Sons, Ltd.     

An efficient recursive total least squares algorithm for FIRadaptive filtering

An algorithm for recursively computing the total least squares (TLS) solution to the adaptive filtering problem is described. This algorithm requires O(N) multiplications per iteration to effectively track the N-dimensional eigenvector associated with the minimum eigenvalue of an augmented sample covariance matrix. It is shown that the recursive least squares (RLS) algorithm generates biased adaptive filter coefficients when the filter input vector contains additive noise. The TLS solution on the other hand, is seen to produce unbiased solutions. Examples of standard adaptive filtering applications that result in noise being added to the adaptive filter input vector are cited. Computer simulations comparing the relative performance of RLS and recursive TLS are described     

Adaptive symbol and parameter estimation in asynchronous multiuserCDMA detectors

Existing multiuser code-division multiple-access (CDMA) detectors either have to rely on strict power control or near-perfect parameter estimation for reliable operation. A novel adaptive multiuser CDMA detector structure is introduced. Using either an extended Kalman filter (EKF) or a recursive least squares (RLS) formulation, adaptive algorithms which jointly estimate the transmitted bits of each user and individual amplitudes and time delays may be derived. The proposed detectors work in a tracking mode after initial delay acquisition is accomplished using other techniques not discussed here. Through computer simulations, we show that the algorithms perform better than a bank of single-user (SU) receivers in terms of near-far resistance. Practical issues such as the selection of adaptation parameters are also discussed     

Set-Membership Adaptive Algorithms Based on Time-Varying Error Bounds for CDMA Interference Suppression

This paper presents set-membership (SM) adaptive algorithms based on time-varying error bounds for code-division multiple-access (CDMA) interference suppression. We introduce a modified family of SM adaptive algorithms for parameter estimation with time-varying error bounds. The considered algorithms include modified versions of the SM normalized least mean square (SM-NLMS), the affine projection (SM-AP), and the bounding ellipsoidal adaptive constrained (BEACON) recursive least-square technique. The important issue of error-bound specification is addressed in a new framework that takes into account parameter estimation dependency, multiaccess, and intersymbol interference (ISI) for direct-sequence CDMA (DS-CDMA) communications. An algorithm for tracking and estimating the interference power is proposed and analyzed. This algorithm is then incorporated into the proposed time-varying error bound mechanisms. Computer simulations show that the proposed algorithms are capable of outperforming previously reported techniques with a significantly lower number of parameter updates and a reduced risk of overbounding or underbounding.      

Modified LMS algorithm for unbiased impulse response estimation innonstationary noise

In the presence of input interference, the Wiener solution for impulse response estimation is biased. It is proved that bias removal can be achieved by proper scaling of the optimal filter coefficients and a modified least mean squares (LMS) algorithm is then developed for accurate system identification in noise. Simulation results show that the proposed method outperforms two total least squares (TLS) based adaptive algorithms under nonstationary interference conditions     

基于区间全局优化的非线性最小二乘估计

分析了使用区间全局优化算法进行非线性系统模型参数估计的原因,介绍了非线性最小二乘估计和区间全局优化算法.在非线性系统模型参数估计中,相对于通过优化目标函数求得待估参数点估计的现有算法,基于区间分析的区间全局优化算法不仅可以求得待估参数的点估计,还可得到肯定包含待估参数真值的估计区间,并且该算法还具有计算结果稳定以及更大范围收敛的性质.通过仿真实验并与其他方法进行比较,结果表明算法的可行性和有效性.     

Application of Fast Kalman Estimation to Adaptive Equalization

Very rapid initial convergence of the equalizer tap coefficients is a requirement of many data communication systems which employ adaptive equalizers to minimize intersymbol interference. As shown in recent papers by Godard, and by Gitlin and Magee, a recursive least squares estimation algorithm, which is a special case of the Kalman estimation algorithm, is applicable to the estimation of the optimal (minimum MSE) set of tap coefficients. It was furthermore shown to yield much faster equalizer convergence than that achieved by the simple estimated gradient algorithm, especially for severely distorted channels. We show how certain "fast recursive estimation" techniques, originally introduced by Morf and Ljung, can be adapted to the equalizer adjustment problem, resulting in the same fast convergence as the conventional Kalman implementation, but with far fewer operations per iteration (proportional to the number of equalizer taps, rather than the square of the number of equalizer taps). These fast algorithms, applicable to both linear and decision feedback equalizers, exploit a certain shift-invariance property of successive equalizer contents. The rapid convergence properties of the "fast Kalman" adaptation algorithm are confirmed by simulation.     

Least Squares Identification for Hammerstein Multi-input Multi-output Systems Based on the Key-Term Separation Technique

System modeling and parameter estimation are basic for system analysis and controller design. This paper considers the parameter identification problem of a Hammerstein multi-input multi-output (H-MIMO) system. In order to avoid the product terms in the identification model, we derive a pseudo-linear identification model of the H-MIMO system through separating a key term from the output equation of the system and present a hierarchical generalized least squares (LS) algorithm for estimating the parameters of the system. Moreover, we present a new LS algorithm to reduce the computational burden. The proposed algorithms are simple in principle and can achieve a higher computational efficiency than the over-parameterization-based LS estimation algorithm. Finally, we test the proposed algorithms by the simulation example and show their effectiveness.     

Iterative least squares estimators in nonlinear image restoration

The concept of iterative least squares estimation as applied to nonlinear image restoration is considered. Regarding the convergence analysis of nonlinear iterative algorithms, the potential of the global convergence theorem (GCT) is explored. The theoretical analysis is performed on a general class of nonlinear algorithms, which defines a signal-dependent linear mapping of the residual. The descent properties of two normed functions are considered. Furthermore, a procedure for the selection of the iteration parameter is introduced. The steepest descent (SD) iterative approach for the solution of the least squares optimization problem is introduced. The convergence properties of the particular algorithm are readily derived on the basis of the generalized analysis and the GCT. The factors that affect the convergence rate of the SD algorithm are thoroughly studied. In the case of the SD algorithm, structural modifications are proposed, and two hybrid-SD algorithms attain convergence in a more uniform fashion with respect to their entries. In general, the algorithms achieve larger convergence rates than the conventional SD technique     

Robust M-estimate adaptive filtering

An M-estimate adaptive filter for robust adaptive filtering in impulse noise is proposed. Instead of using the conventional least-square cost function, a new cost function based on an M-estimator is used to suppress the effect of impulse noise on the filter weights. The resulting optimal weight vector is governed by an M-estimate normal equation. A recursive least M-estimate (RLM) adaptive algorithm and a robust threshold estimation method are derived for solving this equation. The mean convergence performance of the proposed algorithm is also analysed using the modified Huber (1981) function (a simple but good approximation to the Hampel's three-parts-redescending M-estimate function) and the contaminated Gaussian noise model. Simulation results show that the proposed RLM algorithm has better performance than other recursive least squares (RLS) like algorithms under either a contaminated Gaussian or alpha-stable noise environment. The initial convergence, steady-state error, robustness to system change and computational complexity are also found to be comparable to the conventional RLS algorithm under Gaussian noise alone     

Lattice filters for adaptive processing

This paper presents a tutorial review of lattice structures and their use for adaptive prediction of time series. Lattice filters associated with stationary covariance sequences and their properties are discussed. The least squares prediction problem is defined for the given data case, and it is shown that many of the currently used lattice methods are actually approximations to the stationary least squares solution. The recently developed class of adaptive least squares lattice algorithms are described in detail, both in their unnormalized and normalized forms. The performance of the adaptive least squares lattice algorithm is compared to that of some gradient adaptive methods. Lattice forms for ARMA processes, for joint process estimation, and for the sliding-window covariance case are presented. The use of lattice structures for efficient factorization of covariance matrices and solution of Toeplitz sets of equations is briefly discussed.     

Just-in-time adaptive disturbance estimation for run-to-run control of semiconductor processes

Run-to-run control is the term used for the application of discrete parts manufacturing control as practiced in the semiconductor industry. This paper presents a new algorithm for use in run-to-run control that has been designed to address some of the challenging issues unique to batch-type manufacturing. Just-in-time adaptive disturbance estimation (JADE) uses recursive weighted least squares parameter estimation to identify the contributions to variation that are dependent upon manufacturing context. The strengths and weaknesses of the JADE algorithm are demonstrated in a series of test cases developed to separate the various disturbances and processing issues a control system would be expected to encounter.