Real-Time Implementation of Parallel Architecture Based Noise Minimization from Speech Signals on FPGA

In recent years, the real time hardware implementation of LMS based adaptive noise cancellation on FPGA is becoming popular. There are several works reported in this area in the literature. However, NLMS based implementation of adaptive noise cancellation on FPGA using Xilinx System Generator (XSG) is not reported. This paper explores the various forms of parallel architecture based on NLMS algorithm and its hardware implementation on FPGA using XSG for noise minimization from speech signals. In total, the direct form, binary tree direct form and transposed form of parallel architecture of normalized least mean square (NLMS), delayed normalized least mean square and retimed delayed normalized least mean square algorithms are implemented on FPGA using hardware co-simulation model. The performance parameters (SNR and MSE) of these algorithms are analyzed for the adaptive noise cancellation system and the comparison is made with parallel architectures of least mean square (LMS), delayed least mean square, and retimed delayed least mean square algorithms respectively. The hardware utilization of all the said algorithms are also analyzed and compared. The result shows that NLMS based implementations outperform than that of LMS for all forms of parallel architecture for the given parameters with negligence increase in device utility. The binary tree direct form of retimed delayed NLMS achieves the maximum SNR improvement (39.83 dB) in comparison to other NLMS variants at the cost of optimum resource utilization.     

Performance of Smart Antenna Under Different Fading Conditions

Optimizing the current distribution of an evenly spaced antenna array has shown to be an efficient approach for reducing side lobe levels. In this article, the Tchebyscheff distribution-based antenna array synthesis approach is combined with an adaptive signal processing algorithm for beamforming and side lobe level reduction in smart antennas in various fading situations. The performance of smart antennas in uniformly spaced linear, planar, circular, and semi-circular arrays are evaluated. The presence of Rayleigh and Rician channels is examined in the network. The least mean square (LMS) and normalised least mean square (NLMS) algorithms are applied as adaptive algorithms. In fading environments, the NLMS algorithm with Tchebyscheff distribution outperforms than the LMS algorithm with Tchebyscheff distribution, with a side lobe level decrease of 11.23 dB. The lowest side lobe achieved with the NLMS algorithm with Tchebyscheff distribution is???45.59 dB for uniform planar array.

     

一种约束稳定性最小均方噪声对消算法

研究了最小均方误差(LMS)算法、归一化的最小均方(NLMS)算法及变步长NLMS算法在自适应噪声干扰抵消器中的应用,针对目前这些算法在噪声对消器应用中的缺点,将约束稳定性最小均方(CS-LMS)算法应用到噪声处理中,并进一步结合变步长的思想提出来一种新的变步长CS-LMS算法。通过MATLAB进行仿真分析,结果证实提出的算法与其他算法相比,能很好地滤除掉噪声从而得到期望信号,明显的降低了稳态误差,并拥有好的收敛速度。     

A general class of nonlinear normalized adaptive filteringalgorithms

The normalized least mean square (NLMS) algorithm is an important variant of the classical LMS algorithm for adaptive linear filtering. It possesses many advantages over the LMS algorithm, including having a faster convergence and providing for an automatic time-varying choice of the LMS stepsize parameter that affects the stability, steady-state mean square error (MSE), and convergence speed of the algorithm. An auxiliary fixed step-size that is often introduced in the NLMS algorithm has the advantage that its stability region (step-size range for algorithm stability) is independent of the signal statistics. In this paper, we generalize the NLMS algorithm by deriving a class of nonlinear normalized LMS-type (NLMS-type) algorithms that are applicable to a wide variety of nonlinear filter structures. We obtain a general nonlinear NLMS-type algorithm by choosing an optimal time-varying step-size that minimizes the next-step MSE at each iteration of the general nonlinear LMS-type algorithm. As in the linear case, we introduce a dimensionless auxiliary step-size whose stability range is independent of the signal statistics. The stability region could therefore be determined empirically for any given nonlinear filter type. We present computer simulations of these algorithms for two specific nonlinear filter structures: Volterra filters and the previously proposed class of Myriad filters. These simulations indicate that the NLMS-type algorithms, in general, converge faster than their LMS-type counterparts     

基于DFP的二阶Volterra模型及其对Rössler混沌序列的预测

为克服最小二乘法或归一化最小二乘法在二阶Volterra建模时参数选择不当引起的问题,在最小二乘法基础上,应用一种基于后验误差假设的可变收敛因子技术,构建了一种基于Davidon-Fletcher-Powell算法的二阶Volterra模型（DFPSOVF）.给出参数估计中自相关逆矩阵估计的递归更新公式,并对其正定性、有界性和τ（n）的作用进行了研究.将DFPSOVF模型应用于Rössler混沌序列的单步预测,仿真结果表明其能够保证算法的稳定性和收敛性,不存在最小二乘法和归一化最小二乘法的发散问题.     

Convergence and performance analysis of the normalized LMSalgorithm with uncorrelated Gaussian data

It is demonstrated that the normalized least mean square (NLMS) algorithm can be viewed as a modification of the widely used LMS algorithm. The NLMS is shown to have an important advantage over the LMS, which is that its convergence is independent of environmental changes. In addition, the authors present a comprehensive study of the first and second-order behavior in the NLMS algorithm. They show that the NLMS algorithm exhibits significant improvement over the LMS algorithm in convergence rate, while its steady-state performance is considerably worse     

On the convergence behavior of the LMS and the normalized LMSalgorithms

It is shown that the normalized least mean square (NLMS) algorithm is a potentially faster converging algorithm compared to the LMS algorithm where the design of the adaptive filter is based on the usually quite limited knowledge of its input signal statistics. A very simple model for the input signal vectors that greatly simplifies analysis of the convergence behavior of the LMS and NLMS algorithms is proposed. Using this model, answers can be obtained to questions for which no answers are currently available using other (perhaps more realistic) models. Examples are given to illustrate that even quantitatively, the answers obtained can be good approximations. It is emphasized that the convergence of the NLMS algorithm can be speeded up significantly by employing a time-varying step size. The optimal step-size sequence can be specified a priori for the case of a white input signal with arbitrary distribution     

A New Sequential Block Partial Update Normalized Least Mean M-Estimate Algorithm and its Convergence Performance Analysis

This paper proposes a new sequential block partial update normalized least mean square (SBP-NLMS) algorithm and its nonlinear extension, the SBP-normalized least mean M-estimate (SBP–NLMM) algorithm, for adaptive filtering. These algorithms both utilize the sequential partial update strategy as in the sequential least mean square (S–LMS) algorithm to reduce the computational complexity. Particularly, the SBP–NLMM algorithm minimizes the M-estimate function for improved robustness to impulsive outliers over the SBP–NLMS algorithm. The convergence behaviors of these two algorithms under Gaussian inputs and Gaussian and contaminated Gaussian (CG) noises are analyzed and new analytical expressions describing the mean and mean square convergence behaviors are derived. The robustness of the proposed SBP–NLMM algorithm to impulsive noise and the accuracy of the performance analysis are verified by computer simulations.     

Second-order H∞ optimal LMS and NLMS algorithmsbased on a second-order Markov model

It is shown that two algorithms obtained by simplifying a Kalman filter considered for a second-order Markov model are H^∞ suboptimal. Similar to least mean squares (LMS) and normalised LMS (NLMS) algorithms, these second order algorithms can be thought of as approximate solutions to stochastic or deterministic least squares minimisation. It is proved that second-order LMS and NLMS are exact solutions causing the maximum energy gain from the disturbances to the predicted and filtered errors to be less than one, respectively. These algorithms are implemented in two steps. Operation of the first step is like conventional LMS/NLMS algorithms and the second step consists of the estimation of the weight increment vector and prediction of weights for the next iteration. This step applies simple smoothing on the increment of the estimated weights to estimate the speed of the weights. Also they are cost-effective, robust and attractive for improving the tracking performance of smoothly time-varying models     

采用归一化最小均方误差准则的LM-BP算法

传统神经网络通常以最小均方误差(LMS)或最小二乘（RLS）为收敛准则,而在自适应均衡等一些应用中,使用归一化最小均方误差（NLMS）准则可以使神经网络性能更加优越。本文在NLMS准则基础上,提出了一种以Levenberg-Marquardt（LM）训练的神经网络收敛算法。通过将神经网络的误差函数归一化,然后采用LM算法作为训练算法,实现了神经网络的快速收敛。理论分析和实验仿真表明,与采用最速下降法的NLMS准则和采用LM算法的LMS准则相比,本文算法收敛速度快,归一化均方误差更小,应用于神经网络水印系统中实现了水印信息的盲提取,能更好的抵抗噪声、低通滤波和重量化等攻击,性能平均提高了4%。      

A set of algorithms linking NLMS and block RLS algorithms

This paper describes a set of block processing algorithms which contains as extremal cases the normalized least mean squares (NLMS) and the block recursive least squares (BRLS) algorithms. All these algorithms use small block lengths, thus allowing easy implementation and small input-output delay. It is shown that these algorithms require a lower number of arithmetic operations than the classical least mean squares (LMS) algorithm, while converging much faster. A precise evaluation of the arithmetic complexity is provided, and the adaptive behavior of the algorithm is analyzed. Simulations illustrate that the tracking characteristics of the new algorithm are also improved compared to those of the NLMS algorithm. The conclusions of the theoretical analysis are checked by simulations, illustrating that, even in the case where noise is added to the reference signal, the proposed algorithm allows altogether a faster convergence and a lower residual error than the NLMS algorithm. Finally, a sample-by-sample version of this algorithm is outlined, which is the link between the NLMS and recursive least squares (RLS) algorithms     

Convergence Behavior of NLMS Algorithm for Gaussian Inputs: Solutions Using Generalized Abelian Integral Functions and Step Size Selection

This paper studies the mean and mean square convergence behaviors of the normalized least mean square (NLMS) algorithm with Gaussian inputs and additive white Gaussian noise. Using the Price’s theorem and the framework proposed by Bershad in IEEE Transactions on Acoustics, Speech, and Signal Processing (1986, 1987), new expressions for the excess mean square error, stability bound and decoupled difference equations describing the mean and mean square convergence behaviors of the NLMS algorithm using the generalized Abelian integral functions are derived. These new expressions which closely resemble those of the LMS algorithm allow us to interpret the convergence performance of the NLMS algorithm in Gaussian environment. The theoretical analysis is in good agreement with the computer simulation results and it also gives new insight into step size selection.     

Adaptive sparse system identification using normalized least mean fourth algorithm

Normalized least mean square (NLMS) was considered as one of the classical adaptive system identification algorithms. Because most of systems are often modeled as sparse, sparse NLMS algorithm was also applied to improve identification performance by taking the advantage of system sparsity. However, identification performances of NLMS type algorithms cannot achieve high‐identification performance, especially in low signal‐to‐noise ratio regime. It was well known that least mean fourth (LMF) can achieve high‐identification performance by utilizing fourth‐order identification error updating rather than second‐order. However, the main drawback of LMF is its instability and it cannot be applied to adaptive sparse system identifications. In this paper, we propose a stable sparse normalized LMF algorithm to exploit the sparse structure information to improve identification performance. Its stability is shown to be equivalent to sparse NLMS type algorithm. Simulation results show that the proposed normalized LMF algorithm can achieve better identification performance than sparse NLMS one. Copyright © 2013 John Wiley & Sons, Ltd.     

Adaptive algorithms for non-Gaussian noise environments: the orderstatistic least mean square algorithms

Convergence properties are studied for a class of gradient-based adaptive filters known as order statistic least mean square (OSLMS) algorithms. These algorithms apply an order statistic filtering operation to the gradient estimate of the standard least mean square (LMS) algorithm. The order statistic operation in OSLMS algorithms can reduce the variance of the gradient estimate (relative to LMS) when operating in non-Gaussian noise environments. A consequence is that in steady state, the excess mean square error can be reduced. It is shown that when the input signals are iid and symmetrically distributed, the coefficient estimates for the OSLMS algorithms converge on average to a small area around their optimal values. Simulations provide supporting evidence for algorithm convergence. As a measurement of performance, the mean squared coefficient error of OSLMS algorithms has been evaluated under a range of noise distributions and OS operators. Guidelines for selection of the OS operator are presented based on the expected noise environment     

New Variable Step-Sizes Minimizing Mean-Square Deviation for the LMS-Type Algorithms

The least-mean-square-type (LMS-type) algorithms are known as simple and effective adaptation algorithms. However, the LMS-type algorithms have a trade-off between the convergence rate and steady-state performance. In this paper, we investigate a new variable step-size approach to achieve fast convergence rate and low steady-state misadjustment. By approximating the optimal step-size that minimizes the mean-square deviation, we derive variable step-sizes for both the time-domain normalized LMS (NLMS) algorithm and the transform-domain LMS (TDLMS) algorithm. The proposed variable step-sizes are simple quotient forms of the filtered versions of the quadratic error and very effective for the NLMS and TDLMS algorithms. The computer simulations are demonstrated in the framework of adaptive system modeling. Superior performance is obtained compared to the existing popular variable step-size approaches of the NLMS and TDLMS algorithms.     

基于自适应系统辨识的收发隔离技术研究

收发隔离是机载干扰机不可避免的难题。如果收发隔离问题解决不好,轻则削弱干扰机效率,重则造成自发自收,形成自激励。固定步长的归一化最小均方误差(NLMS)算法在解决基于自适应系统辨识的收发隔离的问题时,由于精度不够,隔离效果很不理想。针对此问题提出一种基于先验误差的变步长NLMS算法,该算法依据相邻时刻先验误差的相关系数改变步长因子,改变后的步长因子能够在算法收敛过程中削弱噪声的影响,提高算法精度。理论分析和仿真结果证明:基于文中的变步长NLMS算法的收发隔离方案与基于其他最小均方误差算法的隔离方案相比,隔离性能有较大的改善。     

On the Performance Analysis of the Least Mean M-Estimate and Normalized Least Mean M-Estimate Algorithms with Gaussian Inputs and Additive Gaussian and Contaminated Gaussian Noises

This paper studies the convergence analysis of the least mean M-estimate (LMM) and normalized least mean M-estimate (NLMM) algorithms with Gaussian inputs and additive Gaussian and contaminated Gaussian noises. These algorithms are based on the M-estimate cost function and employ error nonlinearity to achieve improved robustness in impulsive noise environment over their conventional LMS and NLMS counterparts. Using the Price’s theorem and an extension of the method proposed in Bershad (IEEE Transactions on Acoustics, Speech, and Signal Processing, ASSP-34(4), 793–806, 1986; IEEE Transactions on Acoustics, Speech, and Signal Processing, 35(5), 636–644, 1987), we first derive new expressions of the decoupled difference equations which describe the mean and mean square convergence behaviors of these algorithms for Gaussian inputs and additive Gaussian noise. These new expressions, which are expressed in terms of the generalized Abelian integral functions, closely resemble those for the LMS algorithm and allow us to interpret the convergence performance and determine the step size stability bound of the studied algorithms. Next, using an extension of the Price’s theorem for Gaussian mixture, similar results are obtained for additive contaminated Gaussian noise case. The theoretical analysis and the practical advantages of the LMM/NLMM algorithms are verified through computer simulations.     

Asymptotic analysis of stochastic gradient-based adaptive filteringalgorithms with general cost functions

This paper presents an analysis of stochastic gradient-based adaptive algorithms with general cost functions. The analysis holds under mild assumptions on the inputs and the cost function. The method of analysis is based on an asymptotic analysis of fixed stepsize adaptive algorithms and gives almost sure results regarding the behavior of the parameter estimates, whereas previous stochastic analyses typically considered mean and mean square behavior. The parameter estimates are shown to enter a small neighborhood about the optimum value and remain there for a finite length of time. Furthermore, almost sure exponential bounds are given for the rate of convergence of the parameter estimates. The asymptotic distribution of the parameter estimates is shown to be Gaussian with mean equal to the optimum value and covariance matrix that depends on the input statistics. Specific adaptive algorithms that fall under the framework of this paper are signed error least mean square (LMS), dual sign LMS, quantized state LMS, least mean fourth, dead zone algorithms, momentum algorithms, and leaky LMS     

一种新的变步长NLMS盲自适应多用户检测算法

首先推导得到最小输出能力准则的无约束形式,得到易于NLMS算法实现形式;在该NLMS算法实现的基础上,通过重复归一化迭代运算推导,得到重复跌代的等效形式;基于该形式提出一种新的变步长NLMS算法,算法复杂度仍然为O(N),算法具有很好的健壮性,并且具有更好的检测综合性能。仿真表明,与文献［5］所提变参数的变步长NLMS算法相比,本文算法收敛速度更快且输出信干比更好。在同步CDMA系统下,其综合性能达到KALMAN,RLS相当水平。      

Sparse LMS/F algorithms with application to adaptive system identification

Standard least mean square/fourth (LMS/F) is a classical adaptive algorithm that combined the advantages of both least mean square (LMS) and least mean fourth (LMF). The advantage of LMS is fast convergence speed while its shortcoming is suboptimal solution in low signal‐to‐noise ratio (SNR) environment. On the contrary, the advantage of LMF algorithm is robust in low SNR while its drawback is slow convergence speed in high SNR case. Many finite impulse response systems are modeled as sparse rather than traditionally dense. To take advantage of system sparsity, different sparse LMS algorithms with l_p‐LMS and l₀‐LMS have been proposed to improve adaptive identification performance. However, sparse LMS algorithms have the same drawback as standard LMS. Different from LMS filter, standard LMS/F filter can achieve better performance. Hence, the aim of this paper is to introduce sparse penalties to the LMS/F algorithm so that it can further improve identification performance. We propose two sparse LMS/F algorithms using two sparse constraints to improve adaptive identification performance. Two experiments are performed to show the effectiveness of the proposed algorithms by computer simulation. In the first experiment, the number of nonzero coefficients is changing, and the proposed algorithms can achieve better mean square deviation performance than sparse LMS algorithms. In the second experiment, the number of nonzero coefficient is fixed, and mean square deviation performance of sparse LMS/F algorithms is still better than that of sparse LMS algorithms. Copyright © 2013 John Wiley & Sons, Ltd.