A Novel Adaptive Nonlinear Filter-Based Pipelined Feedforward Second-Order Volterra Architecture

Due to the computational complexity of the Volterra filter, there are limitations on the implementation in practice. In this paper, a novel adaptive joint process filter using pipelined feedforward second-order Volterra architecture (JPPSOV) to reduce the computational burdens of the Volterra filter is proposed. The proposed architecture consists of two subsections: nonlinear subsection performing a nonlinear mapping from the input space to an intermediate space by the feedforward second-order Volterra (SOV), and a linear combiner performing a linear mapping from the intermediate space to the output space. The corresponding adaptive algorithms are deduced for the nonlinear and linear combiner subsections, respectively. Moreover, the analysis of theory shows that these adaptive algorithms based on the pipelined architecture are stable and convergence under a certain condition. To evaluate the performance of the JPPSOV, a series of simulation experiments are presented including nonlinear system identification and predicting of speech signals. Compared with the conventional SOV filter, adaptive JPPSOV filter exhibits a litter better convergence performance with less computational burden in terms of convergence speed and steady-state error.     

Active noise control using an adaptive bacterial foraging optimization algorithm

This paper presents an adaptive bacterial foraging optimization (ABFO) algorithm for an active noise control system. The conventional active noise control (ANC) systems often use the gradient-based filtered-X least mean square algorithms to adapt the coefficients of the adaptive controller. Hence, there is a possibility to converge to local minima. In addition, this class of algorithms needs prior identification of the secondary path. The ABFO algorithm helps the ANC system to prevent falling into local minima. The proposed ANC system is also simpler since it does not need any prior information of the secondary path. Moreover, the adaptive strategy of the algorithm results in improved search performance compared with the basic bacterial foraging optimization algorithm, as well as other conventional algorithms. Experimental studies are performed for nonlinear primary path along with linear and nonlinear secondary path. The results show the effectiveness of the proposed ABFO-based ANC system for different kinds of input noise.     

Second-order adaptive Volterra system identification based ondiscrete nonlinear Wiener model

The authors present the nonlinear LMS adaptive filtering algorithm based on the discrete nonlinear Wiener (1942) model for second-order Volterra system identification application. The main approach is to perform a complete orthogonalisation procedure on the truncated Volterra series. This allows the use of the LMS adaptive linear filtering algorithm for calculating all the coefficients with efficiency. This orthogonalisation method is based on the nonlinear discrete Wiener model. It contains three sections: a single-input multi-output linear with memory section, a multi-input, multi-output nonlinear no-memory section and a multi-input, single-output amplification and summary section. For a white Gaussian noise input signal, the autocorrelation matrix of the adaptive filter input vector can be diagonalised unlike when using the Volterra model. This dramatically reduces the eigenvalue spread and results in more rapid convergence. Also, the discrete nonlinear Wiener model adaptive system allows us to represent a complicated Volterra system with only few coefficient terms. In general, it can also identify the nonlinear system without over-parameterisation. A theoretical performance analysis of steady-state behaviour is presented. Computer simulations are also included to verify the theory     

Statistical analysis of neural network modeling and identificationof nonlinear systems with memory

The paper presents a statistical analysis of neural network modeling and identification of nonlinear systems with memory. The nonlinear system model is comprised of a discrete-time linear filter H followed by a zero-memory nonlinear function g(.). The system is corrupted by input and output independent Gaussian noise. The neural network is used to identify and model the unknown linear filter H and the unknown nonlinearity g(.). The network architecture is composed of a linear adaptive filter and a two-layer nonlinear neural network (with an arbitrary number of neurons). The network is trained using the backpropagation algorithm. The paper studies the MSE surface and the stationary points of the adaptive system. Recursions are derived for the mean transient behavior of the adaptive filter coefficients and neural network weights for slow learning. It is shown that the Wiener solution for the adaptive filter is a scaled version of the unknown filter H. Computer simulations show good agreement between theory and Monte Carlo estimations     

A computationally efficient and interference tolerant nonparametricalgorithm for LTI system identification based on higher ordercyclostationarity

A new nonparametric algorithm for the identification of linear time-invariant systems is proposed. The method is based on the cyclic correlations of the input and output signals with a nonlinear transformation of the input signal. Consequently, although it exploits the higher order cyclostationarity properties of the input and output signals, its computational complexity is comparable with that of methods based on second-order statistics. The proposed estimator of the system transfer function is inherently immune to the presence of noise and interference on both input and output signal measurements and turns out to be asymptotically unbiased and consistent. Moreover, bias and variance of the estimate exhibit a rate of convergence to zero equal to that of estimates based on second-order statistics. Finally, simulation results show that the proposed algorithm significantly outperforms, in terms of both bias and variance of the estimates, several nonparametric identification algorithms previously presented in the literature     

A digital technique to estimate second-order distortion usinghigher order coherence spectra

A digital spectral method for evaluating second-order distortion of a nonlinear system, which can be represented by Volterra kernels up to second order and which is subjected to a random noise input, is discussed. The importance of departures from the commonly assumed Gaussian excitation is investigated. The Hinich test is shown to be an appropriate test for orthogonality in the system identification. Tests for Gaussianity of two important sources, which are commonly used for Gaussian inputs in nonlinear system identification, are presented: (1) commercial software routines for simulation experiments, and (2) noise generators for practical experiments. The deleterious effects of assuming a Gaussian input when it is not are demonstrated. The random input method for evaluating the second-order distortion of a nonlinear system is compared with the sine-wave input method using both simulation and experimental data. The approach is applied to a loudspeaker in the low-frequency band     

一种全最小二乘算法及其在非线性滤波器中的应用

针对输入输出观测数据均含有噪声的滤波问题,提出了一种稳定的总体最小二乘自适应算法。该算法以系统的增广权向量的瑞利商(RQ)与对增广权向量的最后元素的约束的和作为总损失函数,利用梯度最陡下降原理导出权向量的自适应迭代算法,并将该算法应用于非线性Volterra滤波器。研究了算法的稳定性能,提出的算法不仅有良好的收敛性能,而且在权向量的自适应迭代时不需要标准化处理,使得算法的实施更为简单。仿真实验表明,无论在线性系统或非线性系统,本文算法的收敛性能,鲁棒抗噪性能和稳态收敛精度明显高于其它同类总体最小二乘算法。     

Image despeckling and deblurring via regularized complex diffusion

In this paper an image restoration and enhancement model is being proposed, which is suitable for multiplicative data-dependent speckle noise (whose intensity is Gamma distributed) under linear shift-invariant blurring artifacts. The proposed strategy devises a nonlinear second-order diffusive-reactive model for enhancing and restoring images degraded by the aforementioned scenario. The reactive term is derived based on the Maximum a posteriori (MAP) estimator, to make it adaptive to the noise distribution in the input data. This noise-adaptive reactive term helps to restore and enhance the images under data-correlated noise setup. Unlike the other second-order nonlinear diffusion methods, the proposed solution preserves edges and details and reduces piecewise constant approximation in the homogeneous intensity regions in the course of its evolution. The experimental results demonstrated in this paper duly support the above claims.     

Blind identification of linear subsystems of LTI-ZMNL-LTI modelswith cyclostationary inputs

Discrete-time nonlinear models consisting of two linear time invariant (LTI) filters separated by a finite-order zero memory nonlinearity (ZMNL) of the polynomial type (the LTI-ZMNL-LTI model) are appropriate in a large number of practical applications. We discuss some approaches to the problem of blind identification of such nonlinear models, It is shown that for an Nth-order nonlinearity, the (possibly non-minimum phase) finite-memory linear subsystems of LTI-ZMNL and LTI-ZMNL-LTI models can be identified using the N+1th-order (cyclic) statistics of the output sequence alone, provided the input is cyclostationary and satisfies certain conditions. The coefficients of the ZMNL are not needed for identification of the linear subsystems and are not estimated. It is shown that the theory presented leads to analytically simple identification algorithms that possess several noise and interference suppression characteristics     

Stochastic gradient adaptation under general error criteria

Examines a family of adaptive filter algorithms of the form W_k+1=W_k+μf(d_k-W_k^t X_k)X_k in which f(·) is a memoryless odd-symmetric nonlinearity acting upon the error. Such algorithms are a generalization of the least-mean-square (LMS) adaptive filtering algorithm for even-symmetric error criteria. For this algorithm family, the authors derive general expressions for the mean and mean-square convergence of the filter coefficients For both arbitrary stochastic input data and Gaussian input data. They then provide methods for optimizing the nonlinearity to minimize the algorithm misadjustment for a given convergence rate. Using the calculus of variations, it is shown that the optimum nonlinearity to minimize misadjustment near convergence under slow adaptation conditions is independent of the statistics of the input data and can be expressed as -p'(x)/p(x), where p(x) is the probability density function of the uncorrelated plant noise. For faster adaptation under the white Gaussian input and noise assumptions, the nonlinearity is shown to be x/{1+μλx²/σ_k ²}, where λ is the input signal power and σ_k² is the conditional error power. Thus, the optimum stochastic gradient error criterion for Gaussian noise is not mean-square. It is shown that the equations governing the convergence of the nonlinear algorithm are exactly those which describe the behavior of the optimum scalar data nonlinear adaptive algorithm for white Gaussian input. Simulations verify the results for a host of noise interferences and indicate the improvement using non-mean-square error criteria     

Efficient algorithms for Volterra system identification

In this paper, nonlinear filtering and identification based on finite-support Volterra models are considered. The Volterra kernels are estimated via input-output statistics or directly in terms of input-output data. It is shown that the normal equations for a finite-support Volterra system excited by zero mean Gaussian input have a unique solution if, and only if, the power spectral process of the input signal is nonzero at least at m distinct frequencies, where m is the memory of the system. A multichannel embedding approach is introduced. A set of primary signals defined in terms of the input signal serve to map efficiently the nonlinear process to an equivalent multichannel format. Efficient algorithms for the estimation of the Volterra parameters are derived for batch, as well as for adaptive processing. An efficient order-recursive method is presented for the determination of the Volterra model structure. The proposed methods are illustrated by simulations     

Direct estimation of blind zero-forcing equalizers based onsecond-order statistics

Most existing zero-forcing equalization algorithms rely either on higher than second-order statistics or on partial or complete channel identification. We describe methods for computing fractionally spaced zero-forcing blind equalizers with arbitrary delay directly from second-order statistics of the observations without channel identification. We first develop a batch-type algorithm; then, adaptive algorithms are obtained by linear prediction and gradient descent optimization. Our adaptive algorithms do not require channel order estimation, nor rank estimation. Compared with other second-order statistics-based approaches, ours do not require channel identification at all. On the other hand, compared with the CMA-type algorithms, ours use only second-order statistics; thus, no local convergence problem exists, and faster convergence can be achieved. Simulations show that our algorithms outperform most typical existing algorithms     

Novel Adaptive Nonlinear Predistorters Based on the Direct Learning Algorithm

The adaptive nonlinear predistorter is an effective technique to compensate for the nonlinear distortion existing in digital communication and control systems. However, available adaptive nonlinear predistorters using indirect learning are sensitive to measurement noise and do not perform optimally. Other available types are either slow to converge, complicated in structure and computationally expensive, or do not consider the memory effects in nonlinear systems such as a high power amplifier (HPA). In this paper, we first propose several novel adaptive nonlinear predistorters based on direct learning algorithms - the nonlinear filtered-x RLS (NFXRLS) algorithm, the nonlinear adjoint LMS (NALMS) algorithm, and the nonlinear adjoint RLS (NARLS) algorithm. Using these new learning algorithms, we design adaptive nonlinear predistorters for an HPA with memory effects or for an HPA following a linear system. Because of the direct learning algorithm, these novel adaptive predistorters outperform nonlinear predistorters that are based on the indirect learning method in the sense of normalized mean square error (NMSE), bit error rate (BER), and spectral regrowth. Moreover, our developed adaptive nonlinear predistorters are computationally efficient and/or converge rapidly when compared to other adaptive nonlinear predistorters that use direct learning, and furthermore can be easily implemented. We further simplify our proposed algorithms by exploring the robustness of our proposed algorithm as well as by examining the statistical properties of what we call the "instantaneous equivalent linear" (IEL) filter. Simulation results confirm the effectiveness of our proposed algorithms     

Adaptive threshold modulation for error diffusion halftoning

Grayscale digital image halftoning quantizes each pixel to one bit. In error diffusion halftoning, the quantization error at each pixel is filtered and fed back to the input in order to diffuse the quantization error among the neighboring grayscale pixels. Error diffusion introduces nonlinear distortion (directional artifacts), linear distortion (sharpening), and additive noise. Threshold modulation, which alters the quantizer input, has been previously used to reduce either directional artifacts or linear distortion. This paper presents an adaptive threshold modulation framework to improve halftone quality by optimizing error diffusion parameters in the least squares sense. The framework models the quantizer implicitly, so a wide variety of quantizers may be used. Based on the framework, we derive adaptive algorithms to optimize 1) edge enhancement halftoning and 2) green noise halftoning. In edge enhancement halftoning, we minimize linear distortion by controlling the sharpening control parameter. We may also break up directional artifacts by replacing the thresholding quantizer with a deterministic bit flipping (DBF) quantizer. For green noise halftoning, we optimize the hysteresis coefficients.     

Modeling (almost) periodic moving average processes using cyclicstatistics

Estimating parameters of almost cyclostationary non-Gaussian moving average (MA) processes using noisy output-only data is considered. It is shown that second-order cyclic correlations of the output are generally insufficient in uniquely characterizing almost periodically time-varying MA(q) models, while third-order and higher order cumulants can be used to estimate their model parameters within a scale factor. Both linear and nonlinear identification algorithms for fixed and time-varying order q(t) are presented. Statistical model order determination procedures are also derived. Implementation issues are discussed and resistance to noise is claimed when the signal of interest has cycles distinct from the additive noise. Simulations are performed to verify the theoretical results     

Adaptive importance sampling [digital communication]

Parametric adaptive importance sampling (IS) algorithms that adapt the IS density to the system of interest during the course of the simulation are discussed. This approach removes the burden of selecting the IS density from the system designer. The performance of two such algorithms is investigated for both linear and nonlinear systems operating in Gaussian noise. In addition, the algorithms are shown to converge to the optimum improved importance sampling density for the special case of a linear system with Gaussian noise     

Estimating second-order Volterra system parameters from noisy measurements based on an LMS variant or an errors-in-variables method

This paper deals with the identification of a nonlinear SISO system modelled by a second-order Volterra series expansion when both the input and the output are disturbed by additive white Gaussian noises. Two methods are proposed. Firstly, we present an unbiased on-line approach based on the LMS. It includes a bias correction scheme which requires the variance of the input additive noise. Secondly, we suggest solving the identification problem as an errors-in-variables issue, by means of the so-called Frisch scheme. Although its computational cost is high, this approach has the advantage of estimating the Volterra kernels and the variances of both the additive noises and the input signal, even if the signal-to-noise ratios at the input and the output are low.     

Stochastic analysis of gradient adaptive identification ofnonlinear systems with memory for Gaussian data and noisy input andoutput measurements

This paper investigates the statistical behavior of two gradient search adaptive algorithms for identifying an unknown nonlinear system comprised of a discrete-time linear system H followed by a zero-memory nonlinearity g(·). The input and output of the unknown system are corrupted by additive independent noises. Gaussian models are used for all inputs. Two competing adaptation schemes are analyzed. The first is a sequential adaptation scheme where the LMS algorithm is first used to estimate the linear portion of the unknown system. The LMS algorithm is able to identify the linear portion of the unknown system to within a scale factor. The weights are then frozen at the end of the first adaptation phase. Recursions are derived for the mean and fluctuation behavior of the LMS algorithm, which are in excellent agreement with Monte Carlo simulations. When the nonlinearity is modeled by a scaled error function, the second part of the sequential gradient identification scheme is shown to correctly learn the scale factor and the error function scale factor. Mean recursions for the scale factors show good agreement with Monte Carlo simulations. For slow learning, the stationary points of the gradient algorithm closely agree with the stationary points of the theoretical recursions. The second adaptive scheme simultaneously learns both the linear and nonlinear portions of the unknown channel. The mean recursions for the linear and nonlinear portions show good agreement with Monte Carlo simulations for slow learning. The stationary points of the gradient algorithm also agree with the stationary points of the theoretical recursions     

Partitioning: A unifying framework for adaptive systems, I: Estimation

In this paper, partitioning and the associated generalized partitioned estimation algorithms are shown to constitute a unifying and powerful framework for optimal adaptive estimation in linear as well as nonlinear problems. Using the partitioning framework, the adaptive estimation problem is treated from a global viewpoint that readily yields and unifies seemingly unrelated results and, most importantly, yields fundamentally new families of nonlinear and linear estimation algorithms in a decoupled parallel-realization form. The generalized partitioned estimation algorithms are shown to have several important properties from both a theoretical and a realization or computational standpoint.     

Adaptive filtering with decorrelation for coloured AR environments

The aim of this paper is to improve the convergence speed and steady state error of LMS-type adaptive algorithms for coloured and nonstationary signals such as in acoustic echo cancellation. The performance of these algorithms is limited by the eigenvalue spread of the correlation matrix of the input signal and also by the power of the additive noise. In this paper, the decorrelating adaptive algorithms are classified into four types: input-decorrelating, error-decorrelating, joint-prefiltering and a combination of joint-prefiltering and input-decorrelating. The last two types of algorithms are studied and guidelines are given to choose the proper algorithms based on the power spectral densities of the input signal and noise. For a prefiltering structure, it is proven that if the adaptive filter operates on any prefiltered pair of input and desired signal the optimal solution will remain unchanged. It is suggested that a new adaptive decorrelation prefilter be included that is designed to achieve two objectives simultaneously: to increase the speed of convergence by reducing the correlation between the prefiltered samples of the input; and to improve the tracking and the steady state performance by reducing the noise power in the prefiltered domain. Simulations and theoretical results confirm that the introduced auxiliary whitening processes improve the performance of the adaptive algorithms by jointly whitening the input and the error signal.