Fast adaptive digital equalization by recurrent neural networks

Neural networks (NNs) have been extensively applied to many signal processing problems. In particular, due to their capacity to form complex decision regions, NNs have been successfully used in adaptive equalization of digital communication channels. The mean square error (MSE) criterion, which is usually adopted in neural learning, is not directly related to the minimization of the classification error, i.e., bit error rate (BER), which is of interest in channel equalization. Moreover, common gradient-based learning techniques are often characterized by slow speed of convergence and numerical ill conditioning. In this paper, we introduce a novel approach to learning in recurrent neural networks (RNNs) that exploits the principle of discriminative learning, minimizing an error functional that is a direct measure of the classification error. The proposed method extends to RNNs a technique applied with success to fast learning of feedforward NNs and is based on the descent of the error functional in the space of the linear combinations of the neurons (the neuron space); its main features are higher speed of convergence and better numerical conditioning w.r.t. gradient-based approaches, whereas numerical stability is assured by the use of robust least squares solvers. Experiments regarding the equalization of PAM signals in different transmission channels are described, which demonstrate the effectiveness of the proposed approach     

Variable neural networks for adaptive control of nonlinear systems

This paper is concerned with the adaptive control of continuous-time nonlinear dynamical systems using neural networks. A novel neural network architecture, referred to as a variable neural network, is proposed and shown to be useful in approximating the unknown nonlinearities of dynamical systems. In the variable neural networks, the number of basis functions can be either increased or decreased with time, according to specified design strategies, so that the network will not overfit or underfit the data set. Based on the Gaussian radial basis function (GRBF) variable neural network, an adaptive control scheme is presented. The location of the centers and the determination of the widths of the GRBFs in the variable neural network are analyzed to make a compromise between orthogonality and smoothness. The weight-adaptive laws developed using the Lyapunov synthesis approach guarantee the stability of the overall control scheme, even in the presence of modeling error(s). The tracking errors converge to the required accuracy through the adaptive control algorithm derived by combining the variable neural network and Lyapunov synthesis techniques. The operation of an adaptive control scheme using the variable neural network is demonstrated using two simulated examples     

A partitioned adaptive approach to nonlinear channel equalization

The problem of identifying digital transmitted symbols over nonlinear communication channels is addressed. The equalization scenario is considered from the decision point of view, and constitutes a joint identification and estimation situation due to incomplete knowledge of the system model. A new class of multilinearization algorithms for nonlinear systems is derived according to partitioning theory concepts. The procedure targets on adaptively selecting the best reference points for linearization from an ensemble of generated trajectories that span the whole state space of the desired signal. In the various simulations examined, the partitioned-based equalizer is found superior to the classical extended Kalman filter     

Functional link neural network cascaded with Chebyshev orthogonal polynomial for nonlinear channel equalization

Nonlinear intersymbol interference (ISI) leads to significant error rate in nonlinear communication and digital storage channel. In this paper, therefore, a novel computationally efficient functional link neural network cascaded with Chebyshev orthogonal polynomial is proposed to combat nonlinear ISI. The equalizer has a simple structure in which the nonlinearity is introduced by functional expansion of the input pattern by trigonometric polynomial and Chebyshev orthogonal polynomial. Due to the input pattern and nonlinear approximation enhancement, the proposed structure can approximate arbitrarily nonlinear decision boundaries. It has been utilized for nonlinear channel equalization. The performance of the proposed adaptive nonlinear equalizer is compared with functional link neural network (FLNN) equalizer, multilayer perceptron (MLP) network and radial basis function (RBF) along with conventional normalized least-mean-square algorithms (NLMS) for different linear and nonlinear channel models. The comparison of convergence rate, bit error rate (BER) and steady state error performance, and computational complexity involved for neural network equalizers is provided.     

Fuzzy neural networks for direct adaptive control

It is well known that sliding-mode control is simple and insensitive to uncertainties and disturbances. However, control input chattering is the main problem of the classical sliding-mode controller (SMC). In this paper, a fuzzy neural network SMC (FNNSMC) is presented for a class of nonlinear systems. The FNNSMC can eliminate the chattering, unlike the SMC, but there is larger rising time in the FNNSMC than in the SMC. In some cases, small rise time is important. To decrease the rising time of the FNNSMC, an adaptive controller is proposed where the SMC and the FNNSMC are incorporated by a smooth transformation. This adaptive control scheme can improve the dynamical performance and eliminate the high-frequency chattering in the control signal. The system stability is proved by using the Lyapunov function. The simulation results demonstrate the advantages of the proposed adaptive controller.     

Conditional distribution learning with neural networks and itsapplication to channel equalization

We present a conditional distribution learning formulation for real-time signal processing with neural networks based on an extension of maximum likelihood theory-partial likelihood (PL) estimation-which allows for (i) dependent observations and (ii) sequential processing. For a general neural network conditional distribution model, we establish a fundamental information-theoretic connection, the equivalence of maximum PL estimation, and accumulated relative entropy (ARE) minimization, and obtain large sample properties of PL for the general case of dependent observations. As an example, the binary case with the sigmoidal perceptron as the probability model is presented. It is shown that the single and multilayer perceptron (MLP) models satisfy conditions for the equivalence of the two cost functions: ARE and negative log partial likelihood. The practical issue of their gradient descent minimization is then studied within the well-formed cost functions framework. It is shown that these are well-formed cost functions for networks without hidden units; hence, their gradient descent minimization is guaranteed to converge to a solution if one exists on such networks. The formulation is applied to adaptive channel equalization, and simulation results are presented to show the ability of the least relative entropy equalizer to realize complex decision boundaries and to recover during training from convergence at the wrong extreme in cases where the mean square error-based MLP equalizer cannot     

Optimal nonlinear adaptive prediction and modeling of MPEG video inATM networks using pipelined recurrent neural networks

This paper investigates the application of a pipelined recurrent neural network (PRNN) to the adaptive traffic prediction of MPEG video signal via dynamic ATM networks. The traffic signal of each picture type (I, P, and B) of MPEG video is characterized by a general nonlinear autoregressive moving average (NARMA) process. Moreover, a minimum mean-squared error predictor based on the NARMA model is developed to provide the best prediction for the video traffic signal. However, the explicit functional expression of the best mean-squared error predictor is actually unknown. To tackle this difficulty, a PRNN that consists of a number of simpler small-scale recurrent neural network (RNN) modules with less computational complexity is conducted to introduce the best nonlinear approximation capability into the minimum mean-squared error predictor model in order to accurately predict the future behavior of MPEG video traffic in a relatively short time period based on adaptive learning for each module from previous measurement data, in order to provide faster and more accurate control action to avoid the effects of excessive load situation. Since those modules of PRNN can be performed simultaneously in a pipelined parallelism fashion, this would lead to a significant improvement in the total computational efficiency of PRNN. In order to further improve the convergence performance of the adaptive algorithm for PRNN, a learning-rate annealing schedule is proposed to accelerate the adaptive learning process. Another advantage of the PRNN-based predictor is its generalization from learning that is useful for learning a dynamic environment for MPEG video traffic prediction in ATM networks where observations may be incomplete, delayed, or partially available. The PRNN-based predictor presented in this paper is shown to be promising and practically feasible in obtaining the best adaptive prediction of real-time MPEG video traffic     

Adaptive decision feedback equalization for digital satellitechannels using multilayer neural networks

This paper introduces an adaptive derision feedback equalization using the multilayer perceptron structure of an M-ary PSK signal through a TDMA satellite radio channel. The transmission is disturbed not only by intersymbol interference (ISI) and additive white Gaussian noise, but also by the nonlinearity of transmitter amplifiers. The conventional decision feedback equalizer (DFE) is not well-suited to detect the transmitted sequence, whereas the neural-based DFE is able to take into account the nonlinearities and therefore to detect the signal much better. Nevertheless, the applications of the traditional multilayer neural networks have been limited to real-valued signals. To overcome this difficulty, a neural-based DFE is proposed to deal with the complex PSK signal over the complex-valued nonlinear MPSK satellite channel without performing time-consuming complex-valued back-propagation training algorithms, while maintaining almost the same computational complexity as the original real-valued training algorithm. Moreover, a modified back-propagation algorithm with better convergence properties is derived on the basis of delta-bar-delta rule. Simulation results for the equalization of QPSK satellite channels show that the neural-based DFE provides a superior bit error rate performance relative to the conventional mean square DFE, especially in poor signal-to-noise ratio conditions     

Digitally programmable nonlinear function generator for neural networks

The design of a new digitally programmable analogue circuit well suited for the implementation of several sets of nonlinear functions by approximating them by using a linear combination of sigmoidal terms is presented. The proposed circuit, allowing the building of several collections of nonlinear functions, would be useful in modelling artificial neural networks, fuzzy as well as partial differential equations based circuits     

Artificial neural networks for nonlinear time-domain filtering ofspeech

A multilayer perceptron (MLP) is applied as a time domain nonlinear filter to two classes of degraded speech, namely Gaussian white noise and nonlinear system degradation introduced by a low bit-rate CELP coder. The goal of the study is to examine the influence of the inherent nonlinearity within the MLP, and this is achieved by varying the levels of nonlinearity within the structure. Direct comparisons of MLPs and linear filters show that with CELP degradation the SNR improvements achieved by the MLP is measurably better than with an equivalent linear structure (3 dB cf 1.5 dB) but when the degradation is additive noise the two structures perform equally well. The study highlights the importance of scaling to achieve optimum performance, and of matching the enhancer to the degradation     

Stability in contractive nonlinear neural networks

We consider models of the form mu chi = -x + p + WF(x) where x = x(t) is a vector whose entries represent the electrical activities in the units of a neural network. W is a matrix of synaptic weights, F is a nonlinear function, and p is a vector (constant or slowly varying over time) of inputs to the units. If the map WF(x) is a contraction, then the system has a unique equilibrium which is globally asymptotically stable; consequently the network acts as a stable encoder in that its steady-state response to an input is independent of the initial state of the network. We consider some relatively mild restrictions on W and F(x), involving the eigenvalues of W and the derivative of F, that are sufficient to ensure that WF(x) is a contraction. We show that in the linear case with spatially-homogeneous synaptic weight, the eigenvalues of W are simply related to the Fourier transform of the connection pattern. This relation makes it possible, given cortical activity patterns as measured by autoradiographic labeling, to construct a pattern of synaptic weights which produces steady state patterns showing similar frequency characteristics. Finally, we consider the relationships, in the spatial and frequency domains, between the equilibrium of the model and that of the linear approximation mu chi = -x + p + Wx; this latter equilibrium can be computed easily from p in the homogeneous case using discrete Fourier transforms.     

Iterative equalization with adaptive soft feedback

In this letter, a novel equalization algorithm applying soft-decision feedback and designed for binary transmission is introduced. In contrast to conventional decision-feedback equalization (DFE), iterations are necessary, because a simple matched filter serves as feedforward filter, which collects signal energy, but creates noncausal intersymbol interference. The rule for generating soft decisions is adapted continuously to the current state of the algorithm. In most cases, standard DFE methods are clearly outperformed. For a class of certain channel impulse responses, performance of maximum-likelihood sequence estimation is attained, in principle. The high performance of the scheme is explained using results from neural network theory     

ECG analysis using nonlinear PCA neural networks for ischemiadetection

The detection of ischemic cardiac beats from a patient's electrocardiogram (EGG) signal is based on the characteristics of a specific part of the beat called the ST segment. The correct classification of the beats relies heavily on the efficient and accurate extraction of the ST segment features. An algorithm is developed for this feature extraction based on nonlinear principal component analysis (NLPCA). NLPCA is a method for nonlinear feature extraction that is usually implemented by a multilayer neural network. It has been observed to have better performance, compared with linear principal component analysis (PCA), in complex problems where the relationships between the variables are not linear. In this paper, the NLPCA techniques are used to classify each segment into one of two classes: normal and abnormal (ST+, ST-, or artifact). During the algorithm training phase, only normal patterns are used, and for classification purposes, we use only two nonlinear features for each ST segment. The distribution of these features is modeled using a radial basis function network (RBFN). Test results using the European ST-T database show that using only two nonlinear components and a training set of 1000 normal samples from each file produce a correct classification rate of approximately 80% for the normal beats and higher than 90% for the ischemic beats     

Minimal radial basis function neural networks for nonlinear channelequalisation

The results of linear and nonlinear channel equalisation in data communications are presented, using a previously developed minimal radial basis function neural network structure, referred to as the minimal resource allocation network (MRAN). The MRAN algorithm uses online learning, and has the capability to grow and prune the RBF network's hidden neurons ensuring a parsimonious network structure. Compared to earlier methods, the proposed scheme does not have to estimate the channel order first, and fix the model parameters. Results showing the superior performance of the MRAN algorithm for two linear channels (minimum and non-minimum phase) for 2PAM signalling, and three nonlinear channels for 2PAM and 4QAM signalling, are presented     

Cellular neural networks: a paradigm for nonlinear spatio-temporalprocessing

The paradigm of Cellular Neural Networks (CNNs)is going to achieve a complete maturity. In fact, from a methodological point of view, important results on their digitally programmable analog dynamics have been gained, completed with thousands of application routines. This has encouraged the spreading of a great number of applications in the most different disciplines. Moreover, their structure, tailor made for VLSI realization, has led to the production of some chip prototypes that, embedded in a computational infrastructure, have produced the first analogic cellular computers. This completes the framework and makes it possible to realize complex spatio-temporal and filtering tasks on a time scale of microseconds. In this paper some sketches on the main aspects of CNNs, from the formal to the hardware prototype point of view, are presented together with some appealing applications to illustrate complex image, visual and spatio-temporal dynamics processing     

平均自适应滤波的信道均衡算法研究

近年来数字传输系统的信道均衡侧重于训练时间的缩短和跟踪速度的加快,需要研究快速收敛的自适应算法。从这点考虑递归最小二乘(RLS)均衡器是最佳的选择,但RLS算法的运算非常复杂而且存在稳定性问题,因而有必要研究一种能够代替传统RLS的算法。在本文中介绍一种基于平均自适应滤波(AFA)算法的均衡器,其主要优点是与RLS算法相当的快速收敛速度,同时运算复杂度较低。     

输出不可量测非线性系统的神经模型参考自适应控制

该文针对被控对象输出不可量测的非线性系统,引入一个便于在线辨识的扩展神经网络模型,提出一种基于前馈-反馈结构的神经网络模型参考自适应控制方法。给出了具有全局收敛性的网络训练算法,并分析了控制系统的稳定性。仿真结果表明该控制方法是有效的,而且对网络初始权值的选取及被控对象特性参数的扰动都具有良好的鲁棒性。     

Performance of adaptive equalization for indoor radiocommunications

Adaptive equalization is used to ensure that the outage probability is less than 10^-3 for a target bit error rate of 10^-4 in buildings with RMS delay spread of up to 100 ns. A time-division multiple-access system with four-level quadrature amplitude modulation point-to-point links strikes the right balance between flexibility and complexity. It is shown that such a system can support rates of at least 1 Mb/s     

一种较低复杂度的UWB信道自适应均衡技术

针对多用户UWB信道存在的符号间干扰和用户间干扰问题,提出了一种用于DS-UWB/TH-UWB接收机的较少复杂度的自适应均衡技术及相应算法,并与传统算法在复杂性和性能方面进行了比较。研究结果表明所提出的算法在运算量上远小于单独使用RLS算法,在输出误差的收敛上远快于单独使用LMS算法。