An augmented extended Kalman filter algorithm for complex-valued recurrent neural networks

An augmented complex-valued extended Kalman filter (ACEKF) algorithm for the class of nonlinear adaptive filters realized as fully connected recurrent neural networks is introduced. This is achieved based on some recent developments in the so-called augmented complex statistics and the use of general fully complex nonlinear activation functions within the neurons. This makes the ACEKF suitable for processing general complex-valued nonlinear and nonstationary signals and also bivariate signals with strong component correlations. Simulations on benchmark and real-world complex-valued signals support the approach.     

Stochastic gradient-adaptive complex-valued nonlinear neural adaptive filters with a gradient-adaptive step size.

A class of variable step-size learning algorithms for complex-valued nonlinear adaptive finite impulse response (FIR) filters is proposed. To achieve this, first a general complex-valued nonlinear gradient-descent (CNGD) algorithm with a fully complex nonlinear activation function is derived. To improve the convergence and robustness of CNGD, we further introduce a gradient-adaptive step size to give a class of variable step-size CNGD (VSCNGD) algorithms. The analysis and simulations show the proposed class of algorithms exhibiting fast convergence and being able to track nonlinear and nonstationary complex-valued signals. To support the derivation, an analysis of stability and computational complexity of the proposed algorithms is provided. Simulations on colored, nonlinear, and real-world complex-valued signals support the analysis.     

A Data-Reusing Nonlinear Gradient Descent Algorithm for a Class of Complex-Valued Neural Adaptive Filters

A complex-valued data-reusing nonlinear gradient descent (CDRNGD) learning algorithm for a class of complex-valued nonlinear neural adaptive filters is introduced and the affinity between the family of data-reusing algorithms and the class of normalised gradient descent algorithms is examined. Error bounds on the class of complex data-reusing algorithms are established and indicate the stability of such algorithms. Experiments on nonlinear inputs show the class of complex data-reusing algorithms outperforming the standard complex nonlinear gradient descent algorithms and converging to the normalised complex non-linear gradient descent algorithm without experiencing the stability problems commonly encountered with normalised gradient descent algorithms. This revised version was published online in June 2006 with corrections to the Cover Date.     

Feedforward neural network for blind equalization with PSK signals

Most of the cost functions used for blind equalization are nonconvex and nonlinear functions of tap weights, when implemented using linear transversal filter structures. Therefore, a blind equalization scheme with a nonlinear structure that can form nonconvex decision regions is desirable. The efficacy of complex-valued feedforward neural networks for blind equalization of linear and nonlinear communication channels has been confirmed by many studies. In this paper we present a complex valued neural network for blind equalization with M-ary phase shift keying (PSK) signals. The complex nonlinear activation functions used in the neural network are especially defined for handling the M-ary PSK signals. The training algorithm based on constant modulus algorithm (CMA) cost function is derived. The improved performance of the proposed neural network in both, stationary and nonstationary environments, is confirmed through computer simulations.     

Symmetric Complex-Valued RBF Receiver for Multiple-Antenna-Aided Wireless Systems

A nonlinear beamforming assisted detector is proposed for multiple-antenna-aided wireless systems employing complex-valued quadrature phase shift-keying modulation. By exploiting the inherent symmetry of the optimal Bayesian detection solution, a novel complex-valued symmetric radial basis function (SRBF)-network-based detector is developed, which is capable of approaching the optimal Bayesian performance using channel-impaired training data. In the uplink case, adaptive nonlinear beamforming can be efficiently implemented by estimating the system's channel matrix based on the least squares channel estimate. Adaptive implementation of nonlinear beamforming in the downlink case by contrast is much more challenging, and we adopt a cluster-variation enhanced clustering algorithm to directly identify the SRBF center vectors required for realizing the optimal Bayesian detector. A simulation example is included to demonstrate the achievable performance improvement by the proposed adaptive nonlinear beamforming solution over the theoretical linear minimum bit error rate beamforming benchmark.      

滑动DCT-LMS自适应算法研究与FPGA实现

研究一种改进的低复杂度复数滑动离散余弦变换(DCT)最小均方(LMS)自适应算法,并设计该算法的FPGA实现结构。在常规LMS算法的输入端前添加改进的滑动DCT,降低输入信号之间的关联性,提高自适应算法收敛速度。改进的滑动DCT算法针对硬件实现进行了优化,提高其在硬件实现中稳定性和精度。给出算法在FPGA实现框图、结果和Matlab仿真结果的对比,以及算法在FPGA中的资源使用。算法已经在实际工程中应用,效果远优于常规LMS自适应算法。     

The complex local mean decomposition

The local mean decomposition (LMD) has been recently developed for the analysis of time series which have nonlinearity and nonstationarity. The smoothed local mean of the LMD surpasses the cubic spline method used by the empirical mode decomposition (EMD) to extract amplitude and frequency modulated components. To process complex-valued data, we propose complex LMD, a natural and generic extension to the complex domain of the original LMD algorithm. It is shown that complex LMD extracts the frequency modulated rotation and envelope components. Simulations on both artificial and real-world complex-valued signals support the analysis.     

Dynamic neural controllers for induction motor

The paper reports application of recently developed adaptive control techniques based on neural networks to the induction motor control. This case study represents one of the more difficult control problems due to the complex, nonlinear, and time-varying dynamics of the motor and unavailability of full-state measurements. A partial solution is first presented based on a single input-single output (SISO) algorithm employing static multilayer perceptron (MLP) networks. A novel technique is subsequently described which is based on a recurrent neural network employed as a dynamical model of the plant. Recent stability results for this algorithm are reported. The technique is applied to multiinput-multioutput (MIMO) control of the motor. A simulation study of both methods is presented. It is argued that appropriately structured recurrent neural networks can provide conveniently parameterized dynamic models for many nonlinear systems for use in adaptive control.     

Orthogonality of decision boundaries in complex-valued neural networks

This letter presents some results of an analysis on the decision boundaries of complex-valued neural networks whose weights, threshold values, input and output signals are all complex numbers. The main results may be summarized as follows. (1) A decision boundary of a single complex-valued neuron consists of two hypersurfaces that intersect orthogonally, and divides a decision region into four equal sections. The XOR problem and the detection of symmetry problem that cannot be solved with two-layered real-valued neural networks, can be solved by two-layered complex-valued neural networks with the orthogonal decision boundaries, which reveals a potent computational power of complex-valued neural nets. Furthermore, the fading equalization problem can be successfully solved by the two-layered complex-valued neural network with the highest generalization ability. (2) A decision boundary of a three-layered complex-valued neural network has the orthogonal property as a basic structure, and its two hypersurfaces approach orthogonality as all the net inputs to each hidden neuron grow. In particular, most of the decision boundaries in the three-layered complex-valued neural network inetersect orthogonally when the network is trained using Complex-BP algorithm. As a result, the orthogonality of the decision boundaries improves its generalization ability. (3) The average of the learning speed of the Complex-BP is several times faster than that of the Real-BP. The standard deviation of the learning speed of the Complex-BP is smaller than that of the Real-BP.It seems that the complex-valued neural network and the related algorithm are natural for learning complex-valued patterns for the above reasons.     

Adaptive Predistortion and Postdistortion for Nonlinear Channel

This paper proposes a new adaptive predistortion-postdistortion scheme based on a recurrent neural network to reduce nonlinear distortion introduced by a high power amplifier in the amplitude and phase of received Quadrature Phase Shift Keying (QPSK) signals in a digital microwave system. The recurrent neural network structure is inspired by the model proposed by Williams and Zipser, with a modified backpropagation algorithm. The input signal is processed by a nonlinear predistorter which reduces the warping effect. The received output signal is passed through a postdistorter to compensate for the warping and clustering effects produced by an amplifier. The proposed scheme yields a significant improvement when it is compared to the system without predistortion-postdistortion, perform-ance is evaluated in terms of the bit error rate and output signal constellation.     

Convolutive Blind Source Separation in the Frequency Domain Based on Sparse Representation

Convolutive blind source separation (CBSS) that exploits the sparsity of source signals in the frequency domain is addressed in this paper. We assume the sources follow complex Laplacian-like distribution for complex random variable, in which the real part and imaginary part of complex-valued source signals are not necessarily independent. Based on the maximum a posteriori (MAP) criterion, we propose a novel natural gradient method for complex sparse representation. Moreover, a new CBSS method is further developed based on complex sparse representation. The developed CBSS algorithm works in the frequency domain. Here, we assume that the source signals are sufficiently sparse in the frequency domain. If the sources are sufficiently sparse in the frequency domain and the filter length of mixing channels is relatively small and can be estimated, we can even achieve underdetermined CBSS. We illustrate the validity and performance of the proposed learning algorithm by several simulation examples.     

基于确定学习的机器人任务空间自适应神经网络控制

针对产生回归轨迹的连续非线性动态系统, 确定学习可实现未知闭环系统动态的局部准确逼近. 基于确定学习理论, 本文使用径向基函数(Radial basis function, RBF)神经网络为机器人任务空间跟踪控制设计了一种新的自适应神经网络控制算法, 不仅实现了闭环系统所有信号的最终一致有界, 而且在稳定的控制过程中, 沿着回归跟踪轨迹实现了部分神经网络权值收敛到最优值以及未知闭环系统动态的局部准确逼近. 学过的知识以时不变且空间分布的方式表达、以常值神经网络权值的方式存储, 可以用来改进系统的控制性能, 也可以应用到后续相同或相似的控制任务中, 节约时间和能量. 最后, 用仿真说明了所设计控制算法的正确性和有效性.     

A new learning algorithm with logarithmic performance index for complex-valued neural networks

In a fully complex-valued feed-forward network, the convergence of the Complex-valued Back Propagation (CBP) learning algorithm depends on the choice of the activation function, learning sample distribution, minimization criterion, initial weights and the learning rate. The minimization criteria used in the existing versions of CBP learning algorithm in the literature do not approximate the phase of complex-valued output well in function approximation problems. The phase of a complex-valued output is critical in telecommunication and reconstruction and source localization problems in medical imaging applications. In this paper, the issues related to the convergence of complex-valued neural networks are clearly enumerated using a systematic sensitivity study on existing complex-valued neural networks. In addition, we also compare the performance of different types of split complex-valued neural networks. From the observations in the sensitivity analysis, we propose a new CBP learning algorithm with logarithmic performance index for a complex-valued neural network with exponential activation function. The proposed CBP learning algorithm directly minimizes both the magnitude and phase errors and also provides better convergence characteristics. Performance of the proposed scheme is evaluated using two synthetic complex-valued function approximation problems, the complex XOR problem, and a non-minimum phase equalization problem. Also, a comparative analysis on the convergence of the existing fully complex and split complex networks is presented.     

Voiced/non-voiced speech classification using adaptive thresholding with bivariate EMD

This paper introduces a robust voiced/non-voiced (VnV) speech classification method using bivariate empirical mode decomposition (bEMD). Fractional Gaussian noise (fGn) is employed as the reference signal to derive a data adaptive threshold for VnV discrimination. The analyzing speech signal and fGn are combined to generate a complex signal which is decomposed into a finite number of complex-valued intrinsic mode functions (IMFs) by using bEMD. The real and imaginary parts of the IMFs represent the IMFs of observed speech and fGn, respectively. The log-energies of both types of IMFs are calculated. There exist similarities between the IMF log-energy representation of fGn and unvoiced speech signals. Hence, the upper confidence limit from IMF log-energies of fGn is used as data adaptive threshold for VnV classification. If the subband log-energy of speech segment exceeds the threshold, the segment is classified as voiced and unvoiced otherwise. The experimental results show that the proposed algorithm performs better than the recently reported methods without requiring any training data for a wide range of SNRs.     

Control design for arbitrary complex nonlinear discrete-time systems based on direct NNMRAC strategy

A novel scheme of neural network model reference adaptive control is proposed for arbitrary complex nonlinear discrete-time systems, i.e., non-minimum phase system, time-delay system and minimum phase system. An improved nearest neighbor clustering algorithm using an optimization strategy is introduced as the on-line learning algorithm to regulate the parameters of the RBFNN, which can simplify the neural network structure and accelerate the convergence speed. The clustering radius can be regulated automatically to guarantee the rationality of radius. Through constructing the pseudo-plant, the direct NNMRAC is also effective to the nonlinear non-minimum phase system. With the help of simulations, the control strategy based on direct RBFNN model reference adaptive control can not only make the multi-dimension nonlinear plants track multi-dimension reference signals quickly, but also endow the control systems with satisfying robustness.     

Sparse phase imaging based on complex domain nonlocal BM3D techniques

The paper is addressed to 2D phase and amplitude estimation of complex-valued signals – that is, in particular, to estimation of modulo-2π interferometric phase images from periodic and noisy observations. These degradation mechanisms make phase image estimation a challenging problem. A sparse nonlocal data-adaptive imaging formalized in complex domain is used for phase and amplitude image reconstruction. Following the procedure of patch-based technique, the image is partitioned into small overlapping square patches. Block Matching Three Dimensional (BM3D) technique is developed for forming complex domain sparse spectral representations of complex-valued data. High Order Singular Value Decomposition (HOSVD) applied to BM3D groups enables the design of the orthonormal complex domain 3D transforms which are data adaptive and different for each BM3Ds group. An iterative version of the complex domain BM3D is designed from variational formulation of the problem. The convergence of this algorithm is shown. The effectiveness of the new sparse coding based algorithms is illustrated in simulation experiments where they demonstrate the state-of-the-art performance.     

一种基于递归神经网络的自适应控制方法研究

本文针对快速、多变量、强非线性的复杂系统的控制问题,在强化学习方式的基础上,提出一种新的自适应控制方法。该方法在没有先验知识的条件下,基于递归神经网络并结合强化学习的自调节能力,通过自身神经网络的在线学习,有效控制不稳定的非线性系统。本文以一级倒立摆系统为实验对象,仿真实验结果表明:所提出的控制方法具有非常好的控制效果和稳定精度,抗干扰能力强。     

一种自适应预测非平稳信号的新方法

本文提出一种动态神经网络为非平稳信号作自适应单步预测，它由级联回归神经网络和抽头延时线组成．用它对非线性动态方程产生的时间序列作自适应预测，实验结果表明，其效果远远超过了传统的前馈神经网络．     

Pipelined Chebyshev Functional Link Artificial Recurrent Neural Network for Nonlinear Adaptive Filter

A novel nonlinear adaptive filter with pipelined Chebyshev functional link artificial recurrent neural network (PCFLARNN) is presented in this paper, which uses a modification real-time recurrent learning algorithm. The PCFLARNN consists of a number of simple small-scale Chebyshev functional link artificial recurrent neural network (CFLARNN) modules. Compared to the standard recurrent neural network (RNN), those modules of PCFLARNN can simultaneously be performed in a pipelined parallelism fashion, and this would lead to a significant improvement in its total computational efficiency. Furthermore, contrasted with the architecture of a pipelined RNN (PRNN), each module of PCFLARNN is a CFLARNN whose nonlinearity is introduced by enhancing the input pattern with Chebyshev functional expansion, whereas the RNN of each module in PRNN utilizing linear input and first-order recurrent term only fails to utilize the high-order terms of inputs. Therefore, the performance of PCFLARNN can further be improved at the cost of a slightly increased computational complexity. In addition, due to the introduced nonlinear functional expansion of each module in PRNN, the number of input signals can be reduced. Computer simulations have demonstrated that the proposed filter performs better than PRNN and RNN for nonlinear colored signal prediction, nonstationary speech signal prediction, and chaotic time series prediction.      

Adaptation of diagonal recurrent neural network model

An adaptive direct recurrent neural network model is developed for nonlinear dynamic system modelling in this paper. The model adaptation is achieved with the extended Kalman filter (EKF). A novel recursive algorithm is proposed to calculate the Jacobian matrix in the model adaptation so that the algorithm is simple and converges fast. The effectiveness of the developed adaptive model is demonstrated by applying to modelling a simulated continuous stirred tank reactor (CSTR). The model converges to the new process dynamics very quickly after a constant disturbance is added, and therefore can be used as an adaptive model in the adaptive model predictive control or internal model control for time-varying systems or fault tolerant control of nonlinear systems.