Fast and improved backpropagation learning of multi-layer artificial neural network using adaptive activation function

In the conventional backpropagation (BP) learning algorithm used for the training of the connecting weights of the artificial neural network (ANN), a fixed slope−based sigmoidal activation function is used. This limitation leads to slower training of the network because only the weights of different layers are adjusted using the conventional BP algorithm. To accelerate the rate of convergence during the training phase of the ANN, in addition to updates of weights, the slope of the sigmoid function associated with artificial neuron can also be adjusted by using a newly developed learning rule. To achieve this objective, in this paper, new BP learning rules for slope adjustment of the activation function associated with the neurons have been derived. The combined rules both for connecting weights and slopes of sigmoid functions are then applied to the ANN structure to achieve faster training. In addition, two benchmark problems: classification and nonlinear system identification are solved using the trained ANN. The results of simulation-based experiments demonstrate that, in general, the proposed new BP learning rules for slope and weight adjustments of ANN provide superior convergence performance during the training phase as well as improved performance in terms of root mean square error and mean absolute deviation for classification and nonlinear system identification problems.     

Pipelined functional link artificial recurrent neural network with the decision feedback structure for nonlinear channel equalization

This paper presents a computationally efficient nonlinear adaptive filter by a pipelined functional link artificial decision feedback recurrent neural network (PFLADFRNN) for the design of a nonlinear channel equalizer. It aims to reduce computational burden and improve nonlinear processing capabilities of the functional link artificial recurrent neural network (FLANN). The proposed equalizer consists of several simple small-scale functional link artificial decision feedback recurrent neural network (FLADFRNN) modules with less computational complexity. Since it is a module nesting architecture comprising a number of modules that are interconnected in a chained form, its performance can be further improved. Moreover, the equalizer with a decision feedback recurrent structure overcomes the unstableness thanks to its nature of infinite impulse response structure. Finally, the performance of the PFLADFRNN modules is evaluated by a modified real-time recurrent learning algorithm via extensive simulations for different linear and nonlinear channel models in digital communication systems. The comparisons of multilayer perceptron, FLANN and reduced decision feedback FLANN equalizers have clearly indicated the convergence rate, bit error rate, steady-state error and computational complexity, respectively, for nonlinear channel equalization.     

Nonlinear adaptive algorithms for equalisation in mobile satellite communications

This paper considers the equalisation problem in Quadrature Phase-Shift Keying (QPSK) modulated signals which have been distorted by the passage through a transmission channel. The channel is modelled as a Rician fading channel to simulate the behaviour of the transmission channel in the mobile satellite context. The equalisation is treated as the generalisation of the channel behaviour, and some algorithms with the structure of an artificial neural network using the Multilayer Perceptron, Volterra Series and Radial Basis Function are described. Results for the BER performance of typical transversal equalisers, with Square-Root Kalman adaptation algorithm, and algorithms with artificial neural network structure are also reported and evaluated. Improved performance is exhibited by the artificial neural network approaches.     

Neuro-fuzzy position control of demining tele-operation system based on RNN modeling

The paper considers the neuro-fuzzy position control of multi-finger robot hand in tele-operation system—an active master–slave hand system (MSHS) for demining. Recently, fuzzy control systems utilizing artificial intelligent techniques are also being actively investigated in robotic area. Neural network with their powerful learning capability are being sought as the basis for many adaptive control systems where on-line adaptation can be implemented. Fuzzy logic on the other hand has been proved to be rather popular in many control system applications providing a rule-base like structure. In this paper, the design and optimization process of fuzzy position controller is supported by learning techniques derived from neural network where a radial basis function (RBF) neural network is implemented to learn fuzzy rules and membership functions with predictor of recurrent neural network (RNN) model. The results of experiment show that based on the predictive capability of RNN model neuro-fuzzy controller with good adaptation and robustness capability can be designed.     

一种混合学习算法的研究

提出一种基于模糊逻辑和神经网络的自学习网络模型。该模型通过特定的学习算法训练样本，能自动生成模糊逻辑规则，调节输入、输出变量的隶属函数。同时，提出了一种结合自组织学习和ＢＰ学习的混合学习算法──ＢＰＳＯＭ，这种算法比通常的ＢＰ机学习算法收敛性好，速度快。     

一种混合学习算法

提出了一种基于模糊逻辑和神经网络的自学习网络模型和一种结合自组织学习和BP学习的BPSOM混合学习算法。该模型通过BPSOM算法训练样本，能自动生成模糊逻辑规则，调节输入、输出变量的隶属函数；而且该算法比通常的BP学习算法收敛性好、速度快.     

基于深度学习OFDM信道补偿技术硬件实现

为了解决部分高性能深度学习神经网络因存在复杂度高及计算量大等缺陷在嵌入式设备中应用效果不理想的问题;以小型化集成智能无线电设备AIR-T为平台实现了基于深度学习的OFDM信道补偿技术;在FPGA芯片上不仅实现了OFDM信号传输系统模块,也实现了传统信道估计与均衡模块,模块对数据进行预处理减轻神经网络工作量以完成神经网络信道补偿技术模块在Jetson TX2平台GPU上的高效实现;由实验记录神经网络训练过程中的计算复杂度和参数拟合速度得知,传统信道估计与均衡模块有效降低了网络训练时的运算次数;由测试性能方面可知,经过神经网络信道补偿后的数据误码率比之前传统信道估计与均衡后的误码率有明显降低;     

An improved recurrent neural network for M-PAM symbol detection

In this paper, a fully connected recurrent neural network (RNN) is presented for the recovery of M-ary pulse amplitude modulated (M-PAM) signals in the presence of intersymbol interference and additive white Gaussian noise. The network makes use of two different activation functions. One is the traditional two-level sigmoid function, which is used at its hidden nodes, and the other is the M-level sigmoid function (MSF), which is used at the output node. The shape of the M-level activation function is controlled by two parameters: the slope and shifting parameters. The effect of these parameters on the learning performance is investigated through extensive simulations. In addition, the network is compared with a linear transversal equalizer, a decision feedback equalizer and a recently proposed RNN equalizer which has used a scaled sigmoid function (SSF) at its output node. Comparisons are made in terms of their learning properties and symbol error rates. It is demonstrated that the proposed RNN equalizer performs better, provided that the MSF parameters are properly selected.     

Fault Diagnosis Based on the Fuzzy‐Recurrent Neural Network

A fuzzy‐recurrent neural network (FRNN) has been constructed by adding some feedback connections to a feedforward fuzzy neural network (FNN). The FRNN expands the modeling ability of a FNN in order to deal with temporal problems. A basic concept of the FRNN is first to use process or expert knowledge, including appropriate fuzzy logic rules and membership functions, to construct an initial structure and to then use parameter‐learning algorithms to fine‐tune the membership functions and other parameters. Its recurrent property makes it suitable for dealing with temporal problems, such as on‐line fault diagnosis. In addition, it also provides human‐understandable meaning to the normal feedforward multilayer neural network, in which the internal units are always opaque to users. In a word, the trained FRNN has good interpreting ability and one‐step‐ahead predicting ability. To demonstrate the performance of the FRNN in diagnosis, a comparison is made with a conventional feedforward network. The efficiency of the FRNN is verified by the results.     

A self-learning fuzzy logic controller using genetic algorithmswith reinforcements

This paper presents a new method for learning a fuzzy logic controller automatically. A reinforcement learning technique is applied to a multilayer neural network model of a fuzzy logic controller. The proposed self-learning fuzzy logic control that uses the genetic algorithm through reinforcement learning architecture, called a genetic reinforcement fuzzy logic controller, can also learn fuzzy logic control rules even when only weak information such as a binary target of “success” or “failure” signal is available. In this paper, the adaptive heuristic critic algorithm of Barto et al. (1987) is extended to include a priori control knowledge of human operators. It is shown that the system can solve more concretely a fairly difficult control learning problem. Also demonstrated is the feasibility of the method when applied to a cart-pole balancing problem via digital simulations     

A SELF-ORGANIZING QUANTUM NEURAL FUZZY NETWORK AND ITS APPLICATIONS

In this paper, a self-organizing quantum neural fuzzy network (QNFN) is proposed. The QNFN model is a four-layer structure. Layer 2 of the QNFN model contains quantum membership functions, which are multilevel activation functions. Each quantum membership function is composed of the sum of sigmoid functions shifted by quantum intervals. A self-constructing learning algorithm, which consists of the self-clustering algorithm (SCA) and the backpropagation algorithm, is also proposed. The proposed the SCA method is a fast, one-pass algorithm for a dynamic estimation of the number of clusters in an input data space. The backpropagation algorithm is used to tune the adjustable parameters. Simulation results were conducted to show the performance and applicability of the proposed model.     

Tuning of a neuro-fuzzy controller by genetic algorithm

Due to their powerful optimization property, genetic algorithms (GAs) are currently being investigated for the development of adaptive or self-tuning fuzzy logic control systems. This paper presents a neuro-fuzzy logic controller (NFLC) where all of its parameters can be tuned simultaneously by GA. The structure of the controller is based on the radial basis function neural network (RBF) with Gaussian membership functions. The NFLC tuned by GA can somewhat eliminate laborious design steps such as manual tuning of the membership functions and selection of the fuzzy rules. The GA implementation incorporates dynamic crossover and mutation probabilistic rates for faster convergence. A flexible position coding strategy of the NFLC parameters is also implemented to obtain near optimal solutions. The performance of the proposed controller is compared with a conventional fuzzy controller and a PID controller tuned by GA. Simulation results show that the proposed controller offers encouraging advantages and has better performance.     

Control of magnetic bearing systems via the Chebyshevpolynomial-based unified model (CPBUM) neural network

A Chebyshev polynomial-based unified model (CPBUM) neural network is introduced and applied to control a magnetic bearing systems. First, we show that the CPBUM neural network not only has the same capability of universal approximator, but also has faster learning speed than conventional feedforward/recurrent neural network. It turns out that the CPBUM neural network is more suitable in the design of controller than the conventional feedforward/recurrent neural network. Second, we propose the inverse system method, based on the CPBUM neural networks, to control a magnetic bearing system. The proposed controller has two structures; namely, off-line and on-line learning structures. We derive a new learning algorithm for each proposed structure. The experimental results show that the proposed neural network architecture provides a greater flexibility and better performance in controlling magnetic bearing systems.     

基于可变学习率BP算法的空调控制系统的研究

通过分析控制器参数学习率和控制器性能之间的关系,设计一种基于可变学习速率反向传播算法VLRBP和模糊神经元网络的变频空调控制系统.该系统不仅可以通过反传误差信号训练控制器参数,而且可以根据网络的当前状态朝最优化方向调整控制器参数的学习率.实验结果表明,该控制系统不仅比传统的空调PID控制器和模糊控制器具有更好的控制性能,而且相比基于标准BP算法和动量BP算法的模糊神经网络控制系统,也具有更快的收敛速度和更好的控制精确度.     

A one-layer recurrent neural network for support vector machine learning.

This paper presents a one-layer recurrent neural network for support vector machine (SVM) learning in pattern classification and regression. The SVM learning problem is first converted into an equivalent formulation, and then a one-layer recurrent neural network for SVM learning is proposed. The proposed neural network is guaranteed to obtain the optimal solution of support vector classification and regression. Compared with the existing two-layer neural network for the SVM classification, the proposed neural network has a low complexity for implementation. Moreover, the proposed neural network can converge exponentially to the optimal solution of SVM learning. The rate of the exponential convergence can be made arbitrarily high by simply turning up a scaling parameter. Simulation examples based on benchmark problems are discussed to show the good performance of the proposed neural network for SVM learning.     

A hybrid of electromagnetism-like mechanism and back-propagation algorithms for recurrent neural fuzzy systems design

This article introduces a novel hybrid evolutionary algorithm for recurrent fuzzy neural systems design in applications of nonlinear systems. The hybrid learning algorithm, IEMBP-improved electromagnetism-like (EM) with back-propagation (BP) technique, combines the advantages of EM and BP algorithms which provides high-speed convergence, higher accuracy and less computational complexity (computation time in seconds). In addition, the IEMBP needs only a small population to outperform the standard EM that uses a larger population. For a recurrent neural fuzzy system, IEMBP simulates the ‘attraction’ and ‘repulsion’ of charged particles by considering each neural system parameters as a charged particle. The EM algorithm is modified in such a way that the competition selection is adopted and the random neighbourhood local search is replaced by BP without evaluations. Thus, the IEMBP algorithm combines the advantages of multi-point search, global optimisation and faster convergence. Finally, several illustration examples for nonlinear systems are shown to demonstrate the performance and effectiveness of IEMBP.     

A novel fuzzy logic system based on N-version programming

For the consideration of different application systems, modeling the fuzzy logic rule, and deciding the shape of membership functions are very critical issues due to they play key roles in the design of fuzzy logic control system. This paper proposes a novel design methodology of fuzzy logic control system using the neural network and fault-tolerant approaches. The connectionist architecture with the learning capability of neural network and N-version programming development of a fault-tolerant technique are implemented in the proposed fuzzy logic control system. In other words, this research involves the modeling of parameterized membership functions and the partition of fuzzy linguistic variables using neural networks trained by the unsupervised learning algorithms. Based on the self-organizing algorithm, the membership function and partition of fuzzy class are not only derived automatically, but also the preconditions of fuzzy IF-THEN rules are organized. We also provide two examples, pattern recognition and tendency prediction, to demonstrate that the proposed system has a higher computational performance and its parallel architecture supports noise-tolerant capability. This generalized scheme is very satisfactory for pattern recognition and tendency prediction problems     

SoHyFIS-Yager: A self-organizing Yager based Hybrid neural Fuzzy Inference System

The Hybrid neural Fuzzy Inference System (HyFIS) is a multilayer adaptive neural fuzzy system for building and optimizing fuzzy models using neural networks. In this paper, the fuzzy Yager inference scheme, which is able to emulate the human deductive reasoning logic, is integrated into the HyFIS model to provide it with a firm and intuitive logical reasoning and decision-making framework. In addition, a self-organizing gaussian Discrete Incremental Clustering (gDIC) technique is implemented in the network to automatically form fuzzy sets in the fuzzification phase. This clustering technique is no longer limited by the need to have prior knowledge about the number of clusters present in each input and output dimensions. The proposed self-organizing Yager based Hybrid neural Fuzzy Inference System (SoHyFIS-Yager) introduces the learning power of neural networks to fuzzy logic systems, while providing linguistic explanations of the fuzzy logic systems to the connectionist networks. Extensive simulations were conducted using the proposed model and its performance demonstrates its superiority as an effective neuro-fuzzy modeling technique.     

Relationships between the A priori and A posteriori errors in nonlinear adaptive neural filters

The lower bounds for the a posteriori prediction error of a nonlinear predictor realized as a neural network are provided. These are obtained for a priori adaptation and a posteriori error networks with sigmoid nonlinearities trained by gradient-descent learning algorithms. A contractivity condition is imposed on a nonlinear activation function of a neuron so that the a posteriori prediction error is smaller in magnitude than the corresponding a priori one. Furthermore, an upper bound is imposed on the learning rate eta so that the approach is feasible. The analysis is undertaken for both feedforward and recurrent nonlinear predictors realized as neural networks.     

Rough Neural Network Based on Bottom-Up Fuzzy Rough Data Analysis

Based on bottom-up fuzzy rough data analysis, a new rough neural network decision-making model is proposed. Through supervised Gaustafason–Kessel (G–K) clustering algorithm, proper fuzzy clusters are found to partition the input data space. At the same time cluster number is searched by monotone increasing process. If the cluster number matches with that exactly exist in data sets then excellent fuzzy rough data modeling (FRDM) model can be built. And by integrating it with neural network technique, corresponding rough neural network is constructed. Our method overcomes the defects of conventional top-down based rough logic neural network (RLNN) method, and it also achieves adaptive learning ability and comprehensive soft decision-making ability compared with FRDM model. The experiment results indicate that our method has stronger generalization ability and more compact network structure than conventional RLNN.