Training Reformulated Radial Basis Function Neural Networks Capable of Identifying Uncertainty in Data Classification

This paper introduces a learning algorithm that can be used for training reformulated radial basis function neural networks (RBFNNs) capable of identifying uncertainty in data classification. This learning algorithm trains a special class of reformulated RBFNNs, known as cosine RBFNNs, by updating selected adjustable parameters to minimize the class-conditional variances at the outputs of their radial basis functions (RBFs). The experiments verify that quantum neural networks (QNNs) and cosine RBFNNs trained by the proposed learning algorithm are capable of identifying uncertainty in data classification, a property that is not shared by cosine RBFNNs trained by the original learning algorithm and conventional feed-forward neural networks (FFNNs). Finally, this study leads to a simple classification strategy that can be used to improve the classification accuracy of QNNs and cosine RBFNNs by rejecting ambiguous feature vectors based on their responses.     

进化编程优化RBF神经网络的结构和参数

本文利用进化编程（ＦＰ）来同时进化径向基函数神经网络（ＲＢＦＮＮ）的结构和参数。与其它进化神经网络方法有以下四个方面的不同：（１）ＥＰ是基于拉马克的进化学说,强调父代与子代之间的行为联接;（２）进化算子中仅有突变,而没有交叉,以消除互换问题;（３）突变操作中,删除总是先于添加进行,以获得最简的网络结构;（４）利用测试样本集构造适应度函数,以提高网络的泛化能力。用进化ＲＢＦＮＮ来预测Ｍａｃｋｅｙ－Ｇ     

Bio-basis function neural network for prediction of protease cleavage sites in proteins

The prediction of protease cleavage sites in proteins is critical to effective drug design. One of the important issues in constructing an accurate and efficient predictor is how to present nonnumerical amino acids to a model effectively. As this issue has not yet been paid full attention and is closely related to model efficiency and accuracy, we present a novel neural learning algorithm aimed at improving the prediction accuracy and reducing the time involved in training. The algorithm is developed based on the conventional radial basis function neural networks (RBFNNs) and is referred to as a bio-basis function neural network (BBFNN). The basic principle is to replace the radial basis function used in RBFNNs by a novel bio-basis function. Each bio-basis is a feature dimension in a numerical feature space, to which a nonnumerical sequence space is mapped for analysis. The bio-basis function is designed using an amino acid mutation matrix verified in biology. Thus, the biological content in protein sequences can be maximally utilized for accurate modeling. Mutual information (MI) is used to select the most informative bio-bases and an ensemble method is used to enhance a decision-making process, hence, improving the prediction accuracy further. The algorithm has been successfully verified in two case studies, namely the prediction of Human Immunodeficiency Virus (HIV) protease cleavage sites and trypsin cleavage sites in proteins.     

Stable neurovisual servoing for robot manipulators

In this paper, we propose a stable neurovisual servoing algorithm for set-point control of planar robot manipulators in a fixed-camera configuration an show that all the closed-loop signals are uniformly ultimately bounded (UUB) and converge exponentially to a small compact set. We assume that the gravity term and Jacobian matrix are unknown. Radial basis function neural networks (RBFNNs) with online real-time learning are proposed for compensating both gravitational forces and errors in the robot Jacobian matrix. The learning rule for updating the neural network weights, similar to a back propagation algorithm, is obtained from a Lyapunov stability analysis. Experimental results on a two degrees of freedom manipulator are presented to evaluate the proposed controller.     

基于自适应扰动观测器的自主船舶协同路径跟踪控制

为实现未知环境扰动下不确定欠驱动自主船舶的协同路径跟踪控制, 本文提出了一种基于自适应扰动观 测器的鲁棒控制算法. 该算法采用径向基函数神经网络(RBFNNs)逼近模型参数不确定, 并利用最小学习参数化 (MLP)技术对神经网络的权重及逼近误差进行压缩, 所设计观测器不需要环境扰动上界的精确信息. 进一步, 基于 代数图论对船间通信进行建模, 设计了一种分散式协同控制律, 有效地降低了通信负载. 凭借Lyapunov稳定性理论 证明了闭环系统内信号的有界性, 且能通过对设计参数的调节使跟踪误差的收敛界为任意小. 最后采用数值仿真 试验验证了所提出算法的有效性和优越性.     

Studying possibility in a clustering algorithm for RBFNN design for function approximation

The function approximation problem has been tackled many times in the literature by using radial basis function neural networks (RBFNNs). In the design of these neural networks there are several stages where, the most critical stage is the initialization of the centers of each RBF since the rest of the steps to design the RBFNN strongly depend on it. The improved clustering for function approximation (ICFA) algorithm was recently introduced and proved successful for the function approximation problem. In the ICFA algorithm, a fuzzy partition of the input data is performed but, a fuzzy partition can behave inadequately in noise conditions. Possibilistic and mixed approaches, combining fuzzy and possibilistic partitions, were developed in order to improve the performance of a fuzzy partition. In this paper, a study of the influence of replacing the fuzzy partition used in the ICFA algorithm with the possibilistic and the fuzzy-possibilistic partitions will be done. A comparative analysis of each kind of partition will be performed in order to see if the possibilistic approach can improve the performance of the ICFA algorithm both in normal and in noise conditions. The results will show how the employment of a mixed approach combining fuzzy and possibilistic approach can lead to improve the results when designing RBFNNs.     

Artificial wavelet neural network and its application in neuro-fuzzy models

In the proposed work, two types of artificial neural networks are proposed by using well-known advantages and valuable features of wavelets and sigmoidal activation functions. Two neurons are derived by adding and multiplying the outputs of the wavelet and the sigmoidal activation functions. These neurons in a feed-forward single hidden layer network result summation wavelet neural network (SWNN) and multiplication wavelet neural network (MWNN). An algorithm is introduced for structure determination of the proposed networks. Approximation properties of SWNN and MWNN have been evaluated with different wavelet functions. The above networks in the consequent part of the neuro-fuzzy model result summation wavelet neuro-fuzzy (SWNF) and multiplication wavelet neuro-fuzzy (MWNF) models. Different types of wavelet function are tested with the proposed networks and fuzzy models on four different dynamical examples. Convergence of the learning process is also guaranteed by adaptive learning rate and performing stability analysis using Lyapunov function.     

Generalised Gaussian radial basis function neural networks

The mixed use of different shapes of radial basis functions (RBFs) in radial basis functions neural networks (RBFNNs) is investigated in this paper. For this purpose, we propose the use of a generalised version of the standard RBFNN, based on the generalised Gaussian distribution. The generalised radial basis function (GRBF) proposed in this paper is able to reproduce other different radial basis functions (RBFs) by changing a real parameter τ. In the proposed methodology, a hybrid evolutionary algorithm (HEA) is employed to estimate the number of hidden neuron, the centres, type and width of each RBF associated with each radial unit. In order to test the performance of the proposed methodology, an experimental study is presented with 20 datasets from the UCI repository. The GRBF neural network (GRBFNN) was compared to RBFNNs with Gaussian, Cauchy and inverse multiquadratic RBFs in the hidden layer and to other classifiers, including different RBFNN design methods, support vector machines (SVMs), a sparse probabilistic classifier (sparse multinominal logistic regression, SMLR) and other non-sparse (but regularised) probabilistic classifiers (regularised multinominal logistic regression, RMLR). The GRBFNN models were found to be better than the alternative RBFNNs for almost all datasets, producing the highest mean accuracy rank.     

Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation

This paper presents a multiobjective evolutionary algorithm to optimize radial basis function neural networks (RBFNNs) in order to approach target functions from a set of input-output pairs. The procedure allows the application of heuristics to improve the solution of the problem at hand by including some new genetic operators in the evolutionary process. These new operators are based on two well-known matrix transformations: singular value decomposition (SVD) and orthogonal least squares (OLS), which have been used to define new mutation operators that produce local or global modifications in the radial basis functions (RBFs) of the networks (the individuals in the population in the evolutionary procedure). After analyzing the efficiency of the different operators, we have shown that the global mutation operators yield an improved procedure to adjust the parameters of the RBFNNs.     

小波神经网络学习的结构风险最小化方法

针对大噪声、小样本情形下神经网络学习的外推能力弱这一突出的问题，根据统计学习理论中结构风险最小化准则的基本原理，提出了一种基于小波神经基元频率谱分布的小波神经网络阵列结构和基于小波多分辨逼近、综合风险分析的小波网络学习算法．该方法充分发挥了小波神经网络的优点，理论基础可靠，实际意义明确，算法实现简便，自适应性强．仿真实验结果和应用实例说明了该方法对于非线性系统在线辨识的有效性，同时也为统计学习理论的工程应用提供了新的途径．     

Application of temporal difference learning rules in short‐term traffic flow prediction

Studying dynamic behaviours of a transportation system requires the use of the system mathematical models as well as prediction of traffic flow in the system. Therefore, traffic flow prediction plays an important role in today's intelligent transportation systems. This article introduces a new approach to short‐term daily traffic flow prediction based on artificial neural networks. Among the family of neural networks, multi‐layer perceptron (MLP), radial basis function (RBF) neural network and wavenets have been selected as the three best candidates for performing traffic flow prediction. Moreover, back‐propagation (BP) has been adapted as the most efficient learning scheme in all the cases. It is shown that the coefficients produced by temporal signals improve the performance of the BP learning (BPL) algorithm. Temporal signals provide researchers with a new model of temporal difference BP learning algorithm (TDBPL). The capability and performance of TDBPL algorithm are examined by means of simulation in order to prove that the wavelet theory, with its multi‐resolution ability in comparison to RBF neural networks, is a suitable algorithm in traffic flow forecasting. It is also concluded that despite MLP applications, RBF neural networks do not provide negative forecasts. In addition, the local minimum problems are inevitable in MLP algorithms, while RBF neural networks and wavenet networks do not encounter them.     

Adaptive Control based on Extended Neural Network for SISO Uncertain Nonlinear Systems

This paper proposes a novel adaptive control criterion for a class of single-input-single-output (SISO) uncertain nonlinear systems by using extended neural networks (ENNs). Distinguished from the traditional neural networks, our ENNs are composed of radial basis function neural networks (RBFNNs), scalers and saturators. And these ENNs are used to approximate the uncertainties in the nonlinear systems. Based on the Lyapunov stability theory and our ENNs, an adaptive control scheme is designed to guarantee that all the signals in the closed-loop system are uniformly ultimately bounded (UUB). It is also worth pointing out that our control method makes the construction of RBFNNs and the design of adaptive laws separated, which means only the outputs of ENNs and one update law of the parameter in the scaler are to be adjusted. Thus, our control scheme can effectively reduce the online computation burden of the adaptive parameters. Finally, simulation examples are given to verify the effectiveness of our theoretical result.     

Prediction and identification using wavelet-based recurrent fuzzy neural networks

This paper presents a wavelet-based recurrent fuzzy neural network (WRFNN) for prediction and identification of nonlinear dynamic systems. The proposed WRFNN model combines the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and the wavelet neural networks (WNN). This paper adopts the nonorthogonal and compactly supported functions as wavelet neural network bases. Temporal relations embedded in the network are caused by adding some feedback connections representing the memory units into the second layer of the feedforward wavelet-based fuzzy neural networks (WFNN). An online learning algorithm, which consists of structure learning and parameter learning, is also presented. The structure learning depends on the degree measure to obtain the number of fuzzy rules and wavelet functions. Meanwhile, the parameter learning is based on the gradient descent method for adjusting the shape of the membership function and the connection weights of WNN. Finally, computer simulations have demonstrated that the proposed WRFNN model requires fewer adjustable parameters and obtains a smaller rms error than other methods.     

Adaptive tracking control of uncertain MIMO nonlinear systems with time-varying delays and unmodeled dynamics

In this paper, adaptive neural tracking control is proposed based on radial basis function neural networks (RBFNNs) for a class of multi-input multi-output (MIMO) nonlinear systems with completely unknown control directions, unknown dynamic disturbances, unmodeled dynamics, and uncertainties with time-varying delay. Using the Nussbaum function properties, the unknown control directions are dealt with. By constructing appropriate Lyapunov-Krasovskii functionals, the unknown upper bound functions of the time-varying delay uncertainties are compensated. The proposed control scheme does not need to calculate the integral of the delayed state functions. Using Young s inequality and RBFNNs, the assumption of unmodeled dynamics is relaxed. By theoretical analysis, the closed-loop control system is proved to be semi-globally uniformly ultimately bounded.     

An efficient multilayered shielded microwave circuit analysis method based on neural networks

In previous works, a neural network based technique to analyze multilayered shielded microwave circuits was developed. The method is based on the approximation of the shielded media Green's functions by radial‐basis‐function neural networks (RBFNNs). The trained neural networks, substitute the original Green's functions during the application of the integral equation approach, allowing a faster analysis than the direct solution. In this article, new and important improvements are applied to the training of the RBFNNs, which permit a reduction in the approximation error introduced by the neural networks. Furthermore, outstanding time reductions in the analysis of printed circuits are achieved, clearly outperforming the former technique. The main improvement consists on a better processing of the Green's function singularity near the source. The singularity produces rapid variations near the source that makes difficult the neural network training. In this work, the singularity is extracted in a more suitable fashion than in previous works. The functions resulting from the singularity extraction present a smooth behavior, so they can be easily approximated by neural networks. In addition, a new subdivision strategy for the input space is proposed to efficiently train the neural networks. Two practical microwave filters are analyzed using the new techniques. Comparisons with measured results are also presented for validation. © 2010 Wiley Periodicals, Inc. Int J RF and Microwave CAE, 2010.     

Adaptive neural control of non-affine pure-feedback non-linear systems with input nonlinearity and perturbed uncertainties

In this article, two robust adaptive control schemes are investigated for a class of completely non-affine pure-feedback non-linear systems with input non-linearity and perturbed uncertainties using radial basis function neural networks (RBFNNs). Based on the dynamic surface control (DSC) technique and using the quadratic Lyapunov function, the explosion of complexity in the traditional backstepping design is avoided when the gain signs are known. In addition, the unknown virtual gain signs are dealt with using the Nussbaum functions. Using the mean value theorem and Young's inequality, only one learning parameter needs to be tuned online at each step of recursion. It is proved that the proposed design method is able to guarantee semi-global uniform ultimate boundedness (SGUUB) of all signals in the closed-loop system. Simulation results verify the effectiveness of the proposed approach.     

Communication channel equalization using complex-valued minimalradial basis function neural networks

A complex radial basis function neural network is proposed for equalization of quadrature amplitude modulation (QAM) signals in communication channels. The network utilizes a sequential learning algorithm referred to as complex minimal resource allocation network (CMRAN) and is an extension of the MRAN algorithm originally developed for online learning in real-valued radial basis function (RBF) networks. CMRAN has the ability to grow and prune the (complex) RBF network's hidden neurons to ensure a parsimonious network structure. The performance of the CMRAN equalizer for nonlinear channel equalization problems has been evaluated by comparing it with the functional link artificial neural network (FLANN) equalizer of J.C. Patra et al. (1999) and the Gaussian stochastic gradient (SG) RBF equalizer of I. Cha and S. Kassam (1995). The results clearly show that CMRANs performance is superior in terms of symbol error rates and network complexity.     

基于遗传算法的径向基神经网络及其在同时测定Fe Mn Cu Zn中应用

介绍了两种新的基于遗传算法的径向基神经网络(GA-Based RBFNN)训练算法.这两种算法均将遗传算法用于优化径向基神经网络的聚类中心和网络结构.第一种GA-Based RBFNN算法对所有训练样本采取二进制编码构成个体,优化径向基函数中心的选取和网络结构;第二种GA-Based RBFNN算法中,RBFNN采用自增长算法训练网络隐含层中心、采用十进制对距离因子ε编码构成染色体,优化网络.将两种GA-Based RBFNN算法应用于Fe、Mn、Cu、Zn同时测定的光谱解析,计算结果表明,本文的GA-Based RBFNN算法较通常的遗传算法与径向基人工神经网络(GA-RBFNN)联用,即在GA选择变量的基础上,再用RBFNN作数据解析的GA-RBFNN方法,在增强网络的泛化能力、提高预测的准确性等方面具有明显的优势.从这两种GA-Based RBFNN的比较看,第二种算法在性能上优于第一种算法.     

A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation

This work presents a new sequential learning algorithm for radial basis function (RBF) networks referred to as generalized growing and pruning algorithm for RBF (GGAP-RBF). The paper first introduces the concept of significance for the hidden neurons and then uses it in the learning algorithm to realize parsimonious networks. The growing and pruning strategy of GGAP-RBF is based on linking the required learning accuracy with the significance of the nearest or intentionally added new neuron. Significance of a neuron is a measure of the average information content of that neuron. The GGAP-RBF algorithm can be used for any arbitrary sampling density for training samples and is derived from a rigorous statistical point of view. Simulation results for bench mark problems in the function approximation area show that the GGAP-RBF outperforms several other sequential learning algorithms in terms of learning speed, network size and generalization performance regardless of the sampling density function of the training data.     

Adaptive actor-critic learning for the control of mobile robots by applying predictive models

In this paper, we propose two methods of adaptive actor-critic architectures to solve control problems of nonlinear systems. One method uses two actual states at time k and time k+1 to update the learning algorithm. The basic idea of this method is that the agent can directly take some knowledge from the environment to improve its knowledge. The other method only uses the state at time k to update the algorithm. This method is called, learning from prediction (or simulated experience). Both methods include one or two predictive models, which are assumed to be applied to construct predictive states and a model-based actor (MBA). Here, the MBA as an actor can be viewed as a network where the connection weights are the elements of the feedback gain matrix. In the critic part, two value-functions are realized as a pure static mapping, which can be reduced to a nonlinear current estimator by using the radial basis function neural networks (RBFNNs). Simulation results obtained for a dynamical model of nonholonomic mobile robots with two independent driving wheels are presented. They show the effectiveness of the proposed approaches for the trajectory tracking control problem.