A Generalized Ellipsoidal Basis Function Based Online Self-constructing Fuzzy Neural Network

In this paper, we propose a Generalized ellipsoidal basis function based online self-constructing fuzzy neural network (GEBF-OSFNN) which extends the ellipsoidal basis function (EBF)-based fuzzy neural networks (FNNs) by permitting input variables to be modeled by dissymmetrical Gaussian functions (DGFs). Due to the flexibility and dissymmetry of left and right widths of the DGF, the partitioning made by DGFs in the input space is more flexible and more interpretable, and therefore results in a parsimonious FNN with high performance under the online learning algorithm. The geometric growing criteria and the error reduction ratio (ERR) method are used as growing and pruning strategies respectively to realize the structure learning algorithm which implements an optimal and compact network structure. The GEBF-OSFNN starts with no hidden neurons and does not need to partition the input space a priori. In addition, all free parameters in premises and consequents are adjusted online based on the ε-completeness of fuzzy rules and the linear least square (LLS) approach, respectively. The performance of the GEBF-OSFNN paradigm is compared with other well-known algorithms like RAN, RANEKF, MRAN, ANFIS, OLS, RBF-AFS, DFNN, GDFNN GGAP-RBF, OS-ELM, SOFNN and FAOS-PFNN, etc., on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world benchmark problems. Simulation results demonstrate that the proposed GEBF-OSFNN approach can facilitate a more powerful and more parsimonious FNN with better performance of approximation and generalization.     

基于EKF的模糊神经网络快速自组织学习算法*

为了快速地构造一个有效的模糊神经网络,提出一种基于扩展卡尔曼滤波(EKF)的模糊神经网络自组织学习算法。在本算法中,按照提出的无须经过修剪过程的生长准则增加规则,加速了网络在线学习过程;使用EKF算法更新网络的自由参数,增强了网络的鲁棒性。仿真结果表明,该算法具有快速的学习速度、良好的逼近精度和泛化能力。     

基于信息强度的RBF神经网络结构设计研究

在系统研究前馈神经网络的基础上,针对径向基函数(Radial basis function, RBF) 网络的结构设计问题,提出一种弹性RBF神经网络结构优化设计方法. 利用隐含层神经元的输出信息(Output-information, OI)以及隐含层神经元与输出层神经元间的交互信息(Multi-information, MI)分析网络的连接强度, 以此判断增加或删除RBF神经网络隐含层神经元, 同时调整神经网络的拓扑结构,有效地解决了RBF神经网络结构设计问题; 利用梯度下降的参数修正算法保证了最终RBF网络的精度, 实现了神经网络的结构和参数自校正. 通过对典型非线性函数的逼近与污水处理过程关键水质参数建模, 结果证明了该弹性RBF具有良好的动态特征响应能力和逼近能力, 尤其是在训练速度、泛化能力、最终网络结构等方面较之最小资源神经网络(Minimal resource allocation net works, MRAN)、增长修剪RBF 神经网络(Generalized growing and pruning RBF, GGAP-RBF)和自组织RBF神经网络(Self-organizing RBF, SORBF)有较大的提高.     

Performance evaluation of a sequential minimal radial basisfunction (RBF) neural network learning algorithm

Presents a detailed performance analysis of the minimal resource allocation network (M-RAN) learning algorithm, M-RAN is a sequential learning radial basis function neural network which combines the growth criterion of the resource allocating network (RAN) of Platt (1991) with a pruning strategy based on the relative contribution of each hidden unit to the overall network output. The resulting network leads toward a minimal topology for the RAN. The performance of this algorithm is compared with the multilayer feedforward networks (MFNs) trained with 1) a variant of the standard backpropagation algorithm, known as RPROP and 2) the dependence identification (DI) algorithm of Moody and Antsaklis (1996) on several benchmark problems in the function approximation and pattern classification areas. For all these problems, the M-RAN algorithm is shown to realize networks with far fewer hidden neurons with better or same approximation/classification accuracy. Further, the time taken for learning (training) is also considerably shorter as M-RAN does not require repeated presentation of the training data.     

RBF神经网络的结构动态优化设计

针对径向基函数(Radial basis function, RBF)神经网络的结构设计问题, 提出一种结构动态优化设计方法. 利用敏感度法(Sensitivity analysis, SA)分析隐含层神经元的输出加权值对神经网络输出的影响, 以此判断增加或删除RBF神经网络隐含层中的神经元, 解决了RBF神经网络结构过大或过小的问题, 并给出了神经网络结构动态变化过程中收敛性证明; 利用梯度下降的参数修正算法保证了最终RBF网络的精度, 实现了神经网络的结构和参数自校正. 通过对非线性函数的逼近与污水处理过程中关键参数的建模结果, 证明了该动态RBF具有良好的自适应能力和逼近能力, 尤其是在泛化能力、最终网络结构等方面较之最小资源神经网络(Minimal resource allocation networks, MRAN)与增长和修剪RBF 神经网络(Generalized growing and pruning radial basis function, GGAP-RBF) 有较大提高.     

基于UKF的增长型模糊神经网络设计

为了实现模糊神经网络结构和参数的同时调整,提出一种基于无迹卡尔曼滤波(UKF)的增长型模糊神经网络(UKF-GFNN).首先,利用UKF对模糊神经网络的参数进行调整;然后,设计一种基于隐含层神经元输出强度的模糊规则增长机制,实现模糊神经网络的结构增长;最后,将所提出的增长型模糊神经网络应用于非线性系统建模.实验结果显示,基于UKF的增长型模糊神经网络能够实现结构和参数的自校正,并且具有较高的建模精度.     

Self-organizing adaptive fuzzy neural control for the synchronization of uncertain chaotic systems with random-varying parameters

This paper proposes a self-organizing adaptive fuzzy neural control (SAFNC) for the synchronization of uncertain chaotic systems with random-varying parameters. The proposed SAFNC system is composed of a computation controller and a robust controller. The computation controller containing a self-organizing fuzzy neural network (SOFNN) identifier is the principle controller. The SOFNN identifier is used to online estimate the compound uncertainties with the structure and parameter learning phases of fuzzy neural network (FNN), simultaneously. The structure-learning phase consists of the growing of membership functions, the splitting of fuzzy rules and the pruning of fuzzy rules, and thus the SOFNN identifier can avoid the time-consuming trial-and-error tuning procedure for determining the network structure of fuzzy neural network. The robust controller is used to attenuate the effects of the approximation error so that the synchronization of chaotic systems is achieved.All the parameter learning algorithms are derived based on the Lyapunov stability theorem to ensure network convergence as well as stable synchronization performance. To demonstrate the effectiveness of the proposed method, simulation results are illustrated in this paper.     

Online identification of nonlinear multivariable processes using self‐generating RBF neural networks

This paper addresses the problem of online model identification for multivariable processes with nonlinear and time‐varying dynamic characteristics. For this purpose, two online multivariable identification approaches with self‐organizing neural network model structures will be presented. The two adaptive radial basis function (RBF) neural networks are called as the growing and pruning radial basis function (GAP‐RBF) and minimal resource allocation network (MRAN). The resulting identification algorithms start without a predefined model structure and the dynamic model is generated autonomously using the sequential input‐output data pairs in real‐time applications. The extended Kalman filter (EKF) learning algorithm has been extended for both of the adaptive RBF‐based neural network approaches to estimate the free parameters of the identified multivariable model. The unscented Kalman filter (UKF) has been proposed as an alternative learning algorithm to enhance the accuracy and robustness of nonlinear multivariable processes in both the GAP‐RBF and MRAN based approaches. In addition, this paper intends to study comparatively the general applicability of the particle filter (PF)‐based approaches for the case of non‐Gaussian noisy environments. For this purpose, the Unscented Particle Filter (UPF) is employed to be used as alternative to the EKF and UKF for online parameter estimation of self‐generating RBF neural networks. The performance of the proposed online identification approaches is evaluated on a highly nonlinear time‐varying multivariable non‐isothermal continuous stirred tank reactor (CSTR) benchmark problem. Simulation results demonstrate the good performances of all identification approaches, especially the GAP‐RBF approach incorporated with the UKF and UPF learning algorithms. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

基于ART 的RBF 网络结构设计

针对径向基函数(RBF)网络隐层结构难以确定的问题,基于自适应共振理论(ART)网络良好的在线分类特性,提出一种RBF网络结构设计算法。该算法将ART网络的聚类特性用于RBF网络结构设计中,通过对输入向量与已存模式的相似度比较将输入向量进行分类,确定隐含层节点个数和初始参数,使网络具有精简的结构。对典型非线性函数逼近的仿真结果表明,所提出的结构具有快速的学习能力和良好的逼近能力。     

An online pruning strategy for supervised ARTMAP-based neural networks

Identifying an appropriate architecture of an artificial neural network (ANN) for a given task is important because learning and generalisation of an ANN is affected by its structure. In this paper, an online pruning strategy is proposed to participate in the learning process of two constructive networks, i.e. fuzzy ARTMAP (FAM) and fuzzy ARTMAP with dynamic decay adjustment (FAMDDA), and the resulting hybrid networks are called FAM/FAMDDA with temporary nodes (i.e. FAM-T and FAMDDA-T, respectively). FAM-T and FAMDDA-T possess a capability of reducing the network complexity online by removing unrepresentative neurons. The performances of FAM-T and FAMDDA-T are evaluated and compared with those of FAM and FAMDDA using a total of 13 benchmark data sets. To demonstrate the applicability of FAM-T and FAMDDA-T, a real fault detection and diagnosis task in a power plant is tested. The results from both benchmark studies and real-world application show that FAMDDA-T and FAM-T are able to yield satisfactory classification performances, with the advantage of having parsimonious network structures.     

Learning algorithms of layered neural networks via extended Kalman filters

Learning algorithms are described for layered feedforward type neural networks, in which a unit generates a real-valued output through a logistic function. The problem of adjusting the weights of internal hidden units can be regarded as a problem of estimating (or identifying) constant parametes with a non-linear observation equation. The present algorithm based on (he extended Kalman filter has just the time-varying learning rate, while the well-known back-propagation (or generalized delta rule) algorithm based on gradient descent has a constant learning rate. From some simulation examples it is shown that when a sufficiently trained network is desired, the learning speed of the proposed algorithm is faster than that of the traditional back-propagation algorithm.     

A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation

This work presents a new sequential learning algorithm for radial basis function (RBF) networks referred to as generalized growing and pruning algorithm for RBF (GGAP-RBF). The paper first introduces the concept of significance for the hidden neurons and then uses it in the learning algorithm to realize parsimonious networks. The growing and pruning strategy of GGAP-RBF is based on linking the required learning accuracy with the significance of the nearest or intentionally added new neuron. Significance of a neuron is a measure of the average information content of that neuron. The GGAP-RBF algorithm can be used for any arbitrary sampling density for training samples and is derived from a rigorous statistical point of view. Simulation results for bench mark problems in the function approximation area show that the GGAP-RBF outperforms several other sequential learning algorithms in terms of learning speed, network size and generalization performance regardless of the sampling density function of the training data.     

一种改进的RBF神经网络参数优化方法

提出了一种改进的RBF神经网络参数优化算法。通过资源分配网络算法确定隐含层节点个数,引入剪枝策略删除对网络贡献不大的节点,用改进的粒子群算法对RBF网络的中心、宽度、权值进行优化,使RBF网络不仅可以得到合适的结构,同时也可以得到合适的控制参数。将此算法用于连续搅拌釜反应器模型的预测,结果表明,此算法优化后的RBF网络结构小,并且具有较高的泛化能力。     

DFNN及其在非线性动态系统中的应用

为了更好地辨识和控制非线性动态系统,在FNN基础上对其进行优化和改进,形成了动态模糊神经网络(DFNN)。给出了基于BP梯度算法的参数迭代学习算法,并应用于某非线性动态系统仿真试验中。仿真试验表明,该网络比单纯的FNN具有更强的辨识和控制能力,应用于非线性动态系统的控制中可以有效解决系统的非线性和不确定性,提高系统的跟踪性能,并且控制系统具有很强的鲁棒性。     

一种径向基函数神经网络预测在超临界温度控制系统中的应用研究

超临界温度控制系统具有较大的惯性、时滞和非线性,且动态特性随运行工况而改变,难以建立其精确的数学模型,本文采用GGAP算法的RBF神经网络构成神经网络预测控制器,将在线学习和预测控制相结合,以某超临界电厂主汽温度为研究对象,MATLAB仿真实验表明,该方法能对超临界温度控制系统实现有效的控制,动态性能较传统的PID控制有较大的提高。     

Indirect adaptive self-organizing RBF neural controller design with a dynamical training approach

This study proposes an indirect adaptive self-organizing RBF neural control (IASRNC) system which is composed of a feedback controller, a neural identifier and a smooth compensator. The neural identifier which contains a self-organizing RBF (SORBF) network with structure and parameter learning is designed to online estimate a system dynamics using the gradient descent method. The SORBF network can add new hidden neurons and prune insignificant hidden neurons online. The smooth compensator is designed to dispel the effect of minimum approximation error introduced by the neural identifier in the Lyapunov stability theorem. In general, how to determine the learning rate of parameter adaptation laws usually requires some trial-and-error tuning procedures. This paper proposes a dynamical learning rate approach based on a discrete-type Lyapunov function to speed up the convergence of tracking error. Finally, the proposed IASRNC system is applied to control two chaotic systems. Simulation results verify that the proposed IASRNC scheme can achieve a favorable tracking performance.     

A recurrent self-organizing neural fuzzy inference network

A recurrent self-organizing neural fuzzy inference network (RSONFIN) is proposed. The RSONFIN is inherently a recurrent multilayered connectionist network for realizing the basic elements and functions of dynamic fuzzy inference, and may be considered to be constructed from a series of dynamic fuzzy rules. The temporal relations embedded in the network are built by adding some feedback connections representing the memory elements to a feedforward neural fuzzy network. Each weight as well as node in the RSONFIN has its own meaning and represents a special element in a fuzzy rule. There are no hidden nodes initially in the RSONFIN. They are created online via concurrent structure identification and parameter identification. The structure learning together with the parameter learning forms a fast learning algorithm for building a small, yet powerful, dynamic neural fuzzy network. Two major characteristics of the RSONFIN can thus be seen: 1) the recurrent property of the RSONFIN makes it suitable for dealing with temporal problems and 2) no predetermination, like the number of hidden nodes, must be given, since the RSONFIN can find its optimal structure and parameters automatically and quickly. Moreover, to reduce the number of fuzzy rules generated, a flexible input partition method, the aligned clustering-based algorithm, is proposed. Various simulations on temporal problems are done and performance comparisons with some existing recurrent networks are also made. Efficiency of the RSONFIN is verified from these results.     

基于DQN的超密集网络能效资源管理

小基站的密集随机部署会产生严重干扰和较高能耗问题,为降低网络干扰、保证用户网络服务质量（QoS）并提高网络能效,构建一种基于深度强化学习（DRL）的资源分配和功率控制联合优化框架。综合考虑超密集异构网络中的同层干扰和跨层干扰,提出对频谱与功率资源联合控制能效以及用户QoS的联合优化问题。针对该联合优化问题的NP-Hard特性,提出基于DRL框架的资源分配和功率控制联合优化算法,并定义联合频谱和功率分配的状态、动作以及回报函数。利用强化学习、在线学习和深度神经网络线下训练对网络资源进行控制,从而找到最佳资源和功率控制策略。仿真结果表明,与枚举算法、Q-学习算法和两阶段算法相比,该算法可在保证用户QoS的同时有效提升网络能效。     

基于自结构动态递归模糊神经网络的无人机姿态控制*

针对无人机非线性、强耦合等特点,提出了基于该自结构动态递归模糊神经网络的姿态控制系统,给出了基于Lyapunov函数的系统稳定性证明。对四层模糊神经网络进行了优化和改进,设计了自结构动态递归模糊神经网络,该网络可以根据系统状态在线更新权值、创建/删除节点、优化网络结构。仿真表明：该控制方法的突出优点是,在兼顾考虑了系统中的不确定性因素、非线性因素及外部干扰并存的情况下,保证系统的稳定性和跟踪性能;同时此网络结构比固定结构的模糊神经网络响应速度快,因此更具优越性。     

Fuzzy generalization of the counter-propagation neural network: a family of soft competitive basis function neural networks

 Based on combining neural network (NN) with fuzzy logical system (FLS), a new family of three-layer feedforward networks, called soft-competition basis function neural networks (SCBFs), is proposed under the framework of the counter-propagation (CP) network. The hidden layer of SCBFs is designed as competitive layer with soft competitive strategy. The output function of their hidden neuron is defined as basis function taking the form of fuzzy membership function. SCBFs possess the ability of functional approximation. They are fuzzy generalization of the CP network and functionally equivalent to TS-model of fuzzy logical system. Therefore, they can be regard as either a NN or a FLS. Their learning algorithms are also discussed in this paper. Finally, some experiments are given to test the performance of SCBFs.