Combined projection and kernel basis functions for classification in evolutionary neural networks

This paper proposes a hybrid neural network model using a possible combination of different transfer projection functions (sigmoidal unit, SU, product unit, PU) and kernel functions (radial basis function, RBF) in the hidden layer of a feed-forward neural network. An evolutionary algorithm is adapted to this model and applied for learning the architecture, weights and node typology. Three different combined basis function models are proposed with all the different pairs that can be obtained with SU, PU and RBF nodes: product–sigmoidal unit (PSU) neural networks, product–radial basis function (PRBF) neural networks, and sigmoidal–radial basis function (SRBF) neural networks; and these are compared to the corresponding pure models: product unit neural network (PUNN), multilayer perceptron (MLP) and the RBF neural network. The proposals are tested using ten benchmark classification problems from well known machine learning problems. Combined functions using projection and kernel functions are found to be better than pure basis functions for the task of classification in several datasets.     

Steel columns under fire—a neural network based strength model

This paper presents a strength model of steel columns under elevated temperatures using the artificial neural network. The many influencing parameters make it difficult to build an analytical steel strength model. Being a flexible model building method, the artificial neural network is an ideal tool to construct the complex relationship between the input and the output parameters accurately. A hybrid neural network, which combines the sigmoid neurons and the radial basis function neurons at the hidden layer, is proposed to better map the input–output relationship both locally and globally. The use of the genetic algorithm approach in searching the best-hidden neurons makes the hybrid neural network less likely to be trapped in local minima than the traditional gradient-based search algorithms. The genetic algorithm based hybrid neural network is applied to model the strength of steel columns under fire. The neural network results are compared with the modified Rankine formula.     

前向神经网络合理隐含层结点个数估计

合理选择隐含层神经元个数是前向神经网络构造中的一个关键问题,对网络的泛化能力、训练速度等都具有重要的影响。该文提出了基于隐含层神经元输出之间的相关分析而进行隐含层神经元合理个数的估计方法,首先建立了基于网络输出和基于网络输出对网络各输入一阶偏导数的隐含层各神经元输出之间的相关程度度量,进而给出了基于模糊等价关系分析的神经元合理个数估计方法。具体应用结果证明了所提出方法的有效性。     

ARRBFNs with SVR for prediction of chaotic time series with outliers

In this article, annealing robust radial basis function networks (ARRBFNs), which consist of a radial basis function network and a support vector regression (SVR), and an annealing robust learning algorithm (ARLA) are proposed for the prediction of chaotic time series with outliers. In order to overcome the initial structural problems of the proposed neural networks, the SVR is utilized to determine the number of hidden nodes, the initial parameters of the kernel, and the initial weights for the proposed ARRBFNs. Then the ARLA that can conquer the outliers is applied to tune the parameters of the kernel and the weights in the proposed ARRBFNs under the initial structure with SVR. The simulation results of Mackey-Glass time series show that the proposed approach with different SVRs can cope with outliers and give a fast learning speed. The results of the simulation are also given to demonstrate the validity of proposed method for chaotic time series with outliers.     

Performance evaluation of a sequential minimal radial basisfunction (RBF) neural network learning algorithm

Presents a detailed performance analysis of the minimal resource allocation network (M-RAN) learning algorithm, M-RAN is a sequential learning radial basis function neural network which combines the growth criterion of the resource allocating network (RAN) of Platt (1991) with a pruning strategy based on the relative contribution of each hidden unit to the overall network output. The resulting network leads toward a minimal topology for the RAN. The performance of this algorithm is compared with the multilayer feedforward networks (MFNs) trained with 1) a variant of the standard backpropagation algorithm, known as RPROP and 2) the dependence identification (DI) algorithm of Moody and Antsaklis (1996) on several benchmark problems in the function approximation and pattern classification areas. For all these problems, the M-RAN algorithm is shown to realize networks with far fewer hidden neurons with better or same approximation/classification accuracy. Further, the time taken for learning (training) is also considerably shorter as M-RAN does not require repeated presentation of the training data.     

基于RBF神经网络的抗噪语音识别

针对目前在噪音环境下语音识别系统性能较差的问题,利用RBF神经网络具有最佳逼近性能、训练速度快等特性,分别采用聚类和全监督训练算法,实现了基于RBF神经网络的抗噪语音识别系统。聚类算法的隐含层训练采用K－均值聚类算法,输出层的学习采用线性最小二乘法;全监督算法中所有参数的调整基于梯度下降法,它是一种有监督学习算法,能够选出性能优良的参数。实验表明,在不同的信噪比下,全监督算法较之聚类算法有更高的识别率。     

采样算子调整的径向基网络增量映射学习算法

为了提高增量映射学习(IPL)算法的效率,调整了径向基神经网络基函数的中心及方差,以达到调整采样算子的目的,同时,通过神经元函数相关性的计算,确定添加新神经元时,相关函数的阈值,为系统结构调整提供相应依据.新方法步骤相对简单,所以算法速度较快;仿真结果表明,由于系统参数得到调整,对于同一问题,改进IPL算法得到的径向基神经网络结构较一般算法得到的网络结构简单,输出结果也较为精确.     

改进递归最小二乘RBF神经网络溶解氧预测

为提高溶解氧预测的准确性,将基于改进型递归最小二乘算法优化的径向基函数（ RBF）神经网络方法应用于溶解氧预测。利用K均值聚类算法进行隐层单元中心选择;利用改进型递归最小二乘算法优化RBF神经网络隐含层到输出层的权值。仿真结果表明：该方法对溶解氧的预测具有较好的非线性拟合能力,预测精度优于RBF神经网络和递归最小二乘算法优化的RBF神经网络。     

一类反馈过程神经元网络模型及学习算法研究

提出了一种带有反馈输入的过程式神经元网络模型,模型为三层结构,其隐层和输出层均为过程神经元。输入层完成连续信号的输入,隐层完成输入信号的空间聚合和向输出层逐点映射,并将输出信号逐点反馈到输入层;输出层完成隐层输出信号的时、空聚合运算和系统输出。在对权函数实施正交基展开的基础上给出了该模型的学习算法。仿真实验证明了该模型的有效性和可行性。     

一种基于正交神经网络的曲线重建方法

提出了一种基于正交神经网络的曲线重建方法。该正交神经网络结构与三层交向网络相同,不同的是正交网的隐单元处理函数采用Ｔｃｈｅｂｙｃｈｅｆｆ正交函数,而不是ｓｉｇｍｏｉｄｉａｌ函数,新的曲线重建方法具有利用较少的数据点列将光滑的曲线以较高的精度重建的特点,网络训练采用Ｇｉｖｅｎｓ正交学习算法,由于它不是一种迭代算法,故学习速度快,而且没有网络初始参数的选取问题,网络训练又能避免陷入局部极小解等问题。实     

一种混合优化的结构自适应极限过程神经网络及应用

为解决过程神经网络的隐层结构和训练速度问题,在极限学习机的基础上,提出一种混合优化的结构自适应极限过程神经网络.首先,采用在隐层中逐次增加过程神经元节点直至满足输出误差的方式完成模型结构自适应;然后,为消除冗余节点,提出对新增临时节点输出实施Gram-Schmidt正交化完成相关性判别;最后,构建一种量子衍生布谷鸟算法,对新增节点输入权函数正交基展开系数实施寻优.仿真实验以Mackey-Glass和页岩油TOC预测为例,通过对比分析验证所提出方法的有效性,仿真结果表明所得模型的逼近效率和训练速度有明显提高.     

Uncertain data stream algorithm based on clustering RBF neural network

In this paper, the existing algorithms for modeling uncertain data streams based on radial basis function neural networks have problems of low accuracy, weak stability and slow speed. A new clustering method for uncertain data streams is proposed. Radial basis function neural network of the algorithm. The algorithm firstly models the uncertain data stream, then combines the fuzzy theory and the neural network principle to obtain the radial basis function neural network, and then obtains the radial basis function neural network through the clustering algorithm of the regular tetrahedral uncertain vector. The central weight and width weights ultimately result in hidden layer output and output layer output results. The experimental results show that the proposed algorithm is an effective algorithm for modeling uncertain data streams using clustering radial basis function neural networks. It has higher precision, stability and speed than similar algorithms.     

一种结构自适应径向基函数神经网络

提出了一种新的结构自适应的径向基函数(RBF)神经网络模型。在该网络中,自组织映射(SOM)神经网络作为聚类网络,采用无监督学习算法对输入样本进行自组织分类,并将分类中心及其对应的权值向量传递给RBF神经网络,作为径向基函数的中心和相应的权值向量;RBF神经网络作为基础网络,采用高斯函数实现输入层到隐层的非线性映射,输出层则采用有监督学习算法训练网络的权值,从而实现输入层到输出层的非线性映射。通过对字母数据集进行仿真,表明该网络具有较好的性能。     

基于在线减法聚类的RBF神经网络结构设计

以设计最小径向基函数（RBF）神经网络结构为着眼点,提出一种在线RBF网络结构设计算法.该算法将在线减法聚类能实时跟踪工况的特性与RBF网络参数学习过程相结合,使得网络既能在线适应实时对象的变化又能维持紧凑的结构,有效地解决了RBF神经网络结构自组织问题.该算法只调整欧氏距离距实时工况最近的核函数,大大提高了网络的学习速度.通过对典型非线性函数逼近和混沌时间序列预测的仿真,表明所提出的算法具有良好的动态特性响应能力和逼近能力.     

一种结构自适应的径向基函数神经网络

提出了一种新的结构自适应的径向基函数(RBF)神经网络模型。在该模型中,自组织映射(SOM)神经网络作为聚类网络,采用无监督学习算法对输入样本进行自组织分类,并将分类中心及其对应的权值向量传递给RBF神经网络,分别作为径向基函数的中心和相应的权值向量;RBF神经网络作为基础网络,采用高斯函数实现输入层到隐层的非线性映射,输出层则采用有监督学习算法训练网络的权值,从而实现输入层到输出层的非线性映射。通过对字母数据集进行仿真,表明该网络具有较好的性能。     

BP神经网络的联合优化算法

针对BP神经网络存在收敛速度慢、易陷入局部极小等缺陷,提出了一种自适应调节学习率和动态调整S型激励函数相结合的改进BP算法。该算法将学习率与误差函数相关联,再对每个隐单元和输出单元的激励函数的斜率进行自动调整。通过实例仿真,将改进算法与标准BP算法、加动量项法和自适应学习率法进行比较,来验证所提出方法的有效性。实验结果表明,联合优化的BP算法能有效加快网络的收敛过程,并具有较强的泛化能力。     

基于可拓聚类的极限学习机神经网络

针对极限学习机(ELM)算法随机选择输入层权值的问题,借鉴第2类型可拓神经网络(ENN-2)聚类的思想,提出了一种基于可拓聚类的ELM(EC-ELM)神经网络。该神经网络是以隐含层神经元的径向基中心向量作为输入层权值,采用可拓聚类算法动态调整隐含层节点数目和径向基中心,并根据所确定的输入层权值,利用Moore-Penrose广义逆快速完成输出层权值的求解。同时,对标准的Friedman#1回归数据集和Wine分类数据集进行测试,结果表明,EC-ELM提供了一种简便的神经网络结构和参数学习方法,并且比基于可拓理论的径向基函数(ERBF)、ELM神经网络具有更高的建模精度和更快的学习速度,为复杂过程的建模提供了新思路。     

一种基于Walsh变换的反馈过程神经网络模型及学习算法

提出了一种带有反馈输入的过程式神经元网络模型，模型为三层结构，其隐层和输出层均为过程神经元．输入层完成连续信号的输入，隐层完成输入信号的空间聚合和向输出层逐点映射，并将输出信号逐点反馈到输入层；输出层完成隐层输出信号的时、空聚合运算和系统输出．在对权函数实施Walsh变换的基础上给出了该模型的学习算法．仿真实验证明了模型和算法的有效性．     

基于ART 的RBF 网络结构设计

针对径向基函数(RBF)网络隐层结构难以确定的问题,基于自适应共振理论(ART)网络良好的在线分类特性,提出一种RBF网络结构设计算法。该算法将ART网络的聚类特性用于RBF网络结构设计中,通过对输入向量与已存模式的相似度比较将输入向量进行分类,确定隐含层节点个数和初始参数,使网络具有精简的结构。对典型非线性函数逼近的仿真结果表明,所提出的结构具有快速的学习能力和良好的逼近能力。     

Application of error minimized extreme learning machine for simultaneous learning of a function and its derivatives

In this paper a new learning algorithm is proposed for the problem of simultaneous learning of a function and its derivatives as an extension of the study of error minimized extreme learning machine for single hidden layer feedforward neural networks. Our formulation leads to solving a system of linear equations and its solution is obtained by Moore-Penrose generalized pseudo-inverse. In this approach the number of hidden nodes is automatically determined by repeatedly adding new hidden nodes to the network either one by one or group by group and updating the output weights incrementally in an efficient manner until the network output error is less than the given expected learning accuracy. For the verification of the efficiency of the proposed method a number of interesting examples are considered and the results obtained with the proposed method are compared with that of other two popular methods. It is observed that the proposed method is fast and produces similar or better generalization performance on the test data.