BP神经网络合理隐结点数确定的改进方法

合理选择隐含层结点个数是BP神经网络构造中的关键问题,对网络的适应能力、学习速率都有重要的影响.在此提出一种确定隐结点个数的改进方法.该方法基于隐含层神经元输出之间的线性相关关系与线性无关关系,对神经网络隐结点个数进行削减,缩减网络规模.以零件工艺过程中的加工参数作为BP神经网络的输入,加工完成的零件尺寸作为BP神经网络的输出建立模型,把该方法应用于此神经网络模型中,其训练结果证明了该方法的有效性.     

基于输出层权值解析修正的神经网络有效训练

根据神经网络训练误差对权值的梯度特征分析,提出了网络输出层权值与网络隐含层权值轮换修正的思想,并基于网络输出层权值与网络隐含层权值之间的依赖关系,建立了网络输出层权值解析修正和隐含层权值修正的具体方法,所提出的方法通过提高网络权值修正的准确性而提高网络训练的有效性。根据网络输出节点的输出误差与其总输入误差的关系,提出了进一步提高所获得网络推广性的具体方法。实例计算结果表明,所提出的方法可以显著地提高网络的训练效率,并有效地增强网络推广性。     

基于互信息的分步式输入变量选择多元序列预测研究

针对多元序列分析中存在的输入变量选择问题,提出一种基于k-!近邻互信息估计的分步式变量选择算法. 该算法通过两步过程分别实现相关变量的选择与弱相关变量的剔除. 同时将分步变量选择算法应用于径向基函数(Radial basis function, RBF) 神经网络结构的优化中.在K均值聚类的基础上,通过分析隐含层神经元的输出权值与神经网络输出的相关性, 对隐含层节点进行选择,改进网络的结构与性能. Friedman数据的仿真实验验证了分步变量选择算法的有效性; Gas furnace多元时间序列以及Boston housing数据的仿真结果表明, 优化后的RBF网络能够在保证模型精度的基础上有效控制网络规模.     

一种新的多层感知机隐含层神经元个数上限计算方法

多层感知机在分类问题中具有广泛的应用。本文针对超平面阈值神经元构成的多层感知机用于分类的情况，求出了输入层神经元最多能把输入空间划分的区域数的解析表达式。该指标在很大程度上说明了感知机输入层的分类能力。本文还对隐含层神经元个数和输入层神经元个数之间的约束关系进行了讨论，得到了更准确的隐含层神经元个数上
上限。当分类空间的雏数远小于输入层神经元个数时，本文得到的隐含层神经元个数上限比现有的结果更小。     

基于贝叶斯正则化BP人工神经网络的煤与瓦斯突出预测的研究

文章介绍了BP人工神经网络和贝叶斯正则化算法的原理,探讨了贝叶斯正则化BP人工神经网络模型的建立,通过改变隐含层神经元个数的实验建立了只含1个隐含层且隐含层仅需1个神经元的煤与瓦斯突出预测模型的最佳网络结构。对该网络采用煤与瓦斯突出的预测指标进行训练、检测的结果表明,该网络预测的煤与瓦斯突出的危险程度与实际情况完全吻合;对该网络输入层输入的煤与瓦斯突出的预测指标、对输出层输出的预测结果的权值进行分析的结果表明,煤层地质构造类型对煤与瓦斯突出的影响为最大。上述研究结果对煤与瓦斯突出的预测预防研究、提高煤与瓦斯突出预测的准确性具有一定的参考价值。     

基于灵敏度分析法的ELM剪枝算法

针对极端学习机(ELM)网络结构设计问题,提出基于灵敏度分析法的ELM剪枝算法.利用隐含层节点输出和相对应的输出层权值向量,定义学习残差对于隐含层节点的灵敏度和网络规模适应度,根据灵敏度大小判断隐含层节点的重要性,利用网络规模适应度确定隐含层节点个数,删除重要性较低的节点.仿真结果表明,所提出的算法能够较为准确地确定与学习样本相匹配的网络规模,解决了ELM网络结构设计问题.     

确定RBF神经网络参数的新方法

提出一种确定RBF网络隐含层神经元和权值的有效方法。该方法将自动聚类算法与对称距离相结合优化每个隐含层神经元的中心向量；利用伪逆方法确定隐层神经元到输出神经元的权值。实验结果表明：该方法比自动聚类算法有更好的分类能力。     

超声传感器补偿算法在机器人避障中的应用

分析了井下钻孔机器人避障中超声波传感器的局限性,并提出解决的方案。着重指出对超声波进行温度、湿度补偿,尝试用Elman反馈神经网络逼近函数。Elman网络隐含层采用"Tansig"激活函数,输出层用"Pureline"激活函数,保证了只要有足够多的隐含层神经元个数,网络就可以任意精度逼近任意函数。经实验验证:对超声测距进行温度、湿度补偿后,其测量精度提高了2个数量级。大大改善系统中避障模块的工作效率,提高了钻孔机器人躲避障碍物的能力。     

用一组单输出的子网络代替多输出的BP网络

BP网络是使用最广泛的神经网络模型。在BP网络模型中,输入信息(X_1,X_2,…,X_m)首先通过各输入结点前向传播到下一层的各结点,这下一层就是所谓的隐含层,隐含层可以是一层也可以是多层,最后通过输出层得到输出信息(y_1,y_2,…,y_n)。在BP网络中,输入结点和输出结点的个数由具体问题决定,隐层单元的个数是可变的。如果具体问题的目标输出是多维的,BP网络就具有多个输出结点。本文提出了一种子网组的结构,用来代替多输出的单个BP网络。子网组即是对问题目标输出的每一维建立一个单输出的BP网络,整个子网组结构的输出由这n个子网络组合而成。本文还给出了一种子网组结构的学习算法,并证明了该学习算法能够达到更好的学习效果。     

基于动态RBF神经网络的板形板厚综合控制仿真研究

基于RBF神经网络的特点提出了一种动态调节隐含层隐节点个数的方法，由2部分组成：首先以网络输出数据的均方误差及其变化率为标准来调节隐含层节点的数目，然后调节优化隐含层节点的中心值，根据广义逆矩阵的方法求出输出层权值．所设计的神经网络具有最少的隐含层节点数，提高了学习训练速度，构造了板形板厚综合控制的数学模型，采用新的模型处理方法，用动态RBF神经网络进行控制仿真，取得了理想的结果．     

Predicting sun spots using a layered perceptron neural network

Interest in neural networks has expanded rapidly in recent years. Selecting the best structure for a given task, however, remains a critical issue in neural-network design. Although the performance of a network clearly depends on its structure, the procedure for selecting the optimal structure has not been thoroughly investigated, it is well known that the number of hidden units must be sufficient to discriminate each observation correctly. A large number of hidden units requires extensive computational time for training and often times prediction results may not be as accurate as expected. This study attempts to apply the principal component analysis (PCA) to determine the structure of a multilayered neural network for time series forecasting problems. The main focus is to determine the number of hidden units for a multilayered feedforward network. One empirical experiment with sunspot data is used to demonstrate the usefulness of the proposed approach.     

Automated learning for reducing the configuration of a feedforwardneural network

In this paper, we present two learning mechanisms for artificial neural networks (ANN's) that can be applied to solve classification problems with binary outputs. These mechanisms are used to reduce the number of hidden units of an ANN when trained by the cascade-correlation learning algorithm (CAS). Since CAS adds hidden units incrementally as learning proceeds, it is difficult to predict the number of hidden units required when convergence is reached. Further, learning must be restarted when the number of hidden units is larger than expected. Our key idea in this paper is to provide alternatives in the learning process and to select the best alternative dynamically based on run-time information obtained. Mixed-mode learning (MM), our first algorithm, provides alternative output matrices so that learning is extended to find one of the many one-to-many mappings instead of finding a unique one-to-one mapping. Since the objective of learning is relaxed by this transformation, the number of learning epochs can be reduced. This in turn leads to a smaller number of hidden units required for convergence. Population-based learning for ANN's (PLAN), our second algorithm, maintains alternative network configurations to select at run time promising networks to train based on error information obtained and time remaining. This dynamic scheduling avoids training possibly unpromising ANNs to completion before exploring new ones. We show the performance of these two mechanisms by applying them to solve the two-spiral problem, a two-region classification problem, and the Pima Indian diabetes diagnosis problem.     

一种估计前馈神经网络中隐层神经元数目的新方法

前馈神经网络中隐层神经元的数目一般凭经验的给出，这种方法往往造成隐单元数目的不足或过甚，从而导致网络存储容量不够或出现学习过拟现象，本研究提出了一种基于信息熵的估计三层前馈神经网络隐结点数目的方法，该方法首先利用训练集来训练具有足够隐单元数目的初始神经网络，然后计算训练集中能被训练过的神经网络正确识别的样本在隐层神经元的激活值，并对其进行排序，计算这些激活值的各种划分的信息增益，从而构造能将整个样本空间正确划分的决策树，最后遍历整棵树寻找重要的相关隐层神经元，并删除冗余无关的其它隐单元，从而估计神经网络中隐层神经元的较佳数目，文章最后以构造用于茶叶品质评定的具有较佳隐单元数目的神经网络为例，介绍本方法的使用，结果表明，本方法能有效估计前馈神经网络的隐单元数目。     

Output recurrent wavelet neural network-based adaptive backstepping controller for a class of MIMO nonlinear non-affine uncertain systems

In this paper, an adaptive backstepping control problem is proposed for a class of multiple-input-multiple-output nonlinear non-affine uncertain systems. An output recurrent wavelet neural network (ORWNN) is used to approximate the unknown nonlinear functions to develop the proposed adaptive backstepping controller. The proposed ORWNN combines the advantages of wavelet-based neural network, fuzzy neural network, and output feedback layer to achieve higher approximation accuracy and faster convergence. According to the estimation of ORWNN, the control scheme is designed by backstepping approach such that the system outputs follow the desired trajectories. Based on the Lyapunov approach, our approach guarantees that the system outputs converge to a small neighborhood of the references signals, that is, all signals of the closed-loop system are semi-globally uniformly ultimately bounded. Finally, simulation results including double pendulums system and two inverted pendulums on carts system are shown to demonstrate the performance and effectiveness of our approach.     

An Iterative Method for Deciding SVM and Single Layer Neural Network Structures

We present two new classifiers for two-class classification problems using a new Beta-SVM kernel transformation and an iterative algorithm to concurrently select the support vectors for a support vector machine (SVM) and the hidden units for a single hidden layer neural network to achieve a better generalization performance. To construct the classifiers, the contributing data points are chosen on the basis of a thresholding scheme of the outputs of a single perceptron trained using all training data samples. The chosen support vectors are used to construct a new SVM classifier that we call Beta-SVN. The number of chosen support vectors is used to determine the structure of the hidden layer in a single hidden layer neural network that we call Beta-NN. The Beta-SVN and Beta-NN structures produced by our method outperformed other commonly used classifiers when tested on a 2-dimensional non-linearly separable data set.     

Median radial basis function neural network

Radial basis functions (RBFs) consist of a two-layer neural network, where each hidden unit implements a kernel function. Each kernel is associated with an activation region from the input space and its output is fed to an output unit. In order to find the parameters of a neural network which embeds this structure we take into consideration two different statistical approaches. The first approach uses classical estimation in the learning stage and it is based on the learning vector quantization algorithm and its second-order statistics extension. After the presentation of this approach, we introduce the median radial basis function (MRBF) algorithm based on robust estimation of the hidden unit parameters. The proposed algorithm employs the marginal median for kernel location estimation and the median of the absolute deviations for the scale parameter estimation. A histogram-based fast implementation is provided for the MRBF algorithm. The theoretical performance of the two training algorithms is comparatively evaluated when estimating the network weights. The network is applied in pattern classification problems and in optical flow segmentation.     

Hierarchical genetic algorithm based RBF neural networks and application for modelling of the automatic depth control electrohydraulic system

The paper presents an approach to model nonlinear dynamic behaviors of the Automatic Depth Control Electrohydraulic System (ADCES) of a certain minesweeping weapon with Radial Basis Function (RBF) neural networks trained by hierarchical genetic algorithm. In the proposed hierarchical genetic algorithm, the control genes are used to determine the number of hidden units, and the parameter genes are used to identify center parameters of hidden units. In order to speed up convergence of the proposed algorithm, width and weight parameters of RBF neural network are calculated by linear algebra methods. The proposed approach is applied to the modelling of the ADCES, and experimental results clearly indicate that the obtained RBF neural network can emulate complex dynamic characteristics of the ADCES satisfactorily. The comparison results also show that the proposed approach performs better than the traditional clustering-based method.     

基于DNN的低资源语音识别特征提取技术

针对低资源训练数据条件下深层神经网络（Deep neural network,DNN）特征声学建模性能急剧下降的问题,提出两种适合于低资源语音识别的深层神经网络特征提取方法.首先基于隐含层共享训练的网络结构,借助资源较为丰富的语料实现对深层瓶颈神经网络的辅助训练,针对BN层位于共享层的特点,引入Dropout,Maxout,Rectified linear units等技术改善多流训练样本分布不规律导致的过拟合问题,同时缩小网络参数规模、降低训练耗时;其次为了改善深层神经网络特征提取方法,提出一种基于凸非负矩阵分解（Convex-non-negative matrix factorization,CNMF）算法的低维高层特征提取技术,通过对网络的权值矩阵分解得到基矩阵作为特征层的权值矩阵,然后从该层提取一种新的低维特征.基于Vystadial 2013的1小时低资源捷克语训练语料的实验表明,在26.7小时的英语语料辅助训练下,当使用Dropout和Rectified linear units时,识别率相对基线系统提升7.0%;当使用Dropout和Maxout时,识别率相对基线系统提升了12.6%,且网络参数数量相对其他系统降低了62.7%,训练时间降低了25%.而基于矩阵分解的低维特征在单语言训练和辅助训练的两种情况下都取得了优于瓶颈特征（Bottleneck features,BNF）的识别率,且在辅助训练的情况下优于深层神经网络隐马尔科夫识别系统,提升幅度从0.8%~3.4%不等.     

Blind equalization of a noisy channel by linear neural network

In this paper, a new neural approach is introduced for the problem of blind equalization in digital communications. Necessary and sufficient conditions for blind equalization are proposed, which can be implemented by a two-layer linear neural network, in the hidden layer, the received signals are whitened, while the network outputs provide directly an estimation of the source symbols. We consider a stochastic approximate learning algorithm for each layer according to the property of the correlation matrices of the transmitted symbols. The proposed class of networks yield good results in simulation examples for the blind equalization of a three-ray multipath channel.