前向神经网络合理隐含层结点个数估计

合理选择隐含层神经元个数是前向神经网络构造中的一个关键问题,对网络的泛化能力、训练速度等都具有重要的影响。该文提出了基于隐含层神经元输出之间的相关分析而进行隐含层神经元合理个数的估计方法,首先建立了基于网络输出和基于网络输出对网络各输入一阶偏导数的隐含层各神经元输出之间的相关程度度量,进而给出了基于模糊等价关系分析的神经元合理个数估计方法。具体应用结果证明了所提出方法的有效性。     

联合多层注意力网络矩阵分解的推荐算法

许多推荐算法如基于矩阵分解因无法充分挖掘用户对项目的偏好信息而无法取得令人满意的推荐效果.为了解决上述问题,该文设计了两个模块,首先,利用多层感知机技术学习输入的信息以获得较好的特征表示,在原始输入时通过点积操作得到关系信息,并将其命名为深度矩阵分解(DeepMF);其次,在多层感知机中加入多层注意力网络,这样能够得到...     

基于BP网络曲线拟合方法的研究

在利用BP神经网络进行曲线拟合时,为了解决如何确定BP神经网络隐含层神经元数问题,提出了一种新的快速构建BP神经网络结构的方法,即如何由输入层神经元数、输出层神经元数及样本点数来确定隐含层神经元数,同时针对在曲线拟合过程中经常出现的一些问题提出了解决方案。实验结果表明,该构建方法和改进方案在提高曲线的拟合精度、加快收敛速度方面收到了较好的效果．     

输入层自构造神经网络及其在光谱分析中的应用

为了解决红外光谱定量分析中的特征提取和校正规模问题，提出了一种输入层自构造神经网络。这种网络能够利用训练数据的某些先验知识，自然选择输入层神经元的个数。在学习过程中，输入神经元个数从最小值1开始，根据网络误差的变化逐步增加，最终确定最佳神经元数量。这种网络模型将特征提取和参数学习过程融一体，有利于提高建模效率。利用仿真红外光谱的定量分析实验表明，这种网络模型不仅能够对光谱数据实现高效率的波长选择，并具有抑制随机噪声和非线性干扰的能力。     

能实现精确映射的前馈神经网络快速算法与结构设计

一、引言人们常用BP算法训练多层感知机,但BP算法的缺点使它在工程应用上受到了限制。尽管从理论上人们证明了对BP算法而言,如果不限制神经元的个数,则三层前馈神经网络可以以任意精度实现给定样本的映射,但对于给定的实际问题,BP算法并没有具体给出确定神经元个数的计算方法,使得应用时须凭经验选择。本文正是针对BP算法的这一缺点,提出了一种基于Moore-Penrose广义逆的代数方法。该方法给出了在实现精确映射要求下,确定神经元个数的两种充分条件,并给出了具体的计算公式。这对于那些要求高精度逼近的场合无疑具有指导意义。本文     

用于大数据分类的快速隐层优化分布式极限学习机

针对大数据分类问题应用设计了一种快速隐层优化方法来解决分布式超限学习机（Extreme Learning Machine,ELM）在训练过程中存在的突出问题--需要独立重复运行多次才能优化隐层结点个数或模型泛化性能。在不增加算法时间复杂度的前提下,新算法能同时训练多个ELM隐层网络,全面兼顾模型泛化能力和隐层结点个数的优化,并通过分布式计算避免大量重复计算。同时,在算法求解过程中通过这种方式能更精确、更直观地学习隐含层结点个数变化带来的影响。比较多种类型标准测试函数的实验结果,相对于分布式ELM,新算法在求解精度、泛化能力、稳定性上大大提高。     

基于时间序列神经网络的鲜切花价格指数短期预测

鲜切花价格指数是反映鲜切花市场现状的风向标，研究鲜切花价格指数变化，掌握鲜花市场的动态和规律性具有重要意义。本文针对具有时序特点的鲜切花价格指数，基于BP模型中的L-M优化算法构建鲜切花价格指数短期预测模型，采用tansig和purelin作为各层之间的传递函数，利用时间序列分析方法确定输入层的神经元个数，通过实验数据对比来确定隐含层的神经元个数。采用平均绝对误差、平均相对误差和均方根误差这3个评价指标对模型的预测精度进行检验，实验结果表明所构建模型是有效的和具有实际应用价值的。     

基于决策树算法确定一个最小神经网络

神经网络隐层神经元的个数对于网络的性能有着重要的影响,通常情况下,对于一个特定问题来说,没有一个确定的方法来决定隐含层到底应该有多少个神经元,一般采用试探的方法通过多次实验来达到理想效果.在分类问题中,决策树和神经网络的结构有着一定的关联性,通过把决策树映射到神经网络结构中来确定隐层神经元的个数的方法能够有效地设计神经网络的结构,从而提高训练的效率并达到良好的分类效果.实验结果表明,该方法能够得到一个有着良好识别率的最小神经网络.方法简单有效,直观且易于操作.     

输入层自构造神经网络及其在光谱分析中的应用

为了解决红外光谱定量分析中的特征提取和校正规模问题,提出了一种输入层自构造神经网络。这种网络能够利用训练数据的某些先验知识,自动选择输入层神经元的个数。在学习过程中,输入神经元个数从最小值1开始,根据网络误差的变化逐步增加,最终确定最佳神经元数量。这种网络模型将特征提取和参数学习过程融为一体,有利于提高建模效率。利用仿真红外光谱的定量分析实验表明,这种网络模型不仅能够对光谱数据实现高效率的波长选择,并具有抑制随机噪声和非线性干扰的能力。     

多层反馈神经网络的FP学习和综合算法

本文给出多层反馈神经网络的ＦＰ学习和综合算法，并讨论此类网络的性质，指出将它应用于聚类分析能给出不粒度的聚类，且具有收敛速度快（是样本个数的线性函数）、算法计算量少（是样本个数和输入、输出维数的双线性函数）、网络元件个数少、权系数简单（只取３个值）、网络容易硬件实现等优点．作为聚类器的神经网络的学习和综合问题已得到较圆满地解决.     

A general insight into the effect of neuron structure on classification

This paper gives a general insight into how the neuron structure in a multilayer perceptron (MLP) can affect the ability of neurons to deal with classification. Most of the common neuron structures are based on monotonic activation functions and linear input mappings. In comparison, the proposed neuron structure utilizes a nonmonotonic activation function and/or a nonlinear input mapping to increase the power of a neuron. An MLP of these high power neurons usually requires a less number of hidden nodes than conventional MLP for solving classification problems. The fewer number of neurons is equivalent to the smaller number of network weights that must be optimally determined by a learning algorithm. The performance of learning algorithm is usually improved by reducing the number of weights, i.e., the dimension of the search space. This usually helps the learning algorithm to escape local optimums, and also, the convergence speed of the algorithm is increased regardless of which algorithm is used for learning. Several 2-dimensional examples are provided manually to visualize how the number of neurons can be reduced by choosing an appropriate neuron structure. Moreover, to show the efficiency of the proposed scheme in solving real-world classification problems, the Iris data classification problem is solved using an MLP whose neurons are equipped by nonmonotonic activation functions, and the result is compared with two well-known monotonic activation functions.     

Application of Sequential Learning Neural Networks to Civil Engineering Modeling Problems

A sequential orthogonal approach to the building and training of a single hidden layer neural network is presented in this paper. The Sequential Learning Neural Network (SLNN) model proposed by Zhang and Morris [1]is used in this paper to tackle the common problem encountered by the conventional Feed Forward Neural Network (FFNN) in determining the network structure in the number of hidden layers and the number of hidden neurons in each layer. The procedure starts with a single hidden neuron and sequentially increases in the number of hidden neurons until the model error is sufficiently small. The classical Gram–Schmidt orthogonalization method is used at each step to form a set of orthogonal bases for the space spanned by output vectors of the hidden neurons. In this approach it is possible to determine the necessary number of hidden neurons required. However, for the problems investigated in this paper, one hidden neuron itself is sufficient to achieve the desired accuracy. The neural network architecture has been trained and tested on two practical civil engineering problems – soil classification, and the prediction o strength and workability of high performance concrete.     

The upper bound of the minimal number of hidden neurons for the parity problem in binary neural networks

Binary neural networks (BNNs) have important value in many application areas.They adopt linearly separable structures,which are simple and easy to implement by hardware.For a BNN with single hidden layer,the problem of how to determine the upper bound of the number of hidden neurons has not been solved well and truly.This paper defines a special structure called most isolated samples (MIS) in the Boolean space.We prove that at least 2 n 1 hidden neurons are needed to express the MIS logical relationship in the Boolean space if the hidden neurons of a BNN and its output neuron form a structure of AND/OR logic.Then the paper points out that the n -bit parity problem is just equivalent to the MIS structure.Furthermore,by proposing a new concept of restraining neuron and using it in the hidden layer,we can reduce the number of hidden neurons to n .This result explains the important role of restraining neurons in some cases.Finally,on the basis of Hamming sphere and SP function,both the restraining neuron and the n -bit parity problem are given a clear logical meaning,and can be described by a series of logical expressions.     

Classification methodologies of multilayer perceptrons with sigmoid activation functions

This paper studies the classification mechanisms of multilayer perceptrons (MLPs) with sigmoid activation functions (SAFs). The viewpoint is presented that in the input space the hyperplanes determined by the hidden basis functions with values 0's do not play the role of decision boundaries, and such hyperplanes do not certainly go through the marginal regions between different classes. For solving an n-class problem, a single-hidden-layer perceptron with at least log2(n-1)?2 hidden nodes is needed. The final number of hidden neurons is still related to the sample distribution shapes and regions, but not to the number of samples and input dimensions. As a result, an empirical formula for optimally selecting the initial number of hidden nodes is proposed. The ranks of response matrixes of hidden layers should be taken as a main basis for pruning or growing the existing hidden neurons. A structure-fixed perceptron ought to learn more than one round from different starting weight points for one classification task, and only the group of weights and biases that has the best generalization performance should be reserved. Finally, three examples are given to verify the above viewpoints.     

A multilayer incremental neural network architecture for classification

A new multilayer incremental neural network (MINN) architecture and its performance in classification of biomedical images is discussed. The MINN consists of an input layer, two hidden layers and an output layer. The first stage between the input and first hidden layer consists of perceptrons. The number of perceptrons and their weights are determined by defining a fitness function which is maximized by the genetic algorithm (GA). The second stage involves feature vectors which are the codewords obtained automaticaly after learning the first stage. The last stage consists of OR gates which combine the nodes of the second hidden layer representing the same class. The comparative performance results of the MINN and the backpropagation (BP) network indicates that the MINN results in faster learning, much simpler network and equal or better classification performance.     

Fast learning Circular Complex-valued Extreme Learning Machine (CC-ELM) for real-valued classification problems

In this paper, we present a fast learning fully complex-valued extreme learning machine classifier, referred to as ‘Circular Complex-valued Extreme Learning Machine (CC-ELM)’ for handling real-valued classification problems. CC-ELM is a single hidden layer network with non-linear input and hidden layers and a linear output layer. A circular transformation with a translational/rotational bias term that performs a one-to-one transformation of real-valued features to the complex plane is used as an activation function for the input neurons. The neurons in the hidden layer employ a fully complex-valued Gaussian-like (‘sech’) activation function. The input parameters of CC-ELM are chosen randomly and the output weights are computed analytically. This paper also presents an analytical proof to show that the decision boundaries of a single complex-valued neuron at the hidden and output layers of CC-ELM consist of two hyper-surfaces that intersect orthogonally. These orthogonal boundaries and the input circular transformation help CC-ELM to perform real-valued classification tasks efficiently.Performance of CC-ELM is evaluated using a set of benchmark real-valued classification problems from the University of California, Irvine machine learning repository. Finally, the performance of CC-ELM is compared with existing methods on two practical problems, viz., the acoustic emission signal classification problem and a mammogram classification problem. These study results show that CC-ELM performs better than other existing (both) real-valued and complex-valued classifiers, especially when the data sets are highly unbalanced.     

稳健二进神经网络及其基本性质研究

从二进前向网络的稳健要求出发,提出了稳健分类的概念,在此基础上给出了稳健分 类超平面的一般形式,从而如果二进前向网络的每一神经元都是稳健神经元,则网络的连接权 仅为-1,0或+1,每一神经元的阈值也只为二分之一的基阈值加上一处于有限区域上整数的 辅阈值,并且辅阈值为神经元各个输入对其的贡献之和.稳健二进前向网络的这些性质使得网 络不仅稳健能力强,而且易于做到隐节点数少、连接少、易于实现.     

一类反馈过程神经元网络模型及其学校算法

提出了一种基于权函数基展开的反馈过程神经元网络模型．该模型为三层结构，由输入层、过程神经元隐层和过程神经元输出层组成．输入层完成系统时变过程信号的输入及隐层过程神经元输出信号向系统的反馈；过程神经元隐层用于完成输入信号的空间加权聚合和激励运算，同时将输出信号传输到输出层并加权反馈到输入层；输出层完成隐层输出信号的空间加权聚集和对时间的聚合运算以及系统输出．文中给出了学习算法，并以旋转机械故障自动诊断问题为例验证了模型和算法的有效性．     

A new implementation of the algorithm of adaptive construction of hierarchical neural network classifiers

The article presents development of the algorithm of adaptive construction of hierarchical neural network classifiers based on automatic modification of the desired output of perceptrons with a small number of neurons in the single hidden layer. The conducted testing of the new program implementation of this approach demonstrated that the considered algorithm was more computationally efficient and provided higher quality of solution of multiple classification problems in comparison with standard multi-layer perceptron.     

Thalassaemia classification by neural networks and genetic programming

This paper presents the use of a neural network and a decision tree, which is evolved by genetic programming (GP), in thalassaemia classification. The aim is to differentiate between thalassaemic patients, persons with thalassaemia trait and normal subjects by inspecting characteristics of red blood cells, reticulocytes and platelets. A structured representation on genetic algorithms for non-linear function fitting or STROGANOFF is the chosen architecture for genetic programming implementation. For comparison, multilayer perceptrons are explored in classification via a neural network. The classification results indicate that the performance of the GP-based decision tree is approximately equal to that of the multilayer perceptron with one hidden layer. But the multilayer perceptron with two hidden layers, which is proven to have the most suitable architecture among networks with different number of hidden layers, outperforms the GP-based decision tree. Nonetheless, the structure of the decision tree reveals that some input features have no effects on the classification performance. The results confirm that the classification accuracy of the multilayer perceptron with two hidden layers can still be maintained after the removal of the redundant input features. Detailed analysis of the classification errors of the multilayer perceptron with two hidden layers, in which a reduced feature set is used as the network input, is also included. The analysis reveals that the classification ambiguity and misclassification among persons with minor thalassaemia trait and normal subjects is the main cause of classification errors. These results suggest that a combination of a multilayer perceptron with a blood cell analysis may give rise to a guideline/hint for further investigation of thalassaemia classification.