模糊神经网络的结构自组织算法及应用

提出了一种新的模糊神经网络自组织算法,该算法能够基于输入输出数据自动进行结构辨识和参数辨识.首先采用一种自组织聚类方法建立起网络的结构和各参数的初值,然后采用监督学习来优化网络参数.通过对非线性函数逼近的分析,明了该自组织算法的有效性,并与其他算法作了比较.最后,以某污水处理厂的实际运行数据为对象,应用该模糊神经网络建立了活性污泥系统出水水质预测模型,仿真结果表明.该模型能够对污水处理系统出水水质进行较好的预测.     

一种多重支持向量机在线学习算法的研究

增量式支持向量机学习算法是一种重要的在线学习方法。传统的单增量支持向量机学习算法使用一个数据样本更新支持向量机模型。在增加或删除的数据样本点较多时,这种模型更新模式耗时巨大,具体原因是每个被插入或删除的样本都要进行一次模型参数更新的判断。该文提出一种基于参数规划的多重增量式的支持向量机优化训练算法,使用该训练算法,多重的支持向量机的训练时间大为减少。在合成数据集及真实测试数据集上的实验结果显示,该文提出的方法可以大大降低多重支持向量机训练算法的计算复杂度并提高分类器的精度。     

基于增量式GHSOM神经网络模型的入侵检测研究

传统的网络入侵检测方法利用已知类型的攻击样本以离线的方式训练入侵检测模型,虽然对已知攻击类型具有较高的检测率,但是不能识别网络上新出现的攻击类型.这样的入侵检测系统存在着建立系统的速度慢、模型更新代价高等不足,面对规模日益扩大的网络和层出不穷的攻击,缺乏自适应性和扩展性,难以检测出网络上新出现的攻击类型.文中对GHSOM(Growing Hierarchical Self-Organizing Maps)神经网络模型进行了扩展,提出了一种基于增量式GHSOM神经网络模型的网络入侵检测方法,在不破坏已学习过的知识的同时,对在线检测过程中新出现的攻击类型进行增量式学习,实现对入侵检测模型的动态扩展.作者开发了一个基于增量式GHSOM神经网络模型的在线网络入侵检测原型系统,在局域网环境下开展了在线入侵检测实验.实验结果表明增量式GHSOM入侵检测方法具有动态自适应性,能够实现在线检测过程中对GHSOM模型的动态更新,而且对于网络上新出现的攻击类型,增量式GHSOM算法与传统GHSOM算法的检测率相当.     

基于遗传优化神经网络的网络入侵特征检测

为了提高网络入侵检测正确率,提出一种遗传优化神经网络的网络入侵特征选择和检测算法。该方法先将网络状态特征和RBF神经网络参数作为遗传算法的个体,把检测正确率作为适应度函数;然后利用遗传算法的选择、交叉和变异等操作对网络状态特征和RBF神经网络参数进行优化,最后利用KDD 1999数据集对算法性能进行测试。测试结果表明：遗传优化神经网络能够快速获得最优网络状态特征和分类器参数,同时提高了网络入侵检测正确率。     

对象集变化时近似集动态维护的矩阵方法

目前粗糙集模型中概念的上、下近似集的计算方法大多是基于静态信息系统的.而实际的信息系统是随时间动态变化的,通常包括对象集、属性集和属性值3种类型的粒度变化,这些变化必然引起概念近似集的动态变化.如何快速、有效地更新概念的近似集是基于粗糙集的动态知识更新中的热点研究问题之一.而利用既有知识的增量式更新方法是一种有效的近似集动态更新方法.在信息系统动态变化的客观环境下,以矩阵作为表达和运算工具从一个全新的视角研究信息系统的论域随时间变化时,变精度粗糙集模型中概念的上、下近似集的增量式更新方法,并构造出近似集增量式更新的矩阵算法,随后分析了算法的时间复杂度.进一步,在MATLAB平台上开发出增量式更新和非增量式更新近似集的两种矩阵算法的程序,最后在UCI的6个数据集上测试了两种矩阵算法的性能并将实验结果进行比较,结果表明增量式更新的矩阵算法可行、简洁和高效.     

学习者知识模型的在线学习算法

学习者知识模型是智能授导系统(ITS)中教学过程实现和策略实施的基础,然而由于判别学习者知识掌握程度的不确定性和学习者知识掌握水平的实时变化,构建能正确反映学习者知识掌握程度及其变化的知识模型十分困难。基于贝叶斯网络,以知识项为基本节点构建学习者知识模型的结构;引入问题节点,根据学习者的学习测试结果,采用Voting EM算法来对知识模型的参数进行在线学习和更新;同时,通过设置置信因子和更新时间标记来改进在线学习的效果。实验表明,模型能够较好地反映学习者知识掌握状况和快速适应学习者知识掌握水平的变化,有助于ITS更好地评价学习者学习效果。     

基于对抗学习和知识蒸馏的神经网络压缩算法

针对基于深度学习的人脸识别模型难以在嵌入式设备进行部署和实时性能差的问题,深入研究了现有的模型压缩和加速算法,提出了一种基于知识蒸馏和对抗学习的神经网络压缩算法。算法框架由三部分组成,预训练的大规模教师网络、轻量级的学生网络和辅助对抗学习的判别器。改进传统的知识蒸馏损失,增加指示函数,使学生网络只学习教师网络正确识别的分类概率;鉴于中间层特征图具有丰富的高维特征,引入对抗学习策略中的判别器,鉴别学生网络与教师网络在特征图层面的差异;为了进一步提高学生网络的泛化能力,使其能够应用于不同的机器视觉任务,在训练的后半部分教师网络和学生网络相互学习,交替更新,使学生网络能够探索自己的最优解空间。分别在CASIA WEBFACE和CelebA两个数据集上进行验证,实验结果表明知识蒸馏得到的小尺寸学生网络相较全监督训练的教师网络,识别准确率仅下降了1.5%左右。同时将本研究所提方法与面向特征图知识蒸馏算法和基于对抗学习训练的模型压缩算法进行对比,所提方法具有较高的人脸识别准确率。     

基于一维卷积神经网络的网络流量分类方法

针对传统机器学习算法对于流量分类的瓶颈问题,提出基于一维卷积神经网络模型的应用程序流量分类算法。将网络流量数据集进行数据预处理,去除无关数据字段,并使数据满足卷积神经网络的输入特性。设计了一种新的一维卷积神经网络模型,从网络结构、超参数空间以及参数优化方面入手构造了最优分类模型。该模型通过卷积层自主学习数据特征,解决了传统基于机器学习的流量分类算法中特征选择问题。通过网络公开数据集进行模型测试,相比于传统的一维卷积神经网络模型,所设计的神经网络模型的分类准确率提升了16.4%,总分类时间节省了71.48%。另外在类精度、召回率以及[F1]分数方面都有较好的提升。     

基于混沌PSO算法优化RBF网络入侵检测模型

针对网络安全中异常入侵检测,给出了一种构建最优神经网络入侵模型的方法。采用混沌扰动改进粒子群优化算法,优化径向基函数RBF神经网络入侵模型。把网络特征子集和RBF神经网络参数编码成一个粒子,通过粒子间的信息交流与协作快速找到全局最优粒子极值。在KDD Cup 99数据集进行仿真实验,实验数据表明,建立了一种检测率高、速度快的网络入侵检测模型。     

基于神经网络的特征结构并行提取算法研究

为并行实时地提取数据协方差矩阵的信号特征结构,从特征结构并行提取的约束优化问题表示入手,利用梯度方法和迭代法构建了可实时并行提取信号特征向量矩阵的直接神经网络求解法和基于能量函数的神经网络求解算法,并形成了相关迭代学习算法.理论分析表明当数据样本足够大时,算法的迭代结果就是数据协方差矩阵信号特征结构的一个良好估计,同时计算机仿真亦验证了算法的有效性.另外,仿真试验亦表明可以通过调节加权矩阵D的对角元来控制算法的收敛速度.     

Predicting Aflatoxin Contamination in Peanuts: A Genetic Algorithm/Neural Network Approach

Aflatoxin contamination in peanut crops is a problem of significant health and financial importance. Predicting aflatoxin levels prior to crop harvest is useful for minimizing the impact of a contaminated crop and is the goal of our research. Backpropagation neural networks have been used to model problems of this type, however development of networks poses the complex problem of setting values for architectural features and backpropagation parameters. Genetic algorithms have been used in other studies to determine parameters for backpropagation neural networks. This paper describes the development of a genetic algorithm/backpropagation neural network hybrid (GA/BPN) in which a genetic algorithm is used to find architectures and backpropagation parameter values simultaneously for a backpropagation neural network that predicts aflatoxin contamination levels in peanuts based on environmental data. Learning rate, momentum, and number of hidden nodes are the parameters that are set by the genetic algorithm. A three-layer feed-forward network with logistic activation functions is used. Inputs to the network are soil temperature, drought duration, crop age, and accumulated heat units. The project showed that the GA/BPN approach automatically finds highly fit parameter sets for backpropagation neural networks for the aflatoxin problem.     

基于贝叶斯方法的神经网络非线性模型辨识

研究了基于贝叶斯推理的多层前向神经网络训练算法,以提高网络的泛化性能。在网络目标函数中引入表示网络结构复杂性的惩罚项,以便能够在训练优化过程中降低网络结构的复杂性,达到避免网络过拟合的目的。训练过程中使用显式的概率分布假设对模型进行分析和推断,根据融入先验分布的假设和依据,获取网络参数和正则化参数的后验条件概率,并基于后验分布的贝叶斯推理得出最优化参数。利用上述算法训练前向网络,对一个微型锅炉对象进行了模型辨识,通过测试,证明所辨识出的对象模型能够较好地表现出对象的动态行为,且具有较好的泛化性能。     

基于跨越连接的多层前馈神经网络结构分析

任何连接方式的神经网络总可以归结为跨越连接网络。在传统多层前馈神经网络算法的基础上,提出了完全全连接神经网络的概念,给出了基于跨越连接的多层前馈神经网络算法。通过分析多层前馈神经网络的误差函数,从理论上证明了：相对于无跨越连接网络,基于跨越连接的多层前馈神经网络能以更加简洁的结构逼近理想状态。最后,用一个隐层神经元解决了XOR问题。     

一种粒子群优化的神经网络综合训练算法研究

基于粒子群优化算法,将灰色关联分析法应用到BP神经网络隐层结点数的确定,实现了BP神经网络结构的优化;然后将贝叶斯正则法应用于神经网络训练,进一步提高网络的泛化性能。仿真结果表明泛化能力明显优于其他改进的BP算法,拟合效果较好。     

An Incremental Algorithm for Parallel Training of the Size and the Weights in a Feedforward Neural Network

An algorithm of incremental approximation of functions in a normed linearspace by feedforward neural networks is presented. The concept of variationof a function with respect to a set is used to estimate the approximationerror together with the weight decay method, for optimizing the size andweights of a network in each iteration step of the algorithm. Two alternatives, recursively incremental and generally incremental, are proposed. In the generally incremental case, the algorithm optimizes parameters of all units in the hidden layer at each step. In the recursively incremental case, the algorithm optimizes the parameterscorresponding to only one unit in the hidden layer at each step. In thiscase, an optimization problem with a smaller number of parameters is beingsolved at each step.     

Shallow and deep learning for image classification

The paper is focused on the idea to demonstrate the advantages of deep learning approaches over ordinary shallow neural network on their comparative applications to image classifying from such popular benchmark databases as FERET and MNIST. An autoassociative neural network is used as a standalone program realized the nonlinear principal component analysis for prior extracting the most informative features of input data for neural networks to be compared further as classifiers. A special study of the optimal choice of activation function and the normalization transformation of input data allows to improve efficiency of the autoassociative program. One more study devoted to denoising properties of this program demonstrates its high efficiency even on noisy data. Three types of neural networks are compared: feed-forward neural net with one hidden layer, deep network with several hidden layers and deep belief network with several pretraining layers realized restricted Boltzmann machine. The number of hidden layer and the number of hidden neurons in them were chosen by cross-validation procedure to keep balance between number of layers and hidden neurons and classification efficiency. Results of our comparative study demonstrate the undoubted advantage of deep networks, as well as denoising power of autoencoders. In our work we use both multiprocessor graphic card and cloud services to speed up our calculations. The paper is oriented to specialists in concrete fields of scientific or experimental applications, who have already some knowledge about artificial neural networks, probability theory and numerical methods.     

FANNC: A Fast Adaptive Neural Network Classifier

In this paper, a fast adaptive neural network classifier named FANNC is proposed. FANNC exploits the advantages of both adaptive resonance theory and field theory. It needs only one-pass learning, and achieves not only high predictive accuracy but also fast learning speed. Besides, FANNC has incremental learning ability. When new instances are fed, it does not need to retrain the whole training set. Instead, it could learn the knowledge encoded in those instances through slightly adjusting the network topology when necessary, that is, adaptively appending one or two hidden units and corresponding connections to the existing network. This characteristic makes FANNC fit for real-time online learning tasks. Moreover, since the network architecture is adaptively set up, the disadvantage of manually determining the number of hidden units of most feed-forward neural networks is overcome. Benchmark tests show that FANNC is a preferable neural network classifier, which is superior to several other neural algorithms on both predictive accuracy and learning speed. Received 10 February 1999 / Revised 21 June 1999 / Accepted in revised form 11 October 1999     

基于极限学习机的股票价格预测

极限学习机（ Extreme Learning Machine , ELM）是一种新型的单馈层神经网络算法,克服了传统的误差反向传播方法需要多次迭代,算法的计算量和搜索空间大的缺点,只需要设置合适的隐含层节点个数,为输入权和隐含层偏差进行随机赋值,一次完成无需迭代。研究表明股票市场是一个非常复杂的非线性系统,需要用到人工智能理论、统计学理论和经济学理论。本文将极限学习机方法引入股票价格预测中,通过对比支持向量机（ Support Vector Machine , SVM）和误差反传神经网络（ Back Propagation Neural Network , BP神经网络）,分析极限学习机在股票价格预测中的可行性和优势。结果表明极限学习机预测精度高,并且在参数选择及训练速度上具有较明显的优势。     

大数据下基于特征图的深度卷积神经网络

针对大数据环境下DCNN(deep convolutional neural network)算法中存在网络冗余参数过多、参数寻优能力不佳和并行效率低的问题,提出了大数据环境下基于特征图和并行计算熵的深度卷积神经网络算法MR-FPDCNN(deep convolutional neural network algorithm based on feature graph and parallel computing entropy using MapReduce)。该算法设计了基于泰勒损失的特征图剪枝策略FMPTL(feature map pruning based on Taylor loss),预训练网络,获得压缩后的DCNN,有效减少了冗余参数,降低了DCNN训练的计算代价。提出了基于信息共享搜索策略ISS(information sharing strategy)的萤火虫优化算法IFAS(improved firefly algorithm based on ISS),根据“IFAS”算法初始化DCNN参数,实现DCNN的并行化训练,提高网络的寻优能力。在Reduce阶段提出了...     

Combined projection and kernel basis functions for classification in evolutionary neural networks

This paper proposes a hybrid neural network model using a possible combination of different transfer projection functions (sigmoidal unit, SU, product unit, PU) and kernel functions (radial basis function, RBF) in the hidden layer of a feed-forward neural network. An evolutionary algorithm is adapted to this model and applied for learning the architecture, weights and node typology. Three different combined basis function models are proposed with all the different pairs that can be obtained with SU, PU and RBF nodes: product–sigmoidal unit (PSU) neural networks, product–radial basis function (PRBF) neural networks, and sigmoidal–radial basis function (SRBF) neural networks; and these are compared to the corresponding pure models: product unit neural network (PUNN), multilayer perceptron (MLP) and the RBF neural network. The proposals are tested using ten benchmark classification problems from well known machine learning problems. Combined functions using projection and kernel functions are found to be better than pure basis functions for the task of classification in several datasets.