Design of highly effective multilayer feedforward neural network by using genetic algorithm

This paper presents a highly effective and precise neural network method for choosing the activation functions (AFs) and tuning the learning parameters (LPs) of a multilayer feedforward neural network by using a genetic algorithm (GA). The performance of the neural network mainly depends on the learning algorithms and the network structure. The backpropagation learning algorithm is used for tuning the network connection weights, and the LPs are obtained by the GA to provide both fast and reliable learning. Also, the AFs of each neuron in the network are automatically chosen by a GA. The present study consists of 10 different functions to accomplish a better convergence of the desired input–output mapping. Test studies are performed to solve a set of two-dimensional regression problems for the proposed genetic-based neural network (GNN) and conventional neural network having sigmoid AFs and constant learning parameters. The proposed GNN has also been tested by applying it to three real problems in the fields of environment, medicine, and economics. Obtained results prove that the proposed GNN is more effective and reliable when compared with the classical neural network structure.     

A neural network approach to freeway network traffic control

The paper investigates the application of a feedforward neural network approach to freeway network control via variable direction recommendations at bifurcation locations. A nonlinear control problem is formulated and solved first by use of computationally expensive nonlinear optimization techniques. A feedforward neural network is then trained by optimally adjusting its weights so as to reproduce the optimal control law for a limited number of traffic scenarios. Generalisation properties of the neural network are investigated and a discussion of advantages and disadvantages compared with alternative control approaches is provided.     

多层前向神经网络的快速学习算法及其应用

针对目前多层前向神经网络学习算法存在的不足,提出一种多层前向神经网络的快速学习算法,它不仅符合生物神经网络的基本特征,而且算法简单,学习收敛速度快,具有线性、非线性逼近精度高等特性.以二杆机械手逆运动学建模作为应用实例,仿真结果表明该方法是有效的,其算法与收敛速度更优于BP网络.     

溶解氧浓度的前馈神经网络建模控制方法

针对污水处理过程溶解氧(DO)浓度控制问题,提出了一种基于前馈神经网络的建模控制方法(FNNMC).本文构造了神经网络建模控制系统,通过对建模神经网络和控制神经网络隐含层学习率的分析,证明了学习算法的收敛性以及整个系统的稳定性.最后,本文基于国际基准的Benchmark Simulation Model No.1 (BSMl)进行了仿真实验,验证了合理选取学习率的重要性,并通过与PID和模型预测控制(MPC)等已有控制方法的比较,验证了神经网络建模控制方法针对污水处理过程溶解氧浓度控制具有良好的建模能力,更高的控制精度以及更好的动态响应能力.     

基于量子门组的卷积神经网络设计与实现

为进一步提高卷积神经网络的训练速度,减少训练成本,建立了量子门组卷积神经网络模型（Quantum Gate Convolutional Neural Network,QGCNN）。为了构建QGCNN网络结构,依据传统CNN结构的特点,给出卷积算术线路（Convolutional Arithmetic Circuit,ConvAC）的定义。用张量分解来说明ConvAC的权值系数之间的关系,为构建QGCNN提供理论依据。将QGCNN分为输入表示层、隐藏层和输出层,在此基础上实现对数据进行量子编码,利用量子门组完成数据初始化,网络参数更新等操作。将QGCNN应用到数字手写体识别中,实验结果表明,该方法在手写体识别的准确率和收敛速度上有不错的效果。     

Financial ratings with scarce information: A neural network approach

On a wake of Basel II Accord in 2004, banks and financial institutions can build an internal rating system. This work focuses on Italian small firms that are more hard to judge because quite often financial data are not simply available. The aim of this paper is to propose a simulation model for assigning rating judgements to these firms, using poor financial information.The proposed model produces a simulated counterpart of Bureau van Dijk-K Finance (BvD) rating judgements. It is clear that there are problems when small firms must be judged because it is difficult to obtain financial data; indeed in Italy these enterprises must deposit the balance-sheet in reduced form. Suggested methodology is a three-layer process where each of them is formed by, respectively, one, two and four feed-forward artificial neural networks with back-propagation algorithm. The proposed model is a good solution for evaluating small firms with poor financial information but not only: the research underlines and supports the ability of artificial neural networks of learning and reproducing some aspects or some features or behaviours of reality.     

非线性量化小脑模型关节控制器神经网络控制器

常规小脑模型关节控制器(CMAC)神经网络采用线性均匀量化,稳态控制精度与量化级数相关,增加量化级数可提高稳态精度但会导致内存空间和计算量的增加.本文提出一种可采用幂函数、高斯、分段3种非线性量化方法的非线性量CMAC神经网络,并分析了非线性量化CMAC的收敛性,解释了非线性量化提高稳态精度的本质.面向一阶惯性环节、二阶系统、一阶时变系统及二阶时变系统,分别跟踪方波、斜坡、正弦波、三角波和加速度等输入信号,仿真验证了非线性量化CMAC神经网络控制器的有效性,给出了不同非线性量化方法的适用性.结果表明,非线性量化CMAC参数容易设定,物理意义清晰,与常规CMAC对比,其快速性和控制精度显著提高,可以有效解决实际复杂非线性时变系统的控制.     

Human-like fault diagnosis using a neural network implementation of plausibility and relevance

In real systems, fault diagnosis is performed by a human diagnostician, and it encounters complex knowledge associations, both for normal and faulty behaviour of the target system. The human diagnostician relies on deep knowledge about the structure and the behaviour of the system, along with shallow knowledge on fault-to-manifestation patterns acquired from practice. This paper proposes a general approach to embed deep and shallow knowledge in neural network models for fault diagnosis by abduction, using neural sites for logical aggregation of manifestations and faults. All types of abduction problems were considered. The abduction proceeds by plausibility and relevance criteria multiply applied. The neural network implements plausibility by feed-forward links between manifestations and faults, and relevance by competition links between faults. Abduction by plausibility and relevance is also used for decision on the next best test along the diagnostic refinement. A case study on an installation in a rolling mill plant is presented.     

基于自联想神经网络的传感器故障诊断

提出一种基于自联想神经网络(AANN)的新算法用于系统中传感器故障诊断。阐述了AANN的结构和算法。具体说明了搜寻2个故障传感器和恢复信号的方法。用改进的AANN诊断有噪声情况下传感器跳变故障并恢复信号。本方法有易实现、结构简单的优点,仿真结果表明:本方法是可行的。     

神经网络前向传播在GPU上的实现

基于CUDA架构在GPU上实现了神经网络前向传播算法,该算法利用神经网络各层内神经元计算的并行性,每层使用一个Kernel函数来并行计算该层神经元的值,每个Kernel函数都根据神经网络的特性和CUDA架构的特点进行优化。实验表明,该算法比普通的CPU上的算法快了约7倍。研究结果对于提高神经网络的运算速度以及CUDA的适用场合都有参考价值。     

A neural network controller for flexible-link robots

The object in this paper is to achieve tracking control of a partially unknown flexible-link robot arm. It is shown how to stabilize the internal dynamics by selecting a physically meaningful modified performance output for tracking; this output is the slow portion of the link-tip motions. That is, the tracking requirement is relaxed so that the internal dynamics are controllable through a boundary layer correction. The controller is composed of singular-perturbation based fast control and an outer-loop slow control. The slow subsystem is controlled by a neural network (NN) for feedback linearization, plus a PD outer-loop for tracking, and a robustifying term to assure the closed-loop stability. No off-line learning or training is needed for the NN. Tracking and stability are proven using Lyapunov techniques that yield a novel modified NN weight tuning algorithm.The research is supported by NSF grant IRI-9216545 and EPRI Grant RP8030-09.     

A fast learning algorithm of neural network with tunable activation function

This paper presents a modified structure of a neural network with tunable activation function and provides a new learning algorithm for the neural network training. Simulation results of XOR problem, Feigenbaum function, and Henon map show that the new algorithm has better performance than BP (back propagation) algorithm in terms of shorter convergence time and higher convergence accuracy. Further modifications of the structure of the neural network with the faster learning algorithm demonstrate simpler structure with even faster convergence speed and better convergence accuracy.     

一种多模块协同参与的神经网络

针对单一神经网络训练时间长、对复杂问题处理精度较低、易陷入局部极小等问题,提出了一种多模块协同参与信息处理的神经网络.该神经网络具有层级结构,基于条件模糊聚类技术对样本进行分类,根据分类结果实现对神经网络的模块划分,采用代数算法对网络权值进行求解,基于距离测度设计了处理输入信息的子网络选择方法.为提高神经网络对复杂问题的逼近能力,选择数目不等的多个子网络参与给定输入的协同处理,采取"分而治之"与"集成学习"相结合方法以提高网络的性能.实验表明,对于复杂问题,这种多模块协同参与的神经网络可以有效地提高网络的逼近精度,训练时间也优于单一网络.     

A neural network job-shop scheduler

This paper focuses on the development of a neural network (NN) scheduler for scheduling job-shops. In this hybrid intelligent system, genetic algorithms (GA) are used to generate optimal schedules to a known benchmark problem. In each optimal solution, every individually scheduled operation of a job is treated as a decision which contains knowledge. Each decision is modeled as a function of a set of job characteristics (e.g., processing time), which are divided into classes using domain knowledge from common dispatching rules (e.g., shortest processing time). A NN is used to capture the predictive knowledge regarding the assignment of operation’s position in a sequence. The trained NN could successfully replicate the performance of the GA on the benchmark problem. The developed NN scheduler was then tested against the GA, Attribute-Oriented Induction data mining methodology and common dispatching rules on a test set of randomly generated problems. The better performance of the NN scheduler on the test problem set compared to other methods proves the feasibility of NN-based scheduling. The scalability of the NN scheduler on larger problem sizes was also found to be satisfactory in replicating the performance of the GA.     

Validating a neural network application: The case of financial diagnosis

It has been argued that neural network applications should be benchmarked using several data sets of realistic and real problems, and competing algorithms (Prechelt, 1995). However, if applying a neural network model to a particular real problem is in focus, validation should be considered as a suitability evaluation in which several bases of evaluation are combined in a composite judgment. In this paper, five bases of such evaluation are introduced and applied to the validation of a neural network model of financial diagnosis.     

A small spiking neural network with LQR control applied to the acrobot

This paper presents the results of a computer simulation which, combined a small network of spiking neurons with linear quadratic regulator (LQR) control to solve the acrobot swing-up and balance task. To our knowledge, this task has not been previously solved with spiking neural networks. Input to the network was drawn from the state of the acrobot, and output was torque, either directly applied to the actuated joint, or via the switching of an LQR controller designed for balance. The neural network’s weights were tuned using a (μ + λ)-evolution strategy without recombination, and neurons’ parameters, were chosen to roughly approximate biological neurons.     

限定记忆的前向神经网络在线学习算法研究

从理论上分析了隐含层激励函数满足Mercer条件的前向神经网络的数学本质,给出了网络学习的指导方向.提出3种网络在线学习算法,它们通过动态调整网络结构和权值来提高网络在线预测性能.算法完全符合统计学习理论提出的结构风险最小化原则,具有较快的学习收敛速度和良好的抗噪声能力.最后通过具体数值实验验证了上述算法的可行性和优越性.     

A Cross-layer based mapping for spiking neural network onto network on chip

Abstract

Network-on-Chip provides a packet-based and scalable inter-connected structure for spiking neural networks. However, existing neural mapping methods just distribute all neurons of a population into an on-chip network core or nearby cores sequentially. As there is no connection among population, the population based mapping degrades inter-neuron communicating performance between different cores. This paper presents a Cross-LAyer based neural MaPping method that maps synaptic connected neurons belonging to adjacent layers into the same on-chip network node. In order to adapt to various input patterns, the strategy also takes input spike rate into consideration and remap neurons for improving mapping efficiency. The method helps to reduce inter-core communication cost. The experimental results demonstrate the efficient results of the proposed mapping strategy in the aspect of spike transfer latency as well as dynamic energy cost improvement. In the applications of handwritten digits and edge extraction, in which the type of interconnection among neurons is different, the neural mapping algorithm reduces spike average transfer latency by maximum 42.83%, and reduces dynamic energy by maximum 36.29%.     

基于自适应模糊神经网络控制器的网络控制系统

网络控制系统中存在着时延、丢包、网络干扰等问题。针对网络控制系统中存在恶化系统的控制性能,甚至导致系统不稳定的因素,提出了一种基于自适应模糊神经网络控制器的网络控制系统,它能根据系统的实际输出与期望输出误差,利用自适应模糊控制和神经网络自学习的原理进行控制参数的自行调整,以符合控制系统的实际要求,同时,分析了网络延时,丢包率及网络干扰因素对系统性能的影响。利用TrueTime工具箱建立了包含自适应模糊神经网络控制器的网络控制系统的仿真模型,并将其分别与基于常规PID控制器的网络控制系统和基于模糊参数PID控制器的网络控制系统进行了比较。实验结果表明,在相同的网络环境下,基于自适应模糊神经网络控制器的网络控制系统的控制效果比基于常规的PID控制器和基于模糊参数PID控制器的要好,且具有较好的抗干扰能力和鲁棒性能。     

塔式神经网络在人脸验证上的新应用

首先提出BP神经网络在人脸验证上的应用方法,并在Cs_PCA方法的基础之上,提出一种“Cs_PCA+塔式神经网络”的人脸验证新模型（Cs_塔式）。传统的神经网络受到输入样本维数大小的限制,必须经过各种降维处理才能加以训练,受各种降维方法的限制,在降维过程中会丢失相应的数据信息,因此验证效果受到影响。针对此种情况提出了Cs_塔式方法,利用同样的方法,普通BP网在Cs_PCA基础上,利用PCA方法降维构成Cs_BP模型,并且遵照LAUSANNE协议在ORL人脸库上与Cs_塔式模型进行了比较。结果表明,塔式网络有着更好的验证效果。