Multiplicative neuron model artificial neural network based on Gaussian activation function

Multiplicative neuron model-based artificial neural networks are one of the artificial neural network types which have been proposed recently and have produced successful forecasting results. Sigmoid activation function was used in multiplicative neuron model-based artificial neural networks in the previous studies. Although artificial neural networks which involve the use of radial basis activation function produce more successful forecasting results, Gaussian activation function has not been used for multiplicative neuron model yet. In this study, rather than using a sigmoid activation function, Gaussian activation function was used in multiplicative neuron model artificial neural network. The weights of artificial neural network and parameters of activation functions were optimized by guaranteed convergence particle swarm optimization. Two major contributions of this study are as follows: the use of Gaussian activation function in multiplicative neuron model for the first time and the optimizing of central and propagation parameters of activation function with the weights of artificial neural network in a single optimization process. The superior forecasting performance of the proposed Gaussian activation function-based multiplicative neuron model artificial neural network was proved by applying it to real-life time series.     

Frequency-based multilayer neural network with on-chip learning andenhanced neuron characteristics

A new digital architecture of the frequency-based multilayer neural network (MNN) with on-chip learning is proposed. As the signal level is expressed by the frequency, the multiplier is replaced by a simple frequency converter, and the neuron unit uses the voting circuit as the nonlinear adder to improve the nonlinear characteristic. In addition, the pulse multiplier is employed to enhance the neuron characteristics. The backpropagation algorithm is modified for the on-chip learning. The proposed MNN architecture is implemented on field programmable gate arrays (FPGA) and the various experiments are conducted to test the performance of the system. The experimental results show that the proposed neuron has a very good nonlinear function owing to the voting circuit. The learning behavior of the MNN with on-chip learning is also tested by experiments, which show that the proposed MNN has good learning and generalization capabilities. Simple and modular structure of the proposed MNN leads to a massive parallel and flexible network architecture, which is well suited for VLSI implementation.     

基于神经元特性的径向基函数神经网络自组织设计方法

针对径向基神经函数(RBF)网络隐层结构难以确定的问题,本文介绍了一种基于神经元特性的RBF神经网络自组织设计方法,该方法将神经元的激活活性、显著性、相关性相结合设计RBF(ASC–RBF)神经网络.首先利用神经元的激活活性,实现隐含层神经元的自适应增加,结合神经元的显著性以及神经元之间的相关性,实现神经元的自适应替换和合并,完成网络自组织设计并提高网络的紧凑性,然后利用二阶梯度算法对网络参数进行修正学习,保证了RBF网络的精度;另外,针对网络结构自组织机制给出了稳定性分析;最后通过两个基准非线性系统建模仿真实验以及实际污水处理过程水质参数预测实验验证,证明该算法的有效性.对比实验结果表明, ASC–RBF神经网络与现有的自组织网络相比,在保证泛化性能的同时,该网络的训练速度更快,而且有更紧凑的网络结构.     

Robust learning algorithm for multiplicative neuron model artificial neural networks

The two most commonly used types of artificial neural networks (ANNs) are the multilayer feed-forward and multiplicative neuron model ANNs. In the literature, although there is a robust learning algorithm for the former, there is no such algorithm for the latter. Because of its multiplicative structure, the performance of multiplicative neuron model ANNs is affected negatively when the dataset has outliers. On this issue, a robust learning algorithm for the multiplicative neuron model ANNs is proposed that uses Huber's loss function as fitness function. The training of the multiplicative neuron model is performed using particle swarm optimization. One principle advantage of this algorithm is that the parameter of the scale estimator, which is an important factor affecting the value of Huber's loss function, is also estimated with the proposed algorithm. To evaluate the performance of the proposed method, it is applied to two well-known real world time series datasets, and also a simulation study is performed. The algorithm has superior performance both when it is applied to real world time series datasets and the simulation study when compared with other ANNs reported in the literature. Another of its advantages is that, for datasets with outliers, the results are very close to the results obtained from the original datasets. In other words, we demonstrate that the algorithm is unaffected by outliers and has a robust structure.     

基于神经网络的需求预测模型

零售业的销售过程中积累了大量数据,如何从这些海量数据中提取知识、建立有效的需求预测模型,为零售商提供市场和趋势分析、降低库存成本是零售行业亟待解决的问题。在传统的零售业需求预测模型——Holt-Winter模型中应用神经网络方法,使得需求预测不依赖于数学模型的精度,预测模型中的季节性影响因子等参数能够根据预测误差作相应调整,避免了传统算法中误差的累积,大大提高了预测精度。利用Excel内嵌的VBA实现了该算法,使需求预测能够根据用户需要实现,并提供可视化的结果。     

Blur identification by multilayer neural network based on multivalued neurons.

A multilayer neural network based on multivalued neurons (MLMVN) is a neural network with a traditional feedforward architecture. At the same time, this network has a number of specific different features. Its backpropagation learning algorithm is derivative-free. The functionality of MLMVN is superior to that of the traditional feedforward neural networks and of a variety kernel-based networks. Its higher flexibility and faster adaptation to the target mapping enables to model complex problems using simpler networks. In this paper, the MLMVN is used to identify both type and parameters of the point spread function, whose precise identification is of crucial importance for the image deblurring. The simulation results show the high efficiency of the proposed approach. It is confirmed that the MLMVN is a powerful tool for solving classification problems, especially multiclass ones.     

Face detection based on multilayer feed-forward neural network and Haar features

Fast and accurate detection of a facial data is crucial for both face and facial expression recognition systems. These systems include internet protocol video surveillance systems, crime scene photographs systems, and criminals' databases. The aim for this study is both improvement of accuracy and speed. The salient facial features are extracted through Haar techniques. The sizes of the images are reduced by Bessel down-sampling algorithm. This method preserved the details and perceptual quality of the original image. Then, image normalization was done by anisotropic smoothing. Multilayer feed-forward neural network with a back-propagation algorithm was used as classifier. A detection accuracy of 98.5% with acceptable false positives was registered with test sets from FDDB, CMU-MIT, and Champions databases. The speed of execution was also promising. An evaluation of the proposed method with other popular detectors on the FDDB set shows great improvement.     

Feed-forward multilayer neural network model for vehicle lateral guidance control

An advanced vehicle lateral guidance control technology is necessary in order to develop intelligent transportation and manufacturing systems with flexibility and immediate adaptability. PID control, optimal control, and fuzzy control have often been used for designing a vehicle lateral guidance controller; in addition automatic guidance methods by spline curve and inverse dynamics are also used in mobile robots (e.g. differential drive), but they are not sufficient to develop a highly intelligent vehicle lateral guidance controller which can adapt to varying environments, because they lack some behavior like learning ability and adaptability. In this paper, the possibility to apply neural networks for developing a vehicle lateral guidance controller is exposed. A new neuron activation function suitable for vehicle lateral guidance control is suggested, a feed-forward multilayer neural network (FMNN) with the suggested neuron activation function is proposed and a vehicle lateral guidance controller (VLGC) is developed by use of the FMNN. The VLGC can be applied to automobiles of different parameters and roads of various widths. It can be also applied to mobile robots. Its input variables are proposed to be defined as kind of relative quantities by using the road width, automotive parameter, automotive position, and orientation on the corner course as 90°. Its output variable is the automotive steering angle. Its teaching data are collected by automobile driving simulation, and its connection weights and threshold values are tuned through the error back-propagation algorithm. The training process and the result of neural network by different learning rate coefficients and momentum parameters are compared. Four VLGCs are generated through training by using different learning rate coefficients, momentum parameters, and repeat training times. Automated guided automobile simulations and mobile robot experiments for each VLGC are carried out. Good training result as well as automated guided simulation and experimental results are obtained.     

基于BP神经网络的颜色补偿模型

针对光源渐变等因素在机器视觉中产生的相关问题,提出了一种基于BP神经网络的图像颜色校正方法.该方法通过合适的训练集对BP神经网络进行大量训练,得到光照变化前后图像像素点之间的映射关系,从而建立了在渐变光照环境下的颜色恒常性模型.该方法不需要内建约束的自适应模型,对于输入的数据不需要对表面属性做特定假设,拥有自适应、自学习的特点.实验结果表明,该模型对室内真实环境中渐变日光下颜色的识别表现出较好的颜色恒常性.     

基于神经网络模型的网络入侵检测的研究

门限循环单元是一种循环神经网络模型的变体,它改进了长短期记忆模型。经验表明,循环神经网络被广泛应用于不同类型的机器学习场景中,在解决自然语言处理、语音识别、文本分类等问题均良中有好表现。通常情况下,循环神经网络模型会使用Softmax函数作为顶层分类器,用交叉熵函数计算损失。在判断网络数据是否为恶意入侵数据这项任务中,可以使用二分类器节省计算开销。因此,对传统模型予以改进,用线性支持向量机取代Softmax函数作为模型的顶层分类器,用基于边界的损失函数取代交叉熵函数。实验结果表明,改进的模型在提升准确率和时间复杂度方面比传统模型表现好。     

Neural network learning with generalized-mean based neuron model

The advances in biophysics of computations and neurocomputing models have brought the foreground importance of dendritic structure of neuron. These structures are assumed as basic computational units of the neuron, capable of realizing the various mathematical operations. The well structured higher order neurons have shown improved computational power and generalization ability. However, these models are difficult to train because of a combinatorial explosion of higher order terms as the number of inputs to the neuron increases. In this paper we present a neural network using new neuron architecture i.e., generalized mean neuron (GMN) model. This neuron model consists of an aggregation function which is based on the generalized mean of all the inputs applied to it. The resulting neuron model has the same number of parameters with improved computational power as the existing multilayer perceptron (MLP) model. The capability of this model has been tested on the classification and time series prediction problems.     

基于前馈多层感知器的网络入侵检测的多数据包分析

提出了一种新型网络入侵检测模型,在该模型中,首先将截获的数据包结合历史数据包数据库进行协议分析,找出可能存在的入侵行为的相关数据包,然后采用前馈多层感知器神经网络对这些相关的数据包进行回归分析,最终获得检测结果。该模型与传统采用单数据包检测方式的网络入侵检测系统(NIDS)模型相比,具有更低的漏检率。     

基于混合卷积神经网络和循环神经网络的入侵检测模型

针对电力信息网络中的高级持续性威胁问题,提出一种基于混合卷积神经网络（CNN）和循环神经网络（RNN）的入侵检测模型。该模型根据网络数据流量的统计特征对当前网络状态进行分类。首先,获取日志文件中网络流量的各统计值,进行特征编码、归一化等预处理工作;然后,通过深度卷积神经网络中可变卷积核提取不同主机入侵流量之间空间相关特征;最后,将已经处理好的包含空间相关特征的数据在时间上错开排列,利用深度循环神经网络挖掘入侵流量的时间相关特征。实验结果表明,该模型相对于传统的机器学习模型在曲线下方的面积（AUC）上提升了7.5%~14.0%,同时误报率降低了83.7%~52.7%。所提模型能准确地识别网络流量的类别,大幅降低误报率。     

Robust fuzzy clustering neural network based on ɛ-insensitive loss function

In the paper, as an improvement of fuzzy clustering neural network FCNN proposed by Zhang et al., a novel robust fuzzy clustering neural network RFCNN is presented to cope with the sensitive issue of clustering when outliers exist. This new algorithm is based on Vapnik's ɛ-insensitive loss function and quadratic programming optimization. Our experimental results demonstrate that RFCNN has much better robustness for outliers than FCNN.     

一类基于神经网络非线性观测器的鲁棒故障检测和诊断

利用神经网络的非线性建模能力，对一类具有建模不确定项的非线性系统提出一种基于观测器的故障检测和诊断的方法。设计的观测器不仅能实现故障检测，而旦应用神经网络设计的故障估计器能在线估计系统中的故障向量。通过分析验证了该方法对系统中的建模误差和外部扰动具有良好的鲁棒性。仿真结果表明所提出的方法是有效的。     

Robust local stability of multilayer recurrent neural networks

We derive a condition for robust local stability of multilayer recurrent neural networks with two hidden layers. The stability condition follows from linking theories about linearization, robustness analysis of linear systems under nonlinear perturbation, and matrix inequalities. A characterization of the basin of attraction of the origin is given in terms of the level set of a quadratic Lyapunov function. Similar to the NL(q) theory, the local stability is imposed around the origin and the apparent basin of attraction is made large by applying the criterion, while the proven basin of attraction is relatively small due to conservatism of the criterion. Modification of the dynamic backpropagation by the new stability condition is discussed and illustrated by simulation examples.     

基于神经网络的级联软件可靠性模型

针对一般经典软件可靠性模型适用范围的局限性问题和预测精度问题,提出了一种新的级联模型.将4个经典软件可靠性模型的输出作为误差背向传播(error back propagation,BP)神经网络的输入,级联组合成一个软件可靠性模型,称之为级联软件可靠性模型.通过对一组经典的实际软件故障数据SYS1进行实验,将级联软件可靠性模型与4个经典软件可靠性模型预测的结果进行对比,结果表明级联软件可靠性模型的预测精度要远远高于4个经典软件可靠性模型,而且具有更好的通用性.     

A Neural Network Model Based on Logical Operations

On the basis of analyzing some neural network storage capacity problems a network model comprising a new encoding and recalling scheme is presented.By using some logical operations which operate on the binary pattern strings during information processing procedure the model can reach a high storage capacity for a certain size of network framework.     

基于神经网络集成的专家系统模型

提出一种基于神经网络集成的专家系统模型,并给出神经网络集成的构造算法.在该模型中神经网络集成作为专家系统的一个内嵌模块,用于专家系统的知识获取,克服了传统专家系统在知识获取中的＂瓶颈＂问题.并将该模型用于图书剔旧系统中,初步建成基于神经网络集成的图书剔旧专家系统原型.     

Robust visual tracking based on convolutional neural network with extreme learning machine

Multimedia Tools and Applications - Recently, deep learning has attracted substantial attention as a promising solution to many problems in computer vision. Among various deep learning...