Revised GMDH-type neural network algorithm self-selecting optimum neural network architecture

In this study, the revised group method of data handling (GMDH)-type neural network (NN) algorithm self-selecting the optimum neural network architecture is applied to the identification of a nonlinear system. In this algorithm, the optimum neural network architecture is automatically organized using two kinds of neuron architecture, such as the polynomial- and sigmoid function-type neurons. Many combinations of the input variables, in which the high order effects of the input variables are contained, are generated using the polynomial-type neurons, and useful combinations are selected using the prediction sum of squares (PSS) criterion. These calculations are iterated, and the multilayered architecture is organized. Furthermore, the structural parameters, such as the number of layers, the number of neurons in the hidden layers, and the useful input variables, are automatically selected in order to minimize the prediction error criterion defined as PSS.     

Plane-Gaussian artificial neural network

Multilayer perceptrons (MLPs) and radial basis functions networks (RBFNs) have been widely concerned in recent years. In this paper, based on k-plane clustering (kPC) algorithm, we propose a novel artificial network model termed as Plane-Gaussian network to enlarge the arsenal of the neural networks. This network adopts a so-called Plane-Gaussian activation function (PGF) in hidden neurons. Replacing traditional central point of Gaussian radial basis function (RBF) with central hyperplane, PGF forms a band-shaped rather than spheral-shaped receptive field in RBF, which makes PGF able to express its peculiar geometrical characteristics: locality and globality. Importantly, it is also proved that PGF network (PGFN) having one hidden layer is capable of universal approximation. As a universal approximator, PGFN gives an informal way of bridging the gap between MLP and RBFN. The experiments report comparison between training time and classification accuracies on some artificial and UCI datasets and conclude that (1) PGFN runs significantly faster than MLP and (2) PGFN has comparable or better classification performance than MLP and RBFN, especially in subspace-distributed datasets.     

Fault diagnosis based on pulse coupled neural network and probability neural network

In operation of mechanical equipment, fault diagnosis plays an important role. In this paper, a novel fault diagnosis method based on pulse coupled neural network (PCNN) and probability neural network (PNN) is presented. The shape information of shaft orbit provides an important basis for fault diagnosis. However, the feature extraction and classification of shaft orbit is difficult to realize automation. The PCNN technique has excellent performance in the feature extraction. In the present study, a PCNN combined with roundness method is used to extract the feature vector of shaft orbit, because time signature from a PCNN has the property of insensitive to rotation, scaling and translation. Meanwhile, roundness is also with the same properties. Further, the PNN is used to train the feature vectors and classify the vibration fault. By comparison with the back-propagation (BP) network and radial-basic function (RBF) network, the experimental result indicated the proposed approach achieved fast and efficient fault diagnosis.     

Mutation-based genetic neural network

Evolving gradient-learning artificial neural networks (ANNs) using an evolutionary algorithm (EA) is a popular approach to address the local optima and design problems of ANN. The typical approach is to combine the strength of backpropagation (BP) in weight learning and EA's capability of searching the architecture space. However, the BP's "gradient descent" approach requires a highly computer-intensive operation that relatively restricts the search coverage of EA by compelling it to use a small population size. To address this problem, we utilized mutation-based genetic neural network (MGNN) to replace BP by using the mutation strategy of local adaptation of evolutionary programming (EP) to effect weight learning. The MGNN's mutation enables the network to dynamically evolve its structure and adapt its weights at the same time. Moreover, MGNN's EP-based encoding scheme allows for a flexible and less restricted formulation of the fitness function and makes fitness computation fast and efficient. This makes it feasible to use larger population sizes and allows MGNN to have a relatively wide search coverage of the architecture space. MGNN implements a stopping criterion where overfitness occurrences are monitored through "sliding-windows" to avoid premature learning and overlearning. Statistical analysis of its performance to some well-known classification problems demonstrate its good generalization capability. It also reveals that locally adapting or scheduling the strategy parameters embedded in each individual network may provide a proper balance between the local and global searching capabilities of MGNN.     

Adaptive fuzzy inference neural network

An adaptive fuzzy inference neural network (AFINN) is proposed in this paper. It has self-construction ability, parameter estimation ability and rule extraction ability. The structure of AFINN is formed by the following four phases: (1) initial rule creation, (2) selection of important input elements, (3) identification of the network structure and (4) parameter estimation using LMS (least-mean square) algorithm. When the number of input dimension is large, the conventional fuzzy systems often cannot handle the task correctly because the degree of each rule becomes too small. AFINN solves such a problem by modification of the learning and inference algorithm.     

基于混合卷积神经网络和循环神经网络的入侵检测模型

针对电力信息网络中的高级持续性威胁问题,提出一种基于混合卷积神经网络（CNN）和循环神经网络（RNN）的入侵检测模型。该模型根据网络数据流量的统计特征对当前网络状态进行分类。首先,获取日志文件中网络流量的各统计值,进行特征编码、归一化等预处理工作;然后,通过深度卷积神经网络中可变卷积核提取不同主机入侵流量之间空间相关特征;最后,将已经处理好的包含空间相关特征的数据在时间上错开排列,利用深度循环神经网络挖掘入侵流量的时间相关特征。实验结果表明,该模型相对于传统的机器学习模型在曲线下方的面积（AUC）上提升了7.5%~14.0%,同时误报率降低了83.7%~52.7%。所提模型能准确地识别网络流量的类别,大幅降低误报率。     

基于ART 的RBF 网络结构设计

针对径向基函数(RBF)网络隐层结构难以确定的问题,基于自适应共振理论(ART)网络良好的在线分类特性,提出一种RBF网络结构设计算法。该算法将ART网络的聚类特性用于RBF网络结构设计中,通过对输入向量与已存模式的相似度比较将输入向量进行分类,确定隐含层节点个数和初始参数,使网络具有精简的结构。对典型非线性函数逼近的仿真结果表明,所提出的结构具有快速的学习能力和良好的逼近能力。     

A general regression neural network

A memory-based network that provides estimates of continuous variables and converges to the underlying (linear or nonlinear) regression surface is described. The general regression neural network (GRNN) is a one-pass learning algorithm with a highly parallel structure. It is shown that, even with sparse data in a multidimensional measurement space, the algorithm provides smooth transitions from one observed value to another. The algorithmic form can be used for any regression problem in which an assumption of linearity is not justified.     

A dynamic K-winners-take-all neural network

In this paper, a dynamic K-winners-take-all (KWTA) neural network, which can quickly identify the K-winning neurons whose activations are larger than the remaining ones, is proposed and analyzed. For N competitors, the proposed KWTA network is composed of N feedforward hardlimit neurons and three feedback neurons, which are used to determine the dynamic threshold. From theoretical analysis and simulation results, we found that the convergence of the proposed KWTA network, which requires Log(2)(N+1) iterations in average to complete a KWTA process, is independent of K, the number of the desired winners, and faster than that of the existing KWTA networks.     

A neural network job-shop scheduler

This paper focuses on the development of a neural network (NN) scheduler for scheduling job-shops. In this hybrid intelligent system, genetic algorithms (GA) are used to generate optimal schedules to a known benchmark problem. In each optimal solution, every individually scheduled operation of a job is treated as a decision which contains knowledge. Each decision is modeled as a function of a set of job characteristics (e.g., processing time), which are divided into classes using domain knowledge from common dispatching rules (e.g., shortest processing time). A NN is used to capture the predictive knowledge regarding the assignment of operation’s position in a sequence. The trained NN could successfully replicate the performance of the GA on the benchmark problem. The developed NN scheduler was then tested against the GA, Attribute-Oriented Induction data mining methodology and common dispatching rules on a test set of randomly generated problems. The better performance of the NN scheduler on the test problem set compared to other methods proves the feasibility of NN-based scheduling. The scalability of the NN scheduler on larger problem sizes was also found to be satisfactory in replicating the performance of the GA.     

Artificial neural network and wavelet neural network approaches for modelling of a solar air heater

This paper reports on a modelling study of new solar air heater (SAH) system by using artificial neural network (ANN) and wavelet neural network (WNN) models. In this study, a device for inserting an absorbing plate made of aluminium cans into the double-pass channel in a flat-plate SAH. A SAH system is a multi-variable system that is hard to model by conventional methods. As regards the ANN and WNN methods, it has a superior capability for generalization, and this capability is independent on the dimensionality of the input data’s. In this study, an ANN and WNN based methods were intended to adopt SAH system for efficient modelling. To evaluate prediction capabilities of different types of neural network models (ANN and WNN), their best architecture and effective training parameters should be found. The performance of the proposed methodology was evaluated by using several statistical validation parameters. Comparison between predicted and experimental results indicates that the proposed WNN model can be used for estimating the some parameters of SAHs with reasonable accuracy.     

The hysteretic Hopfield neural network

A new neuron activation function based on a property found in physical systems-hysteresis-is proposed. We incorporate this neuron activation in a fully connected dynamical system to form the hysteretic Hopfield neural network (HHNN). We then present an analog implementation of this architecture and its associated dynamical equation and energy function. We proceed to prove Lyapunov stability for this new model, and then solve a combinatorial optimization problem (i.e., the N-queen problem) using this network. We demonstrate the advantages of hysteresis by showing increased frequency of convergence to a solution, when the parameters associated with the activation function are varied.     

Multiresolution-based bilinear recurrent neural network

A multiresolution-based bilinear recurrent neural network (MBLRNN) is proposed in this paper. The proposed MBLRNN is based on the BLRNN that has robust abilities in modeling and predicting time series. The learning process is further improved by using a multiresolution-based learning algorithm for training the BLRNN so as to make it more robust for the prediction of time series data. The proposed MBLRNN is applied to the problems of network traffic prediction and electric load forecasting. Experiments and results on both practical problems show that the proposed MBLRNN outperforms both the traditional multilayer perceptron type neural network (MLPNN) and the BLRNN in the prediction accuracy.

一种多模块协同参与的神经网络

针对单一神经网络训练时间长、对复杂问题处理精度较低、易陷入局部极小等问题,提出了一种多模块协同参与信息处理的神经网络.该神经网络具有层级结构,基于条件模糊聚类技术对样本进行分类,根据分类结果实现对神经网络的模块划分,采用代数算法对网络权值进行求解,基于距离测度设计了处理输入信息的子网络选择方法.为提高神经网络对复杂问题的逼近能力,选择数目不等的多个子网络参与给定输入的协同处理,采取"分而治之"与"集成学习"相结合方法以提高网络的性能.实验表明,对于复杂问题,这种多模块协同参与的神经网络可以有效地提高网络的逼近精度,训练时间也优于单一网络.     

High-order MS CMAC neural network

A macro structure cerebellar model articulation controller (MS CMAC) was developed by connecting several 1D CMAC in a tree structure, which decomposes a multidimensional problem into a set of 1D subproblems, to reduce the computational complexity in multidimensional CMAC. Additionally, a trapezium scheme is proposed to assist MS CMAC to model nonlinear systems. However, this trapezium scheme cannot perform a real smooth interpolation, and its working parameters are obtained through cross-validation. A quadratic splines scheme is developed herein to replace the trapezium scheme in MS CMAC, named high-order MS CMAC (HMS CMAC). The quadratic splines scheme systematically transforms the stepwise weight contents of CMAC in MS CMAC into smooth weight contents to perform the smooth outputs. Test results affirm that the HMS CMAC has acceptable generalization in continuous function-mapping problems for nonoverlapping association in training instances. Nonoverlapping association in training instances not only significantly reduces the number of training instances needed, but also requires only one learning cycle in the learning stage.     

A-optimal convolutional neural network

Neural Computing and Applications - In this paper, we propose a novel data representation-classification model learning algorithm. The model is a convolutional neural network (CNN), and we learn...     

A neural network model of causality

This paper proposes a model for commonsense causal reasoning, based on the basic idea of neural networks. After an analysis of the advantages and limitations of existing accounts of causality, a fuzzy logic based formalism FEL is proposed that takes into account the inexactness and the cumulative evidentiality of commonsense causal reasoning, overcoming the limitations of existing accounts. Analyses concerning how FEL handles various aspects of commonsense causal reasoning are performed, in an abstract way. FEL can be implemented (naturally) in a neural (connectionist) network. This work also serves to link rule-based reasoning with neural network models, in that a rule-encoding scheme (FEL) is equated directly to a neural network model.     

Improved neural network for SVM learning

The recurrent network of Xia et al. (1996) was proposed for solving quadratic programming problems and was recently adapted to support vector machine (SVM) learning by Tan et al. (2000). We show that this formulation contains some unnecessary circuits which, furthermore, can fail to provide the correct value of one of the SVM parameters and suggest how to avoid these drawbacks.     

Quantized STDP-based online-learning spiking neural network

Neural Computing and Applications - In this work, we report a spike-timing-dependent plasticity (STDP)-based weight-quantized/binarized online-learning spiking neural network (SNN). The SNN uses...     

A neural network regulator for turbogenerators

A neural network (NN) based regulator for nonlinear, multivariable turbogenerator control is presented. A hierarchical architecture of an NN is proposed for regulator design, consisting of two subnetworks which are used for input-output (I-O) mapping and control, respectively, based on the back-propagation (BP) algorithm. The regulator has the flexibility for accepting more sensory information to cater to multi-input, multioutput systems. Its operation does not require a reference model or inverse system model and it can produce more acceptable control signals than are obtained by using sign of plant errors during training I-O mapping of turbogenerator systems using NNs has been investigated and the regulator has been implemented on a complex turbogenerator system model. Simulation results show satisfactory control performance and illustrate the potential of the NN regulator in comparison with an existing adaptive controller.