Supervised Learning of Logical Operations in Layered Spiking Neural Networks with Spike Train Encoding

Few algorithms for supervised training of spiking neural networks exist that can deal with patterns of multiple spikes, and their computational properties are largely unexplored. We demonstrate in a set of simulations that the ReSuMe learning algorithm can successfully be applied to layered neural networks. Input and output patterns are encoded as spike trains of multiple precisely timed spikes, and the network learns to transform the input trains into target output trains. This is done by combining the ReSuMe learning algorithm with multiplicative scaling of the connections of downstream neurons. We show in particular that layered networks with one hidden layer can learn the basic logical operations, including Exclusive-Or, while networks without hidden layer cannot, mirroring an analogous result for layered networks of rate neurons. While supervised learning in spiking neural networks is not yet fit for technical purposes, exploring computational properties of spiking neural networks advances our understanding of how computations can be done with spike trains.     

Analysis of a variable speed vapor compression system using artificial neural networks

An artificial neural network (ANN) is a mathematical model that is inspired by the operation of biological neural networks. However, this is typically considered a computational model. An ANN can easily adapt to multiple situations and extract information that is apparently hidden in a system.An ANN can be used in three basic configurations: mapping, auto-association and classification. An ANN in a mapping configuration can be used to model a two port system with inputs and outputs. Therefore, a vapor compression system can be modeled using an ANN in a mapping configuration. That is, some parameters from the compression system can be used for input while other parameters can be used as output. The simulation experiments include the design, training and validation of a set of ANNs to find the best architecture while minimizing over-fitting.This paper presents a new method to model a variable speed vapor compression system. This method accurately estimates the number of neurons in the hidden layer of an ANN. The analysis and the experimental results provide new insights to understand the dependency between the input and output parameters in a vapor compression system, concluding that the model can predict the energetic performance of a variable speed vapor compression system. Additionally, the simulation results indicate that an ANN can extract, from the data sets, information that is implicit in the configuration of the vapor compression system.     

Detection of Seizure Activity in EEG by an Artificial Neural Network: A Preliminary Study

Neural networks, inspired by the organizational principles of the human brain, have recently been used in various fields of application such as pattern recognition, identification, classification, speech, vision, signal processing, and control systems. In this study, a two-layered neural network has been trained for the recognition of temporal patterns of the electroencephalogram (EEG). This network is called a Learning Vector Quantization (LVQ) neural network since it learns the characteristics of the signal presented to it as a vector. The first layer is a competitive layer which learns to classify the input vectors. The second, linear, layer transforms the output of the competitive layer to target classes defined by the user. We have tested and evaluated the LVQ network. The network successfully detects epileptiform discharges (EDs) when trained using EEG records scored by a neurologist. Epochs of EEG containing EDs from one subject have been used for training the network, and EEGs of other subjects have been used for testing the network. The results demonstrate that the LVQ detector can generalize the learning to previously “unseen” records of subjects. This study shows that the LVQ network offers a practical solution for ED detection which is easily adjusted to an individual neurologist's style and is as sensitive and specific as an expert visual analysis.     

Development of a hybrid PSO–ANN model for estimating glucose and xylose yields for microwave-assisted pretreatment and the enzymatic hydrolysis of lignocellulosic biomass

In this paper, two artificial intelligent systems, the artificial neural network (ANN) and particle swarm optimization (PSO), were combined to form a hybrid PSO–ANN model that was used to improve estimates of glucose and xylose yields from the microwave–acid pretreatment and enzymatic hydrolysis of lignocellulosic biomass based on pretreatment parameters. ANN is a powerful tool capable of determining the relationship between the desired input and output data while PSO was used as a robust population-based search algorithm to optimize the performance of the ANN model. Specifically, it was used to determine the optimum number of neurons in the hidden layer and the best value of the learning rate of the ANN model. The optimization method includes minimizing the fitness function mean absolute error that was found to be 0.0176. The PSO algorithm suggested an optimum number of neurons in the hidden layer as 15 and a learning rate of 0.761 these consequently used to construct the ANN model. After constructing the hybrid PSO–ANN model, the performance of the intelligent system was examined by determining the regression coefficient (R ²) for estimating the experimental values of glucose and xylose and compared to the results from a response surface methodology (RSM) model. The results of R ² of the hybrid PSO–ANN model for glucose and xylose were 0.9939 and 0.9479, respectively, while the RSM model results for the same sugars were 0.8901 and 0.8439. This analysis reveals that the hybrid PSO–ANN model offers a higher degree of accuracy in comparison with the more commonly used RSM model.

     

基于多分辨分析的脑电癫痫波自动检测*

分析了小波多分辨分析特征提取的特点,提出了八通道脑电信号癫痫波自动检测的方法。每个通道的信号利用小波变换进行五层分解,以提取小波变换各子带的小波系数和信号偏差组成特征值计算自适应阈值,并将其应用到关键子带,提取出信号中的癫痫波。研究的重点是对脑电信号进行分解选择合适的小波;确定适当的分解层次以及自适应阈值的计算。实验结果表明,方法能够为癫痫脑电的特征提取提供快速而有效的手段。     

Noise removal in electroencephalogram signals using an artificial neural network based on the simultaneous perturbation method

Electroencephalogram (EEG) recordings often experience interference by different kinds of noise, including white, muscle and baseline, severely limiting its utility. Artificial neural networks (ANNs) are effective and powerful tools for removing interference from EEGs. Several methods have been developed, but ANNs appear to be the most effective for reducing muscle and baseline contamination, especially when the contamination is greater in amplitude than the brain signal. An ANN as a filter for EEG recordings is proposed in this paper, developing a novel framework for investigating and comparing the relative performance of an ANN incorporating real EEG recordings. This method is based on a growing ANN that optimized the number of nodes in the hidden layer and the coefficient matrices, which are optimized by the simultaneous perturbation method. The ANN improves the results obtained with the conventional EEG filtering techniques: wavelet, singular value decomposition, principal component analysis, adaptive filtering and independent components analysis. The system has been evaluated within a wide range of EEG signals. The present study introduces a new method of reducing all EEG interference signals in one step with low EEG distortion and high noise reduction.     

Artificial neural networks solution to display residual hoop stress field encircling a split-sleeve cold expanded aircraft fastener hole

Cold expansion of holes is a technique, generating intricate three-dimensional residual stresses around fastener holes essentially vital for airplane fatigue resistance. In this work, attention was given to Artificial Neural Networks (ANN) modeling to build up and train simulations of stress topography surrounding a 4% expanded hole. For this, experimental data of recently abridged step drilling-Fourier method was employed. At input layer of ANN; information available for steps through thickness and radial directions, angular variation around the hole, and at output layer, residual hoop stresses were exercised to train and test multilayered, hierarchically connected and directed networks with varying number of hidden layers. It was shown that Levenberg–Marquardt (LM) model with 9 neurons in hidden layer yielded the best of the results, as error percentages were remarkably small both in training and testing sequences. Several results of step drilling-Fourier solution (ATÖzdemir method), diffraction methods and current ANN predictions were overlaid and similarities in residual stress distributions perceived to valid only at regions where strain gradient was not changing precipitously. Nevertheless, best fit to strain data at confusing zones was achieved after ANN modeling.     

Neural networks with local receptive fields and superlinear VC dimension

Local receptive field neurons comprise such well-known and widely used unit types as radial basis function (RBF) neurons and neurons with center-surround receptive field. We study the Vapnik-Chervonenkis (VC) dimension of feedforward neural networks with one hidden layer of these units. For several variants of local receptive field neurons, we show that the VC dimension of these networks is superlinear. In particular, we establish the bound Omega(W log k) for any reasonably sized network with W parameters and k hidden nodes. This bound is shown to hold for discrete center-surround receptive field neurons, which are physiologically relevant models of cells in the mammalian visual system, for neurons computing a difference of gaussians, which are popular in computational vision, and for standard RBF neurons, a major alternative to sigmoidal neurons in artificial neural networks. The result for RBF neural networks is of particular interest since it answers a question that has been open for several years. The results also give rise to lower bounds for networks with fixed input dimension. Regarding constants, all bounds are larger than those known thus far for similar architectures with sigmoidal neurons. The superlinear lower bounds contrast with linear upper bounds for single local receptive field neurons also derived here.     

A Novel Pruning Algorithm for Optimizing Feedforward Neural Network of Classification Problems

Optimizing the structure of neural networks is an essential step for the discovery of knowledge from data. This paper deals with a new approach which determines the insignificant input and hidden neurons to detect the optimum structure of a feedforward neural network. The proposed pruning algorithm, called as neural network pruning by significance (N2PS), is based on a new significant measure which is calculated by the Sigmoidal activation value of the node and all the weights of its outgoing connections. It considers all the nodes with significance value below the threshold as insignificant and eliminates them. The advantages of this approach are illustrated by implementing it on six different real datasets namely iris, breast-cancer, hepatitis, diabetes, ionosphere and wave. The results show that the proposed algorithm is quite efficient in pruning the significant number of neurons on the neural network models without sacrificing the networks performance.     

Comparative analysis on hidden neurons estimation in multi layer perceptron neural networks for wind speed forecasting

In this paper methodologies are proposed to estimate the number of hidden neurons that are to be placed numbers in the hidden layer of artificial neural networks (ANN) and certain new criteria are evolved for fixing this hidden neuron in multilayer perceptron neural networks. On the computation of the number of hidden neurons, the developed neural network model is applied for wind speed forecasting application. There is a possibility of over fitting or under fitting occurrence due to the random selection of hidden neurons in ANN model and this is addressed in this paper. Contribution is done in developing various 151 different criteria and the evolved criteria are tested for their validity employing various statistical error means. Simulation results prove that the proposed methodology minimized the computational error and enhanced the prediction accuracy. Convergence theorem is employed over the developed criterion to validate its applicability for fixing the number of hidden neurons. To evaluate the effectiveness of the proposed approach simulations were carried out on collected real-time wind data. Simulated results confirm that with minimum errors the presented approach can be utilized for wind speed forecasting. Comparative analysis has been performed for the estimation of the number of hidden neurons in multilayer perceptron neural networks. The presented approach is compact, enhances the accuracy rate with reduced error and faster convergence.     

An efficient hybrid solution for pronouncing Farsi text

This paper presents a hybrid Artificial Neural Network (ANN)/rule-based phoneme synthesizer for Farsi text. A new variable called Vowel State (VS) is also introduced which efficiently decreases the size of computations. The proposed system uses VS along with the written form of the words to determine their pronunciation. Some rules are also applied to the system to decrease network interconnections. These rules are performed in the Normalization Unit and the Phoneme Extraction Unit before and after the ANN. A multilayer perceptron with 48, 90 and 3 neurons in the input layer, hidden layer and output layer respectively is chosen to determine the Vowel State of each letter. Each letter in a word enters the ANN in the heart of a window in order to recognize the VS of the letter. An asymmetric 8-letter windowing is suggested; 3 letters before the letter along with 4 letters after it enter the ANN as the input. For training and testing the system, 2359 words are extracted from some online available resources. The results of test show that using 65% of the corpus as training set and testing over the entire corpus gives the result of 84% correct answers for the ANN, and 90% correct answer for the entire system considering additional rules. The performance index of the ANN increases to 88, 92, 98 and 99 percent when the percentage of the training set increases to 75, 85, 95 and 100, respectively. The system also shows the performance of 93% correct answers on an independent word database. The proposed hybrid ANN/rule-based method imposes small size of calculation in comparison with similar systems. In this system, the output of ANN is a 5-value variable Vowel State instead of numerous phonemes. This fact decreases the neurons in the output layer of ANN considerably. Besides, adding some rules before and after the network helps to increase performance. The overall system shows a decrement of 70% in the size of computations comparing with other reported approaches.     

Classification of EEG signals using neural network and logistic regression

Epileptic seizures are manifestations of epilepsy. Careful analyses of the electroencephalograph (EEG) records can provide valuable insight and improved understanding of the mechanisms causing epileptic disorders. The detection of epileptiform discharges in the EEG is an important component in the diagnosis of epilepsy. As EEG signals are non-stationary, the conventional method of frequency analysis is not highly successful in diagnostic classification. This paper deals with a novel method of analysis of EEG signals using wavelet transform and classification using artificial neural network (ANN) and logistic regression (LR). Wavelet transform is particularly effective for representing various aspects of non-stationary signals such as trends, discontinuities and repeated patterns where other signal processing approaches fail or are not as effective. Through wavelet decomposition of the EEG records, transient features are accurately captured and localized in both time and frequency context. In epileptic seizure classification we used lifting-based discrete wavelet transform (LBDWT) as a preprocessing method to increase the computational speed. The proposed algorithm reduces the computational load of those algorithms that were based on classical wavelet transform (CWT). In this study, we introduce two fundamentally different approaches for designing classification models (classifiers) the traditional statistical method based on logistic regression and the emerging computationally powerful techniques based on ANN. Logistic regression as well as multilayer perceptron neural network (MLPNN) based classifiers were developed and compared in relation to their accuracy in classification of EEG signals. In these methods we used LBDWT coefficients of EEG signals as an input to classification system with two discrete outputs: epileptic seizure or non-epileptic seizure. By identifying features in the signal we want to provide an automatic system that will support a physician in the diagnosing process. By applying LBDWT in connection with MLPNN, we obtained novel and reliable classifier architecture. The comparisons between the developed classifiers were primarily based on analysis of the receiver operating characteristic (ROC) curves as well as a number of scalar performance measures pertaining to the classification. The MLPNN based classifier outperformed the LR based counterpart. Within the same group, the MLPNN based classifier was more accurate than the LR based classifier.     

EEG signal classification using wavelet feature extraction and a mixture of expert model

Mixture of experts (ME) is modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME network structure for detection of epileptic seizure. The detection of epileptiform discharges in the EEG is an important component in the diagnosis of epilepsy. EEG signals were decomposed into the frequency sub-bands using discrete wavelet transform (DWT). Then these sub-band frequencies were used as an input to a ME network with two discrete outputs: normal and epileptic. In order to improve accuracy, the outputs of expert networks were combined according to a set of local weights called the “gating function”. The invariant transformations of the ME probability density functions include the permutations of the expert labels and the translations of the parameters in the gating functions. The performance of the proposed model was evaluated in terms of classification accuracies and the results confirmed that the proposed ME network structure has some potential in detecting epileptic seizures. The ME network structure achieved accuracy rates which were higher than that of the stand-alone neural network model.     

多层感知器网络内部判决模式的研究

人工神经网络（ANN）内部行为的研究,无论是对生物神经系统内部工作机理、ANN理论,还是对ANN应用都有重要意义。本文在作者原有工作基础上加以发展,针对多层感知器网络应用于模式识别、分类、 数逼近与参数估计的内部行为,作出了明确解释;以单陷层结构为典型,定义了隐层神经元输出为网络输出量的“（正、负）内部分量”,陷层权重分布为网络求解问题的“内部判决模式”;并给出了应用这一理论分析的实例。     

In search of an optimization technique for Artificial Neural Network to classify abnormal heart sounds

Artificial Neural Network (ANN) finds use in classification of heart sounds for its discriminative training ability and easy implementation. The selection of number of nodes for an ANN remains an important issue as an overparameterized ANN gets trained along with the redundant information that results in poor validation. Also a larger network means more computational expense, resulting more hardware and time related cost. Therefore, a compact and optimum design of neural network is needed towards real-time detection of pathological patterns, if any from heart sound signals. This work attempts to (i) design a compact form of output layer with less number of nodes than output classes, (ii) select a set of input features that are effective for identification of heart sound signals using Singular Value Decomposition (SVD), QR factorization with column pivoting (QRcp) and Fisher's F-ratio, (iii) make certain optimum selection of nodes in the hidden layer for a more effective ANN structure using SVD and (iv) select and prune weights based on the concept of local relative sensitivity index (LRSI) for empirically chosen overparameterized ANN structure for phonocardiogram (PCG) classification. It is observed that the proposed techniques perform better in terms of reduction of model residues and time complexity for classifying 12 different pathological cases and normal heart sound.     

基于序列输入的神经网络模型算法及应用

为提高神经网络的逼近能力,提出一种基于序列输入的神经网络模型及算法。模型隐层为序列神经元,输出层为普通神经元。输入为多维离散序列,输出为普通实值向量。先将各维离散输入序列值按序逐点加权映射,再将这些映射结果加权聚合之后映射为隐层序列神经元的输出,最后计算网络输出。采用Levenberg-Marquardt算法设计了该模型学习算法。仿真结果表明,当输入节点和序列长度比较接近时,模型的逼近能力明显优于普通神经网络。     

A non-symbolic implementation of abdominal pain estimation in childhood

The abdominal pain is a very common disease in childhood, which lurks complications. Pediatric surgeons have to estimate at least 15 clinical and laboratory factors in order to make a diagnosis and decide about performing a surgical operation of the abdomen. Artificial Neural Networks (ANNs) are particular implementations of Artificial Intelligence (AI) systems and they are used in a wide area of application fields. This study examines the implementation of ANN architectures, using Multi-Layer Perceptron (MLP) neural networks and Probabilistic Neural Networks (PNN) architectures, in order to specify the appropriate ANN structure for abdominal pain estimation in childhood. The architecture with the best performance is a fully interconnected MLP neural network with an input layer of 15 nodes, one hidden layer of 5 neurons and an output layer, with error back-propagation algorithm being used as the learning scheme. In the output layer, the estimation of appendicitis’ stage is reached automatically. The proposed ANN achieved a percentage of 88.5% of correct classification on testing set cases. Further analysis of obtained results, exhibited the ability of ANN for distinguishing the necessity of a case for operative treatment of abdominal pain based on diagnostic features, attaining a percentage of 100% of successful prognosis over the cases of testing set. The aim of proposed MLP neural network is to assist surgeons in appendicitis prediction, avoiding an unnecessary operative treatment.     

Sequence Input-Based Quantum-Inspired Neural Networks with Applications

To enhance the approximation and generalization ability of artificial neural networks (ANNs) by employing the principle of quantum rotation gate and controlled-not gate, a quantum-inspired neuron with sequence input is proposed. In the proposed model, the discrete sequence input is represented by the qubits, which, as the control qubits of the controlled-not gate after being rotated by the quantum rotation gates, control the target qubit for reverse. The model output is described by the probability amplitude of state \(|1\rangle \) in the target qubit. Then a quantum-inspired neural networks (QINN) is designed by employing the quantum-inspired neurons to the hidden layer and the common neurons to the output layer. The algorithm of QINN is derived by employing the Levenberg–Marquardt algorithm. Simulation results of some benchmark problems show that, under a certain condition, the QINN is obviously superior to the classical ANN.     

基于深度神经网络的语音驱动发音器官的运动合成

实现一种基于深度神经网络的语音驱动发音器官运动合成的方法,并应用于语音驱动虚拟说话人动画合成. 通过深度神经网络(Deep neural networks, DNN)学习声学特征与发音器官位置信息之间的映射关系,系统根据输入的语音数据估计发音器官的运动轨迹,并将其体现在一个三维虚拟人上面. 首先,在一系列参数下对比人工神经网络(Artificial neural network, ANN)和DNN的实验结果,得到最优网络; 其次,设置不同上下文声学特征长度并调整隐层单元数,获取最佳长度; 最后,选取最优网络结构,由DNN 输出的发音器官运动轨迹信息控制发音器官运动合成,实现虚拟人动画. 实验证明,本文所实现的动画合成方法高效逼真.     

Prediction of the effect of varying cure conditions and w/c ratio on the compressive strength of concrete using artificial neural networks

The present study aims at developing an artificial neural network (ANN) to predict the compressive strength of concrete. A data set containing a total of 72 concrete samples was used in the study. The following constituted the concrete mixture parameters: two distinct w/c ratios (0.63 and 0.70), three different types of cements and three different cure conditions. Measurement of compressive strengths was performed at 3, 7, 28 and 90 days. Two different ANN models were developed, one with 4 input and 1 output layers, 9 neurons and 1 hidden layer, and the other with 5, 6 neurons, 2 hidden layers. For the training of the developed models, 60 experimental data sets obtained prior to the process were used. The 12 experimental data not used in the training stage were utilized to test ANN models. The researchers have reached the conclusion that ANN provides a good alternative to the existing compressive strength prediction methods, where different cements, ages and cure conditions were used as input parameters.