Information recall using relative spike timing in a spiking neural network

We present a neural network that is capable of completing and correcting a spiking pattern given only a partial, noisy version. It operates in continuous time and represents information using the relative timing of individual spikes. The network is capable of correcting and recalling multiple patterns simultaneously. We analyze the network's performance in terms of information recall. We explore two measures of the capacity of the network: one that values the accurate recall of individual spike times and another that values only the presence or absence of complete patterns. Both measures of information are found to scale linearly in both the number of neurons and the period of the patterns, suggesting these are natural measures of network information. We show a smooth transition from encodings that provide precise spike times to flexible encodings that can encode many scenes. This makes it plausible that many diverse tasks could be learned with such an encoding.     

A novel and efficient classifier using spiking neural network

The Journal of Supercomputing - Classification plays a crucial role in big data, especially in e-commerce operations. Deep learning (DL) research has become a new means to provide a better solution...     

Design of ensemble neural network using entropy theory

Ensemble neural networks (ENNs) are commonly used neural networks in many engineering applications due to their better generalization properties. An ENN usually includes several component networks in its structure, and each component network commonly uses a single feed-forward network trained with the back-propagation learning rule. As the neural network architecture has a significant influence on its generalization ability, it is crucial to develop a proper algorithm to determine the ENN architecture. In this paper, an ENN, which combines the component networks using the entropy theory, is proposed. The entropy-based ENN searches the best structure of each component network first, and employs entropy as an automating design tool to determine the best combining weights. Two analytical functions - the peak function and the Friedman function are used to assess the accuracy of the proposed ensemble approach. Then, the entropy-based ENN is applied to the modeling of peak particle velocity (PPV) damage criterion for rock mass. These computational experiments have verified that the proposed entropy-based ENN outperforms the simple averaging ENN and the single NN.     

Stochastic dynamics of a finite-size spiking neural network

We present a simple Markov model of spiking neural dynamics that can be analytically solved to characterize the stochastic dynamics of a finite-size spiking neural network. We give closed-form estimates for the equilibrium distribution, mean rate, variance, and autocorrelation function of the network activity. The model is applicable to any network where the probability of firing of a neuron in the network depends on only the number of neurons that fired in a previous temporal epoch. Networks with statistically homogeneous connectivity and membrane and synaptic time constants that are not excessively long could satisfy these conditions. Our model completely accounts for the size of the network and correlations in the firing activity. It also allows us to examine how the network dynamics can deviate from mean field theory. We show that the model and solutions are applicable to spiking neural networks in biophysically plausible parameter regimes.     

跨脉冲传播的深度脉冲神经网络训练方法

基于反向传播的脉冲神经网络（SNNs）的训练方法仍面临着诸多问题与挑战,包括脉冲发放过程不可微分、脉冲神经元具有复杂的时空动力过程等。此外,SNNs反向传播训练方法往往没有考虑误差信号在相邻脉冲间的关系,大大降低了网络模型的准确性。为此,提出一种跨脉冲误差传播的深度脉冲神经网络训练方法（cross-spike error backpropagation,CSBP）,将神经元的误差反向传播分成脉冲发放时间随突触后膜电位变化关系和相邻脉冲发放时刻点间的依赖关系两种依赖关系。其中,通过前者解决了脉冲不可微分的问题,通过后者明确了脉冲间的依赖关系,使得误差信号能跨脉冲传播,提升了生物合理性。此外,并对早期脉冲残差网络架构存在的模型表示能力不足问题进行研究,通过修改脉冲残余块的结构顺序,进一步提高了网络性能。实验结果表明,所提方法比基于脉冲时间的最优训练算法有着明显的提升,相同架构下,在CIFAR10数据集上提升2.98%,在DVS-CIFAR10数据集上提升2.26%。     

Finding iterative roots with a spiking neural network

In recent years, both multilayer perceptrons and networks of spiking neurons have been used in applications ranging from detailed models of specific cortical areas to image processing. A more challenging application is to find solutions to functional equations in order to gain insights to underlying phenomena. Finding the roots of real valued monotonically increasing function mappings is the solution to a particular class of functional equation. Furthermore, spiking neural network approaches in solving problems described by functional equations, may be an useful tool to provide important insights to how different regions of the brain may co-ordinate signaling within and between modalities, thus providing a possible basis to construct a theory of brain function. In this letter, we present for the first time a spiking neural network architecture based on integrate-and-fire units and delays, that is capable of calculating the functional or iterative root of nonlinear functions, by solving a particular class of functional equation.     

Condition monitoring of the cutting process using a self-organizing spiking neural network map

This paper presents a new approach to sensor based condition monitoring using a self-organizing spiking neuron network map. Experimental evidence suggests that biological neural networks, which communicate through spikes, use the timing of these spikes to encode and compute information in a more efficient way. The paper introduces the basis of a simplified version of the Self-Organizing neural architecture based on Spiking Neurons. The fundamental steps for the development of this computational model are presented as well as some experimental evidence of its performance. It is shown that this computational architecture has a greater potential to unveil embedded information in tool wear monitoring data sets and that faster learning occurs if compared to traditional sigmoidal neural networks.     

基于自适应编码的脉冲神经网络

脉冲神经网络(SNN)采用脉冲序列表征和传递信息,与传统人工神经网络相比更具有生物可解释性,但典型SNN的特征提取能力受到其结构限制,对于图像数据等多分类任务的识别准确率不高,不能与卷积神经网络相媲美。为此提出一种新型的自适应编码脉冲神经网络(SCSNN),将CNN的特征提取能力与SNN的生物可解释性结合起来,采用生物神经元动态脉冲触发特性构建网络结构,并设计了一种新的替代梯度反向传播方法直接训练网络参数。所提出的SCSNN分别在MNIST和Fashion-MNIST数据集进行验证,取得较好的识别结果,在MNIST数据集上准确率达到了99.62%,在Fashion-MNIST数据集上准确率达到了93.52%,验证了其有效性。     

Quantized STDP-based online-learning spiking neural network

Neural Computing and Applications - In this work, we report a spike-timing-dependent plasticity (STDP)-based weight-quantized/binarized online-learning spiking neural network (SNN). The SNN uses...     

基于小波变换和神经网络集成的笔迹鉴别方法*

在笔迹鉴别中为了便于获取特征字符的细微特征,基于线性矩和小波变换提出了提取特征字符纹理特征的方法.小波变换能有效地提取字符的结构特征,而矩能够很好地对其进行描述.在该方法中,一幅特征字图像可以用一个含有52个元素的特征矢量表示,然后通过训练多个神经网络,并应用神经网络集成的方法将其结果合成,对特征空间进行正确分类.分别在特征字和候选人数变化的情况下进行实验,实验结果显示识别准确率较同类算法平均提高百分之五.     

Online tool wear prediction in drilling operations using selective artificial neural network ensemble model

Online tool wear prediction plays a key role in industry automation for higher productivity and product quality. In recent past, several artificial neural network (ANN) models using multiple sensor signals as inputs for prediction as well as classification of tool wear have been proposed. However, a single ANN used in these models is often tries, which could limit their wide applications due to the complicated procedure of constructing a single ANN model. This study proposed a selective ANN ensemble approach DPSOEN, where several selected component ANNs are jointly used to online predict flank wear in drilling operation. DPSOEN provides more simple training and better generalization performance than using single ANN and hence is easier to be used by operators who often are not good at ANN techniques. Two benchmark cases were used to evaluate the performance of DPSOEN in predicting flank wear. It shows improved generalization performance that outperforms those of single ANN and Ensemble ALL approach. The investigation proposed a heuristic approach for applying the DPSOEN-based model as an effective and useful tool to predict tool wear online with potential applications for tool condition monitoring in general. Analysis from this study provides guidelines in developing ANN ensemble-based tool wear prediction systems.     

Sparsification from dilute connectivity in a neural network model of memory.

Several hypotheses concerning implementations of associative memory in the brain rely on analyses of the capabilities of simple network models. However, the low connectivity of cerebral networks imposes constraints which sometimes do not arise clearly from such analyses. We investigate an aspect of a simple, dilute network's operation that is sometimes overlooked, namely the setting of activation thresholds. An examination of several criteria for optimal threshold assignment affords several new insights. It becomes apparent that the network's capacity (which is simply derived) is insufficient to characterize the quality of its performance. We derive the degree of 'sparsification' or decrease in firing probability that arises from dilution, and also the consequent losses in representational ability, and propose that they should also be taken into account. To evaluate the model's performance and suitability, we argue that one should explicitly consider the trade-off that exists between storage of patterns and preservation of information, and its consequent constraints.     

Adaptive driver model using a neural network

An adaptive driver model for longitudinal movements of a vehicle has been developed. It incorporates a conventional feedback brake controller, and both fixed and adaptive neural network controllers to produce the throttle demand. It has been interfaced with a vehicle model in a Simulink environment, and simulation studies indicate a high level of performance. Implementation of the adaptive driver model within a real-time environment has also been realized successfully. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

融合LSTM结构的脉冲神经网络模型研究与应用

SNN是更具生物可解释性的新型网络模型。针对传统SNN模型表征能力有限,难以应用于实际任务的问题,对SNN处理脑电识别任务进行了研究,提出具有长短期记忆结构的SNN模型。首先采用改进的BSA编码算法处理脑电信号;然后构建具有自适应阈值的脉冲神经元模型;在此基础上,基于PyTorch框架建立结合LSTM结构的SNN模型;最后使用替代梯度的方法克服了脉冲序列不可微分的问题,在保留神经元动态特性的同时基于反向传播方法直接训练SNN。实验结果表明,改进的BSA更具灵活性和可靠性,同时,融合LSTM结构的SNN模型提高了网络的表征能力,在脑电识别任务中取得了与传统深度学习模型可竞争的精度。     

Vectorized algorithms for spiking neural network simulation

High-level languages (Matlab, Python) are popular in neuroscience because they are flexible and accelerate development. However, for simulating spiking neural networks, the cost of interpretation is a bottleneck. We describe a set of algorithms to simulate large spiking neural networks efficiently with high-level languages using vector-based operations. These algorithms constitute the core of Brian, a spiking neural network simulator written in the Python language. Vectorized simulation makes it possible to combine the flexibility of high-level languages with the computational efficiency usually associated with compiled languages.     

Hardware spiking neural network prototyping and application

EMBRACE has been proposed as a scalable, reconfigurable, mixed signal, embedded hardware Spiking Neural Network (SNN) device. EMBRACE, which is yet to be realised, targets the issues of area, power and scalability through the use of a low area, low power analogue neuron/synapse cell, and a digital packet-based Network on Chip (NoC) communication architecture. The paper describes the implementation and testing of EMBRACE-FPGA, an FPGA-based hardware SNN prototype. The operation of the NoC inter-neuron communication approach and its ability to support large scale, reconfigurable, highly interconnected SNNs is illustrated. The paper describes an integrated training and configuration platform and an on-chip fitness function, which supports GA-based evolution of SNN parameters. The practicalities of using the SNN development platform and SNN configuration toolset are described. The paper considers the impact of latency jitter noise introduced by the NoC router and the EMBRACE-FPGA processor-based neuron/synapse model on SNN accuracy and evolution time. Benchmark SNN applications are described and results demonstrate the evolution of high quality and robust solutions in the presence of noise. The reconfigurable EMBRACE architecture enables future investigation of adaptive hardware applications and self repair in evolvable hardware.     

基于互信息的生物神经网络功能性连接辨识

生物神经网络（ BNN）功能性连接的辨识方法被广泛地应用于使用BNN的多通道时间序列数据构建网络连接结构,帮助加深对BNN结构和功能间关系的认识和理解。首先,建立基于积分点火（ IF）机制的BNN模型,获得多通道神经元脉冲序列;然后,运用互信息（ MI）方法计算出各神经元间的MI值,超过一定阈值的MI表明两个神经元间存在相互连接关系。仿真结果表明：基于MI的网络辨识方法计算开销较小,对BNN功能性连接结构具有较高的辨识度。     

Energy proportional streaming spiking neural network in a reconfigurable system

This paper presents a high-performance architecture for spiking neural networks that optimizes data precision and streaming of configuration data stored in main memory. The neural network is based on the Izhikevich model and mapped to a CPU-FPGA hybrid device using a high-level synthesis flow. The active area of the network is configurable and this feature is used to create an energy proportional system. Voltage and frequency scaling are applied to the processing hardware and memory system to deliver enough processing and memory bandwidth to maintain real-time performance at minimum power and energy levels. The experiments show that the application of voltage and frequency scaling to DDR memory and programmable logic can reduce its energy requirements by up to 77% and 76% respectively. The performance evaluation show that the solution is superior to competing high-performance hardware, while voltage and frequency scaling reduces overall energy requirements to less than 2% of a software-only implementation at the same level of performance.     

一种卷积神经网络集成的多样性度量方法

分类器模型之间的多样性是分类器集成的一个重要性能指标。目前大多数多样性度量方法都是基于基分类器模型的0/1输出结果（即Oracle 输出）进行计算,针对卷积神经网络的概率向量输出结果,仍需要将其转化为Oracle输出方式进行度量,这种方式未能充分利用卷积神经网络输出的概率向量所包含的丰富信息。针对此问题,利用多分类卷积神经网络模型的输出特性,提出了一种基于卷积神经网络的概率向量输出方式的集成多样性度量方法,建立多个不同结构的卷积神经网络基模型并在CIFAR-10和CIFAR-100数据集上进行实验。实验结果表明,与双错度量、不一致性度量和Q统计多样性度量方法相比,所提出的方法能够更好地体现模型之间的多样性,为模型选择集成提供更好的指导。     

神经网络集成的城市道路状态判别模型研究

针对城市道路交通状态影响因素多、判别难的特点,在分析K-均值聚类算法和概率神经网络（PNN）的基础上,利用多源检测信息的互补性,提出一种基于快速全局聚类分析的概率神经网络集成模型,通过聚类提高集成网络间的差异度,同时利用主成分分析（PCA）优化概率神经网络结构,仿真实验表明该模型与传统的集成方法Bagging相比,能够利用更简单的网络结构,快速有效地识别出城市道路交通状态,为交通预警和诱导策略的制定提供数据依据。