Polychronization: computation with spikes

We present a minimal spiking network that can polychronize, that is, exhibit reproducible time-locked but not synchronous firing patterns with millisecond precision, as in synfire braids. The network consists of cortical spiking neurons with axonal conduction delays and spike-timing-dependent plasticity (STDP); a ready-to-use MATLAB code is included. It exhibits sleeplike oscillations, gamma (40 Hz) rhythms, conversion of firing rates to spike timings, and other interesting regimes. Due to the interplay between the delays and STDP, the spiking neurons spontaneously self-organize into groups and generate patterns of stereotypical polychronous activity. To our surprise, the number of coexisting polychronous groups far exceeds the number of neurons in the network, resulting in an unprecedented memory capacity of the system. We speculate on the significance of polychrony to the theory of neuronal group selection (TNGS, neural Darwinism), cognitive neural computations, binding and gamma rhythm, mechanisms of attention, and consciousness as "attention to memories."     

基于卷积计算的多层脉冲神经网络的监督学习

针对脉冲神经元基于精确定时的多脉冲编码信息的特点,提出了一种基于卷积计算的多层脉冲神经网络监督学习的新算法。该算法应用核函数的卷积计算将离散的脉冲序列转换为连续函数,在多层前馈脉冲神经网络结构中,使用梯度下降的方法得到基于核函数卷积表示的学习规则,并用来调整神经元连接的突触权值。在实验部分,首先验证了该算法学习脉冲序列的效果,然后应用该算法对Iris数据集进行分类。结果显示,该算法能够实现脉冲序列复杂时空模式的学习,对非线性模式分类问题具有较高的分类正确率。     

面向图像识别的多层脉冲神经网络学习算法综述

相较于第1代和第2代神经网络,第3代神经网络的脉冲神经网络是一种更加接近于生物神经网络的模型,因此更具有生物可解释性和低功耗性。基于脉冲神经元模型,脉冲神经网络可以通过脉冲信号的形式模拟生物信号在神经网络中的传播,通过脉冲神经元的膜电位变化来发放脉冲序列,脉冲序列通过时空联合表达不仅传递了空间信息还传递了时间信息。当前面向模式识别任务的脉冲神经网络模型性能还不及深度学习,其中一个重要原因在于脉冲神经网络的学习方法不成熟,深度学习中神经网络的人工神经元是基于实数形式的输出,这使得其可以使用全局性的反向传播算法对深度神经网络的参数进行训练,脉冲序列是二值性的离散输出,这直接导致对脉冲神经网络的训练存在一定困难,如何对脉冲神经网络进行高效训练是一个具有挑战的研究问题。本文首先总结了脉冲神经网络研究领域中的相关学习算法,然后对其中主要的方法:直接监督学习、无监督学习的算法以及ANN2SNN的转换算法进行分析介绍,并对其中代表性的工作进行对比分析,最后基于对当前主流方法的总结,对未来更高效、更仿生的脉冲神经网络参数学习方法进行展望。     

Spiking神经元输入脉冲扰动敏感性研究

脉冲神经网络是一种基于生物的网络模型,它的输入输出为具有时间特性的脉冲序列,其运行机制相比其他传统人工神经网络更加接近于生物神经网络。神经元之间通过脉冲序列传递信息,这些信息通过脉冲的激发时间编码能够更有效地发挥网络的学习性能。脉冲神经元的时间特性导致了其工作机制较为复杂,而spiking神经元的敏感性反映了当神经元输入发生扰动时输出的spike的变化情况,可以作为研究神经元内部工作机制的工具。不同于传统的神经网络,spiking神经元敏感性定义为输出脉冲的变化时刻个数与运行时间长度的比值,能直接反映出输入扰动对输出的影响程度。通过对不同形式的输入扰动敏感性的分析,可以看出spiking神经元的敏感性较为复杂,当全体突触发生扰动时,神经元为定值,而当部分突触发生扰动时,不同突触的扰动会导致不同大小的神经元敏感性。     

基于脉冲神经网络的红外目标提取

模拟生物信息处理机制,设计了一种用于红外目标提取的脉冲神经网络(SNN)。首先,利用输入层脉冲神经元将激励图像转化为脉冲序列;其次,采用中间层脉冲神经元输出脉冲的密度编码红外图像目标的轮廓像素和非目标轮廓像素;最后,根据输出层神经元输出脉冲的密度是否超过阈值提取红外目标。实验结果表明,设计的脉冲神经网络具有较好的红外目标提取性能,并且符合生物视觉信息处理机制。     

脉冲神经网络:模型、学习算法与应用

脉冲神经网络是目前最具有生物解释性的人工神经网络,是类脑智能领域的核心组成部分.首先介绍各类常用的脉冲神经元模型以及前馈和循环型脉冲神经网络结构;然后介绍脉冲神经网络的时间编码方式,在此基础上,系统地介绍脉冲神经网络的学习算法,包括无监督学习和监督学习算法,其中监督学习算法按照梯度下降算法、结合STDP规则的算法和基于脉冲序列卷积核的算法3大类别分别展开详细介绍和总结;接着列举脉冲神经网络在控制领域、模式识别领域和类脑智能研究领域的应用,并在此基础上介绍各国脑计划中,脉冲神经网络与神经形态处理器相结合的案例;最后分析脉冲神经网络目前所存在的困难和挑战.     

Spiking neural P systems with neuron division and budding

Spiking neural P systems are a class of distributed and parallel computing models inspired by spiking neurons.In this work,the features of neuron division and neuron budding are introduced into the framework of spiking neural P systems,which are processes inspired by neural stem cell division. With neuron division and neuron budding,a spiking neural P system can generate exponential work space in polynomial time as the case for P systems with active membranes.In this way,spiking neural P systems can efficie...     

Improved Izhikevich neurons for spiking neural networks

Spiking neural networks constitute a modern neural network paradigm that overlaps machine learning and computational neurosciences. Spiking neural networks use neuron models that possess a great degree of biological realism. The most realistic model of the neuron is the one created by Alan Lloyd Hodgkin and Andrew Huxley. However, the Hodgkin–Huxley model, while accurate, is computationally very inefficient. Eugene Izhikevich created a simplified neuron model based on the Hodgkin–Huxley equations. This model has better computational efficiency than the original proposed by Hodgkin and Huxley, and yet it can successfully reproduce all known firing patterns. However, there are not many articles dealing with implementations of this model for a functional neural network. This study presents a spiking neural network architecture that utilizes improved Izhikevich neurons with the purpose of evaluating its speed and efficiency. Since the field of spiking neural networks has reinvigorated the interest in biological plausibility, biological realism was an additional goal. The network is tested on the correct classification of logic gates (including XOR) and on the iris dataset. Results and possible improvements are also discussed.     

Optoelectronic implementation of pulsed neurons and neural networks using bispin-devices

Optoelectronic spiking neuron that is based on bispin-device is described. The neuron has separate optical inputs for excitatory and inhibitory signals, which are represented with pulses of single polarity. Experimental data, which demonstrates similarity in form of output pulses and set of functions of the suggested neuron and a biological one is given. An example of hardware implementation of optoelectronic pulsed neural network (PNN) that is based on proposed neurons is described. Main elements of the neural network are a line of pulsed neurons and a connection array, part of which is made as a spatial light modulator (SLM) with memory. Usage of SLM allows modification of weights of connections in the learning process of the network. It is possible to create adaptive (capable of additional learning and relearning) optoelectronic PNNs with about 2000 neurons.     

基于遗传算法与可拓神经网络的故障诊断

可拓神经网络是基于可拓理论和神经网络而设计的一种新的方法,它即充分利用了可拓学定性描述和定量描述的优点,又考虑了神经网络并行结构的特点.它由输入和输出两层可拓神经元构成,在每个输入神经元和输出神经元之间有两个连接权值.然后利用遗传算法全局搜索能力,对建立的可拓神经网络的权值进行优化,在优化过程中利用可拓神经网络输出的正确次数与可拓神经网络输入的样本总数的比值作为适应度函数,染色体根据物元的节域进行实数编码,计算出的可拓距离的最大值对应的物元与样本一致时,输出正确次数累加一次.算法终止条件为误差值达到要求.最后利用该方法开发了励磁系统的故障诊断系统.并对可控硅的缺相故障进行了成功的诊断.试验结果证明,该方法比传统的神经网络具有速度快,准确度高的特点.     

基于生物机制脉冲神经网络的特征提取

脉冲神经元可以被用于处理生物刺激并且可以解释大脑复杂的智能行为。脉冲神经网络以非常逼近生物的神经元模型作为处理单元,可以直接用来仿真脑科学中发现的神经网络计算模型,输出的脉冲信号还可与生物神经系统对接。而小波变换是一个非常有利的时频分析工具,它可以有效的压缩图像并且提取图像的特征。本文中将提出一种与人类视觉系统的开/关神经元阵列相结合的脉冲神经网络,来实现针对视觉图像的快速小波变换。仿真结果显示,这个脉冲神经网络可以很好地保留视觉图像的关键特征。     

Delay learning and polychronization for reservoir computing

We propose a multi-timescale learning rule for spiking neuron networks, in the line of the recently emerging field of reservoir computing. The reservoir is a network model of spiking neurons, with random topology and driven by STDP (spike-time-dependent plasticity), a temporal Hebbian unsupervised learning mode, biologically observed. The model is further driven by a supervised learning algorithm, based on a margin criterion, that affects the synaptic delays linking the network to the readout neurons, with classification as a goal task. The network processing and the resulting performance can be explained by the concept of polychronization, proposed by Izhikevich [Polychronization: computation with spikes, Neural Comput. 18(2) (2006) 245–282], on physiological grounds. The model emphasizes that polychronization can be used as a tool for exploiting the computational power of synaptic delays and for monitoring the topology and activity of a spiking neuron network.     

Stochastic dynamics of a finite-size spiking neural network

We present a simple Markov model of spiking neural dynamics that can be analytically solved to characterize the stochastic dynamics of a finite-size spiking neural network. We give closed-form estimates for the equilibrium distribution, mean rate, variance, and autocorrelation function of the network activity. The model is applicable to any network where the probability of firing of a neuron in the network depends on only the number of neurons that fired in a previous temporal epoch. Networks with statistically homogeneous connectivity and membrane and synaptic time constants that are not excessively long could satisfy these conditions. Our model completely accounts for the size of the network and correlations in the firing activity. It also allows us to examine how the network dynamics can deviate from mean field theory. We show that the model and solutions are applicable to spiking neural networks in biophysically plausible parameter regimes.     

A new approach to higher-level information fusion using associative learning in semantic networks of spiking neurons

This paper presents a new approach to higher-level information fusion in which knowledge and data are represented using semantic networks composed of coupled spiking neuron nodes. Networks of simulated spiking neurons have been shown to exhibit synchronization, in which sub-assemblies of nodes become phase locked to one another. This phase locking reflects the tendency of biological neural systems to produce synchronized neural assemblies, which have been hypothesized to be involved in binding of low-level features in the perception of objects. The approach presented in this paper embeds spiking neurons in a semantic network, in which a synchronized sub-assembly of nodes represents a hypothesis about a situation. Likewise, multiple synchronized assemblies that are out-of-phase with one another represent multiple hypotheses. The initial network is hand-coded, but additional semantic relationships can be established by associative learning mechanisms. This approach is demonstrated by simulation of proof-of-concept scenarios involving the tracking of suspected criminal vehicles between meeting places in an urban environment. Our results indicate that synchronized sub-assemblies of spiking nodes can be used to represent multiple simultaneous events occurring in the environment and to effectively learn new relationships between semantic items in response to these events. In contrast to models of synchronized spiking networks that use physiologically realistic parameters in order to explain limits in human short-term memory (STM) capacity, our networks are not subject to the same limitations in representational capacity for multiple simultaneous events. Simulations demonstrate that the representational capacity of our networks can be very large, but as more simultaneous events are represented by synchronized sub-assemblies, the effective learning rate for establishing new relationships decreases. We propose that this effect could be countered by speeding up the spiking dynamics of the networks (a tactic of limited availability to biological systems). Such a speedup would allow the number of simultaneous events to increase without compromising the learning rate.     

多核并行脉冲神经网络模拟器的设计

脉冲神经网络属于第三代人工神经网络,它是更具有生物可解释性的神经网络模型。随着人们对脉冲神经网络不断深入地研究,不仅神经元空间结构更为复杂,而且神经网络结构规模也随之增大。以串行计算的方式,难以在个人计算机上实现脉冲神经网络的模拟仿真。为此,设计了一个多核并行的脉冲神经网络模拟器,对神经元进行编码与映射,自定义路由表解决了多核间的网络通信,以时间驱动为策略,实现核与核间的动态同步,在模拟器上进行脉冲神经网络的并行计算。以Izhikevich脉冲神经元为模型,在模拟环境下进行仿真实验,结果表明多核并行计算相比传统的串行计算在效率方面约有两倍的提升,可为类似的脉冲神经网络的模拟并行化设计提供参考。     

A Modified Spiking Neuron that Involves Derivative of the State Function at Firing Time

In usual spiking neural networks, the real world information is interpreted as spike time. A spiking neuron of the spiking neural network receives input vector of spike times, and activates a state function x(t) by increasing the time t until the value of x(t) reaches certain threshold value at a firing time t ^a . And t ^a is the output of the spiking neuron. In this paper we propose, and investigate the performance of, a modified spiking neuron, of which the output is a linear combination of the firing time t ^a and the derivative x??(t ^a ). The merit of the modified spiking neuron is shown by numerical experiments for solving some benchmark problems: The computational time of a modified spiking neuron is a little greater than that of a usual spiking neuron, but the accuracy of a modified spiking neuron is almost as good as a usual spiking neural network with a hidden layer.     

仿生型脉冲神经网络学习算法和网络模型

为解决脉冲神经网络训练困难的问题,基于仿生学思路,提出脉冲神经网络的权值学习算法和结构学习算法,设计一种含有卷积结构的脉冲神经网络模型,搭建适合脉冲神经网络的软件仿真平台。实验结果表明,权值学习算法训练的网络对MNIST数据集识别准确率能够达到84.12%,具备良好的快速收敛能力和低功耗特点;结构学习算法能够自动生成网络结构,具有高度生物相似性。     

Supervised Learning of Logical Operations in Layered Spiking Neural Networks with Spike Train Encoding

Few algorithms for supervised training of spiking neural networks exist that can deal with patterns of multiple spikes, and their computational properties are largely unexplored. We demonstrate in a set of simulations that the ReSuMe learning algorithm can successfully be applied to layered neural networks. Input and output patterns are encoded as spike trains of multiple precisely timed spikes, and the network learns to transform the input trains into target output trains. This is done by combining the ReSuMe learning algorithm with multiplicative scaling of the connections of downstream neurons. We show in particular that layered networks with one hidden layer can learn the basic logical operations, including Exclusive-Or, while networks without hidden layer cannot, mirroring an analogous result for layered networks of rate neurons. While supervised learning in spiking neural networks is not yet fit for technical purposes, exploring computational properties of spiking neural networks advances our understanding of how computations can be done with spike trains.     

一种量子衍生神经网络模型

为提高神经网络的逼近能力,通过在普通BP网络中引入量子旋转门,提出了一种新颖的量子衍生神经网络模型. 该模型隐层由量子神经元组成,每个量子神经元携带一组量子旋转门,用于更新隐层的量子权值,输入层和输出层均为普通神经元. 基于误差反传播算法设计了该模型的学习算法. 模式识别和函数逼近的实验结果验证了提出模型及算法的有效性.