STDP provides the substrate for igniting synfire chains by spatiotemporal input patterns

Spike-timing-dependent synaptic plasticity (STDP), which depends on the temporal difference between pre- and postsynaptic action potentials, is observed in the cortices and hippocampus. Although several theoretical and experimental studies have revealed its fundamental aspects, its functional role remains unclear. To examine how an input spatiotemporal spike pattern is altered by STDP, we observed the output spike patterns of a spiking neural network model with an asymmetrical STDP rule when the input spatiotemporal pattern is repeatedly applied. The spiking neural network comprises excitatory and inhibitory neurons that exhibit local interactions. Numerical experiments show that the spiking neural network generates a single global synchrony whose relative timing depends on the input spatiotemporal pattern and the neural network structure. This result implies that the spiking neural network learns the transformation from spatiotemporal to temporal information. In the literature, the origin of the synfire chain has not been sufficiently focused on. Our results indicate that spiking neural networks with STDP can ignite synfire chains in the cortices.     

Efficient computation based on stochastic spikes

The speed and reliability of mammalian perception indicate that cortical computations can rely on very few action potentials per involved neuron. Together with the stochasticity of single-spike events in cortex, this appears to imply that large populations of redundant neurons are needed for rapid computations with action potentials. Here we demonstrate that very fast and precise computations can be realized also in small networks of stochastically spiking neurons. We present a generative network model for which we derive biologically plausible algorithms that perform spike-by-spike updates of the neuron's internal states and adaptation of its synaptic weights from maximizing the likelihood of the observed spike patterns. Paradigmatic computational tasks demonstrate the online performance and learning efficiency of our framework. The potential relevance of our approach as a model for cortical computation is discussed.     

基于生物机制脉冲神经网络的特征提取

脉冲神经元可以被用于处理生物刺激并且可以解释大脑复杂的智能行为。脉冲神经网络以非常逼近生物的神经元模型作为处理单元,可以直接用来仿真脑科学中发现的神经网络计算模型,输出的脉冲信号还可与生物神经系统对接。而小波变换是一个非常有利的时频分析工具,它可以有效的压缩图像并且提取图像的特征。本文中将提出一种与人类视觉系统的开/关神经元阵列相结合的脉冲神经网络,来实现针对视觉图像的快速小波变换。仿真结果显示,这个脉冲神经网络可以很好地保留视觉图像的关键特征。     

多核并行脉冲神经网络模拟器的设计

脉冲神经网络属于第三代人工神经网络,它是更具有生物可解释性的神经网络模型。随着人们对脉冲神经网络不断深入地研究,不仅神经元空间结构更为复杂,而且神经网络结构规模也随之增大。以串行计算的方式,难以在个人计算机上实现脉冲神经网络的模拟仿真。为此,设计了一个多核并行的脉冲神经网络模拟器,对神经元进行编码与映射,自定义路由表解决了多核间的网络通信,以时间驱动为策略,实现核与核间的动态同步,在模拟器上进行脉冲神经网络的并行计算。以Izhikevich脉冲神经元为模型,在模拟环境下进行仿真实验,结果表明多核并行计算相比传统的串行计算在效率方面约有两倍的提升,可为类似的脉冲神经网络的模拟并行化设计提供参考。     

On computing Boolean functions by a spiking neuron

Computations by spiking neurons are performed using the timing of action potentials. We investigate the computational power of a simple model for such a spiking neuron in the Boolean domain by comparing it with traditional neuron models such as threshold gates (or McCulloch–Pitts neurons) and sigma-pi units (or polynomial threshold gates). In particular, we estimate the number of gates required to simulate a spiking neuron by a disjunction of threshold gates and we establish tight bounds for this threshold number. Furthermore, we analyze the degree of the polynomials that a sigma-pi unit must use for the simulation of a spiking neuron. We show that this degree cannot be bounded by any fixed value. Our results give evidence that the use of continuous time as a computational resource endows single-cell models with substantially larger computational capabilities. This revised version was published online in June 2006 with corrections to the Cover Date.     

实用神经网络的生物模拟器 待续

本文介绍一种能模拟单个神经元、触突以及系统的模拟器Ｓｉｍｕｌａｔｏｒ。用它模拟在生物界所熟知的一种无脊椎软体类似海参的食肉类动作Ｐｌｅｕｒｏ－ｂｒａｎｃｈｅａ。它的各种形为如吞食食物，遇敌退缩等用Ｆｏｒｔｒａｎ７７版本编程并在Ｓｕｎ３／５０工作站上进行模拟获得了满意结果。     

Stochastic dynamics of a finite-size spiking neural network

We present a simple Markov model of spiking neural dynamics that can be analytically solved to characterize the stochastic dynamics of a finite-size spiking neural network. We give closed-form estimates for the equilibrium distribution, mean rate, variance, and autocorrelation function of the network activity. The model is applicable to any network where the probability of firing of a neuron in the network depends on only the number of neurons that fired in a previous temporal epoch. Networks with statistically homogeneous connectivity and membrane and synaptic time constants that are not excessively long could satisfy these conditions. Our model completely accounts for the size of the network and correlations in the firing activity. It also allows us to examine how the network dynamics can deviate from mean field theory. We show that the model and solutions are applicable to spiking neural networks in biophysically plausible parameter regimes.     

Categorization of neural excitability using threshold models

A classification of spiking neurons according to the transition from quiescence to periodic firing of action potentials is commonly used. Nonbursting neurons are classified into two types, type I and type II excitability. We use simple phenomenological spiking neuron models to derive a criterion for the determination of the neural excitability based on the afterpotential following a spike. The crucial characteristic is the existence for type II model of a positive overshoot, that is, a delayed after depolarization, during the recovery process of the membrane potential. Our prediction is numerically tested using well-known type I and type II models including the Connor, Walter, & McKown (1977) model and the Hodgkin-Huxley (1952) model.     

Learning sensory maps with real-world stimuli in real time using abiophysically realistic learning rule

We present a real-time model of learning in the auditory cortex that is trained using real-world stimuli. The system consists of a peripheral and a central cortical network of spiking neurons. The synapses formed by peripheral neurons on the central ones are subject to synaptic plasticity. We implemented a biophysically realistic learning rule that depends on the precise temporal relation of pre- and postsynaptic action potentials. We demonstrate that this biologically realistic real-time neuronal system forms stable receptive fields that accurately reflect the spectral content of the input signals and that the size of these representations can be biased by global signals acting on the local learning mechanism. In addition, we show that this learning mechanism shows fast acquisition and is robust in the presence of large imbalances in the probability of occurrence of individual stimuli and noise.     

Delay learning and polychronization for reservoir computing

We propose a multi-timescale learning rule for spiking neuron networks, in the line of the recently emerging field of reservoir computing. The reservoir is a network model of spiking neurons, with random topology and driven by STDP (spike-time-dependent plasticity), a temporal Hebbian unsupervised learning mode, biologically observed. The model is further driven by a supervised learning algorithm, based on a margin criterion, that affects the synaptic delays linking the network to the readout neurons, with classification as a goal task. The network processing and the resulting performance can be explained by the concept of polychronization, proposed by Izhikevich [Polychronization: computation with spikes, Neural Comput. 18(2) (2006) 245–282], on physiological grounds. The model emphasizes that polychronization can be used as a tool for exploiting the computational power of synaptic delays and for monitoring the topology and activity of a spiking neuron network.     

A brain-like neural network for periodicity analysis.

This paper introduces a brain-like neural model for sound processing. The periodicity analyzing network (PAN) is a bio-inspired neural network of spiking neurons. The PAN consists of complex models of neurons, which can be used for understanding the dynamics of individual neurons and neuronal networks. On a technical level, the PAN is able to compute the ratio of modulation and carrier frequency of harmonic sound signals. The PAN model may, therefore, be used in audio signal processing applications, such as sound source separation, periodicity analysis, and the cocktail party problem.     

基于卷积计算的多层脉冲神经网络的监督学习

针对脉冲神经元基于精确定时的多脉冲编码信息的特点,提出了一种基于卷积计算的多层脉冲神经网络监督学习的新算法。该算法应用核函数的卷积计算将离散的脉冲序列转换为连续函数,在多层前馈脉冲神经网络结构中,使用梯度下降的方法得到基于核函数卷积表示的学习规则,并用来调整神经元连接的突触权值。在实验部分,首先验证了该算法学习脉冲序列的效果,然后应用该算法对Iris数据集进行分类。结果显示,该算法能够实现脉冲序列复杂时空模式的学习,对非线性模式分类问题具有较高的分类正确率。     

Dynamic stochastic synapses as computational units

In most neural network models, synapses are treated as static weights that change only with the slow time scales of learning. It is well known, however, that synapses are highly dynamic and show use-dependent plasticity over a wide range of time scales. Moreover, synaptic transmission is an inherently stochastic process: a spike arriving at a presynaptic terminal triggers the release of a vesicle of neurotransmitter from a release site with a probability that can be much less than one. We consider a simple model for dynamic stochastic synapses that can easily be integrated into common models for networks of integrate-and-fire neurons (spiking neurons). The parameters of this model have direct interpretations in terms of synaptic physiology. We investigate the consequences of the model for computing with individual spikes and demonstrate through rigorous theoretical results that the computational power of the network is increased through the use of dynamic synapses.     

面向图像识别的多层脉冲神经网络学习算法综述

相较于第1代和第2代神经网络,第3代神经网络的脉冲神经网络是一种更加接近于生物神经网络的模型,因此更具有生物可解释性和低功耗性。基于脉冲神经元模型,脉冲神经网络可以通过脉冲信号的形式模拟生物信号在神经网络中的传播,通过脉冲神经元的膜电位变化来发放脉冲序列,脉冲序列通过时空联合表达不仅传递了空间信息还传递了时间信息。当前面向模式识别任务的脉冲神经网络模型性能还不及深度学习,其中一个重要原因在于脉冲神经网络的学习方法不成熟,深度学习中神经网络的人工神经元是基于实数形式的输出,这使得其可以使用全局性的反向传播算法对深度神经网络的参数进行训练,脉冲序列是二值性的离散输出,这直接导致对脉冲神经网络的训练存在一定困难,如何对脉冲神经网络进行高效训练是一个具有挑战的研究问题。本文首先总结了脉冲神经网络研究领域中的相关学习算法,然后对其中主要的方法:直接监督学习、无监督学习的算法以及ANN2SNN的转换算法进行分析介绍,并对其中代表性的工作进行对比分析,最后基于对当前主流方法的总结,对未来更高效、更仿生的脉冲神经网络参数学习方法进行展望。     

Spiking神经元输入脉冲扰动敏感性研究

脉冲神经网络是一种基于生物的网络模型,它的输入输出为具有时间特性的脉冲序列,其运行机制相比其他传统人工神经网络更加接近于生物神经网络。神经元之间通过脉冲序列传递信息,这些信息通过脉冲的激发时间编码能够更有效地发挥网络的学习性能。脉冲神经元的时间特性导致了其工作机制较为复杂,而spiking神经元的敏感性反映了当神经元输入发生扰动时输出的spike的变化情况,可以作为研究神经元内部工作机制的工具。不同于传统的神经网络,spiking神经元敏感性定义为输出脉冲的变化时刻个数与运行时间长度的比值,能直接反映出输入扰动对输出的影响程度。通过对不同形式的输入扰动敏感性的分析,可以看出spiking神经元的敏感性较为复杂,当全体突触发生扰动时,神经元为定值,而当部分突触发生扰动时,不同突触的扰动会导致不同大小的神经元敏感性。     

Modeling synaptic plasticity in conjuction with the timing of pre- and postsynaptic action potentials

We present a spiking neuron model that allows for an analytic calculation of the correlations between pre- and postsynaptic spikes. The neuron model is a generalization of the integrate-and-fire model and equipped with a probabilistic spike-triggering mechanism. We show that under certain biologically plausible conditions, pre- and postsynaptic spike trains can be described simultaneously as an inhomogeneous Poisson process. Inspired by experimental findings, we develop a model for synaptic long-term plasticity that relies on the relative timing of pre- and post-synaptic action potentials. Being given an input statistics, we compute the stationary synaptic weights that result from the temporal correlations between the pre- and postsynaptic spikes. By means of both analytic calculations and computer simulations, we show that such a mechanism of synaptic plasticity is able to strengthen those input synapses that convey precisely timed spikes at the expense of synapses that deliver spikes with a broad temporal distribution. This may be of vital importance for any kind of information processing based on spiking neurons and temporal coding.     

Learning a sparse code for temporal sequences using STDP and sequence compression

A spiking neural network that learns temporal sequences is described. A sparse code in which individual neurons represent sequences and subsequences enables multiple sequences to be stored without interference. The network is founded on a model of sequence compression in the hippocampus that is robust to variation in sequence element duration and well suited to learn sequences through spike-timing dependent plasticity (STDP). Three additions to the sequence compression model underlie the sparse representation: synapses connecting the neurons of the network that are subject to STDP, a competitive plasticity rule so that neurons specialize to individual sequences, and neural depolarization after spiking so that neurons have a memory. The response to new sequence elements is determined by the neurons that have responded to the previous subsequence, according to the competitively learned synaptic connections. Numerical simulations show that the model can learn sets of intersecting sequences, presented with widely differing frequencies, with elements of varying duration.     

An automatic self-installation and calibration method for a 3D position sensing system using ultrasonics

This work addresses 3D position sensing systems that estimate the location of a wave source by triangulating its position based on the time-of-flights (TOFs) to various receivers fixed to an inertial frame of reference. Typical applications of such systems are finding the location of the transmitter that may be fixed to an autonomously guided vehicle (AGV) operating in an enclosed work environment, a robot end-effector, or virtual reality environments. These environments constitute a large working volume, and the receivers have to be fixed in this environment and their locations known exactly. This is a major source of problems in the installation/calibration stage since the receivers are usually distributed in space and finding their exact location entails using a separate 3D calibrating device which may or may not be as accurate as the location system itself. This paper presents a method to use the system itself to set up an inertial frame of reference and find out the locations of the receivers within this frame by simply using an accurate ID positioning system, e.g. an accurate ruler or a simple distance measuring system that uses ultrasonic or infrared sensors. The method entails moving the transmitter to known locations on a single plane, and using the TOFs to estimate the location of the receivers. A typical application would be that an AGV carries a set of receivers to a hazardous environment such as a nuclear power plant, places the receivers arbitrarily, carries out the self-installation/calibration procedure, maps out the environment, and begins to function autonomously, the whole procedure being done without human intervention or supervision.     

The high-conductance state of cortical networks

We studied the dynamics of large networks of spiking neurons with conductance-based (nonlinear) synapses and compared them to networks with current-based (linear) synapses. For systems with sparse and inhibition-dominated recurrent connectivity, weak external inputs induced asynchronous irregular firing at low rates. Membrane potentials fluctuated a few millivolts below threshold, and membrane conductances were increased by a factor 2 to 5 with respect to the resting state. This combination of parameters characterizes the ongoing spiking activity typically recorded in the cortex in vivo. Many aspects of the asynchronous irregular state in conductance-based networks could be sufficiently well characterized with a simple numerical mean field approach. In particular, it correctly predicted an intriguing property of conductance-based networks that does not appear to be shared by current-based models: they exhibit states of low-rate asynchronous irregular activity that persist for some period of time even in the absence of external inputs and without cortical pacemakers. Simulations of larger networks (up to 350,000 neurons) demonstrated that the survival time of self-sustained activity increases exponentially with network size.     

不同耦合类型的三维Hybrid神经元同步动力学

神经元网络的信息传递与多个神经元间的耦合同步密切相关.大量研究表明,神经元耦合系统的同步化问题是研究大脑处理信息的关键.本文基于三维混合神经元模型研究了神经元的复杂放电,该模型由Wilson模型的快子系统和H-R模型的慢子系统组成.该模型能够重现大脑皮层神经元一系列包括规则峰放电、快速峰放电和簇放电等神经动力学行为.本文基于三维混合神经元模型,探讨了三维混合神经元在电突触耦合、化学突触耦合和磁通耦合条件下的同步放电行为,模拟膜电位序列图、相位差图等探讨恒同及非恒同下耦合强度对神经元同步放电的影响.本研究将为人们进一步了解神经系统疾病的发病机制提供指导和帮助,并为神经科学领域提供可能的研究思路.