Random neural networks with synchronized interactions

Large-scale distributed systems, such as natural neuronal and artificial systems, have many local interconnections, but they often also have the ability to propagate information very fast over relatively large distances. Mechanisms that enable such behavior include very long physical signaling paths and possibly saccades of synchronous behavior that may propagate across a network. This letter studies the modeling of such behaviors in neuronal networks and develops a related learning algorithm. This is done in the context of the random neural network (RNN), a probabilistic model with a well-developed mathematical theory, which was inspired by the apparently stochastic spiking behavior of certain natural neuronal systems. Thus, we develop an extension of the RNN to the case when synchronous interactions can occur, leading to synchronous firing by large ensembles of cells. We also present an O(N3) gradient descent learning algorithm for an N-cell recurrent network having both conventional excitatory-inhibitory interactions and synchronous interactions. Finally, the model and its learning algorithm are applied to a resource allocation problem that is NP-hard and requires fast approximate decisions.     

How noise affects the synchronization properties of recurrent networks of inhibitory neurons

GABAergic interneurons play a major role in the emergence of various types of synchronous oscillatory patterns of activity in the central nervous system. Motivated by these experimental facts, modeling studies have investigated mechanisms for the emergence of coherent activity in networks of inhibitory neurons. However, most of these studies have focused either when the noise in the network is absent or weak or in the opposite situation when it is strong. Hence, a full picture of how noise affects the dynamics of such systems is still lacking. The aim of this letter is to provide a more comprehensive understanding of the mechanisms by which the asynchronous states in large, fully connected networks of inhibitory neurons are destabilized as a function of the noise level. Three types of single neuron models are considered: the leaky integrate-and-fire (LIF) model, the exponential integrate-and-fire (EIF), model and conductance-based models involving sodium and potassium Hodgkin-Huxley (HH) currents. We show that in all models, the instabilities of the asynchronous state can be classified in two classes. The first one consists of clustering instabilities, which exist in a restricted range of noise. These instabilities lead to synchronous patterns in which the population of neurons is broken into clusters of synchronously firing neurons. The irregularity of the firing patterns of the neurons is weak. The second class of instabilities, termed oscillatory firing rate instabilities, exists at any value of noise. They lead to cluster state at low noise. As the noise is increased, the instability occurs at larger coupling, and the pattern of firing that emerges becomes more irregular. In the regime of high noise and strong coupling, these instabilities lead to stochastic oscillations in which neurons fire in an approximately Poisson way with a common instantaneous probability of firing that oscillates in time.     

考虑树突整合效应的神经元网络的放电和同步特性

生物神经网络的同步被认为在大脑神经信息的处理过程中发挥了重要作用.本文在Hodgkin-Huxley(HH)神经元网络模型中考虑树突整合效应,得到修正后的DHH(Dendritic-integration-rule-based HH)神经元网络模型,研究了网络的放电和同步特性.首先以三个抑制性神经元构成的耦合系统为例,发现树突整合效应的加入提高了神经元的放电阈值;然后分别建立全局耦合的抑制性和兴奋性神经元网络,发现大的耦合强度能够诱导抑制性和兴奋性神经元网络达到几乎完全同步的状态,并且对神经元的放电幅值有较大的影响;更有趣的是,当树突整合系数为某一值时,抑制性神经元网络的同步达到最高,而兴奋性神经网络的同步达到最低.     

The effects of pair-wise and higher order correlations on the firing rate of a post-synaptic neuron

Coincident firing of neurons projecting to a common target cell is likely to raise the probability of firing of this postsynaptic cell. Therefore, synchronized firing constitutes a significant event for postsynaptic neurons and is likely to play a role in neuronal information processing. Physiological data on synchronized firing in cortical networks are based primarily on paired recordings and cross-correlation analysis. However, pair-wise correlations among all inputs onto a postsynaptic neuron do not uniquely determine the distribution of simultaneous postsynaptic events. We develop a framework in order to calculate the amount of synchronous firing that, based on maximum entropy, should exist in a homogeneous neural network in which the neurons have known pair-wise correlations and higher-order structure is absent. According to the distribution of maximal entropy, synchronous events in which a large proportion of the neurons participates should exist even in the case of weak pair-wise correlations. Network simulations also exhibit these highly synchronous events in the case of weak pair-wise correlations. If such a group of neurons provides input to a common postsynaptic target, these network bursts may enhance the impact of this input, especially in the case of a high postsynaptic threshold. The proportion of neurons participating in synchronous bursts can be approximated by our method under restricted conditions. When these conditions are not fulfilled, the spike trains have less than maximal entropy, which is indicative of the presence of higher-order structure. In this situation, the degree of synchronicity cannot be derived from the pair-wise correlations.     

Asynchronous states and the emergence of synchrony in large networks of interacting excitatory and inhibitory neurons

We investigate theoretically the conditions for the emergence of synchronous activity in large networks, consisting of two populations of extensively connected neurons, one excitatory and one inhibitory. The neurons are modeled with quadratic integrate-and-fire dynamics, which provide a very good approximation for the subthreshold behavior of a large class of neurons. In addition to their synaptic recurrent inputs, the neurons receive a tonic external input that varies from neuron to neuron. Because of its relative simplicity, this model can be studied analytically. We investigate the stability of the asynchronous state (AS) of the network with given average firing rates of the two populations. First, we show that the AS can remain stable even if the synaptic couplings are strong. Then we investigate the conditions under which this state can be destabilized. We show that this can happen in four generic ways. The first is a saddle-node bifurcation, which leads to another state with different average firing rates. This bifurcation, which occurs for strong enough recurrent excitation, does not correspond to the emergence of synchrony. In contrast, in the three other instability mechanisms, Hopf bifurcations, which correspond to the emergence of oscillatory synchronous activity, occur. We show that these mechanisms can be differentiated by the firing patterns they generate and their dependence on the mutual interactions of the inhibitory neurons and cross talk between the two populations. We also show that besides these codimension 1 bifurcations, the system can display several codimension 2 bifurcations: Takens-Bogdanov, Gavrielov-Guckenheimer, and double Hopf bifurcations.     

Polychronization: computation with spikes

We present a minimal spiking network that can polychronize, that is, exhibit reproducible time-locked but not synchronous firing patterns with millisecond precision, as in synfire braids. The network consists of cortical spiking neurons with axonal conduction delays and spike-timing-dependent plasticity (STDP); a ready-to-use MATLAB code is included. It exhibits sleeplike oscillations, gamma (40 Hz) rhythms, conversion of firing rates to spike timings, and other interesting regimes. Due to the interplay between the delays and STDP, the spiking neurons spontaneously self-organize into groups and generate patterns of stereotypical polychronous activity. To our surprise, the number of coexisting polychronous groups far exceeds the number of neurons in the network, resulting in an unprecedented memory capacity of the system. We speculate on the significance of polychrony to the theory of neuronal group selection (TNGS, neural Darwinism), cognitive neural computations, binding and gamma rhythm, mechanisms of attention, and consciousness as "attention to memories."     

Correlation between uncoupled conductance-based integrate-and-fire neurons due to common and synchronous presynaptic firing

We investigate the firing characteristics of conductance-based integrate-and-fire neurons and the correlation of firing for uncoupled pairs of neurons as a result of common input and synchronous firing of multiple synaptic inputs. Analytical approximations are derived for the moments of the steady state potential and the effective time constant. We show that postsynaptic firing barely depends on the correlation between inhibitory inputs; only the inhibitory firing rate matters. In contrast, both the degree of synchrony and the firing rate of excitatory inputs are relevant. A coefficient of variation CV > 1 can be attained with low inhibitory firing rates and (Poisson-modulated) synchronized excitatory synaptic input, where both the number of presynaptic neurons in synchronous firing assemblies and the synchronous firing rate should be sufficiently large. The correlation in firing of a pair of uncoupled neurons due to common excitatory input is initially increased for increasing firing rates of independent inhibitory inputs but decreases for large inhibitory firing rates. Common inhibitory input to a pair of uncoupled neurons barely induces correlated firing, but amplifies the effect of common excitation. Synchronous firing assemblies in the common input further enhance the correlation and are essential to attain experimentally observed correlation values. Since uncorrelated common input (i.e., common input by neurons, which do not fire in synchrony) cannot induce sufficient postsynaptic correlation, we conclude that lateral couplings are essential to establish clusters of synchronously firing neurons.     

Spatiotemporal coding in the cortex: information flow-based learning in spiking neural networks

We introduce a learning paradigm for networks of integrate-and-fire spiking neurons that is based on an information-theoretic criterion. This criterion can be viewed as a first principle that demonstrates the experimentally observed fact that cortical neurons display synchronous firing for some stimuli and not for others. The principle can be regarded as the postulation of a nonparametric reconstruction method as optimization criteria for learning the required functional connectivity that justifies and explains synchronous firing for binding of features as a mechanism for spatiotemporal coding. This can be expressed in an information-theoretic way by maximizing the discrimination ability between different sensory inputs in minimal time.     

Population density methods for large-scale modelling of neuronal networks with realistic synaptic kinetics: cutting the dimension down to size

Population density methods provide promising time-saving alternatives to direct Monte Carlo simulations of neuronal network activity, in which one tracks the state of thousands of individual neurons and synapses. A population density method has been found to be roughly a hundred times faster than direct simulation for various test networks of integrate-and-fire model neurons with instantaneous excitatory and inhibitory post-synaptic conductances. In this method, neurons are grouped into large populations of similar neurons. For each population, one calculates the evolution of a probability density function (PDF) which describes the distribution of neurons over state space. The population firing rate is then given by the total flux of probability across the threshold voltage for firing an action potential. Extending the method beyond instantaneous synapses is necessary for obtaining accurate results, because synaptic kinetics play an important role in network dynamics. Embellishments incorporating more realistic synaptic kinetics for the underlying neuron model increase the dimension of the PDF, which was one-dimensional in the instantaneous synapse case. This increase in dimension causes a substantial increase in computation time to find the exact PDF, decreasing the computational speed advantage of the population density method over direct Monte Carlo simulation. We report here on a one-dimensional model of the PDF for neurons with arbitrary synaptic kinetics. The method is more accurate than the mean-field method in the steady state, where the mean-field approximation works best, and also under dynamic-stimulus conditions. The method is much faster than direct simulations. Limitations of the method are demonstrated, and possible improvements are discussed.     

Modified firing-rate model reproduces synchronization of a neuronal population receiving complex input

Synchronization plays important role in generation of brain activity patterns. Experimental data show that neurons demonstrate more reproducible activity for noise-like input than for constant current injection, and that effect can not be reproduced by standard oversimplified Firing-Rate (FR) models. The paper proposes a modification of FR model which reproduces these kinds of activity. The FR model approximates the firing rate of an infinite number of leaky integrate-and-fire neurons, considered as a population, and in contrary to conventional models it accounts for not only a steady-state firing regime but a fast rising excitation as well. Comparison of our simulations with the experimental data shows that the synchronous firing of the neuronal population strongly depends on the synchrony of neuronal states just before spiking. This effect is reproduced by the proposed FR model in contrary to the conventional FR models and is in agreement with the direct Monte-Carlo simulation of individual neurons.     

Dynamics of strongly-coupled spiking neurons

We present a dynamical theory of integrate-and-fire neurons with strong synaptic coupling. We show how phase-locked states that are stable in the weak coupling regime can destabilize as the coupling is increased, leading to states characterized by spatiotemporal variations in the interspike intervals (ISIs). The dynamics is compared with that of a corresponding network of analog neurons in which the outputs of the neurons are taken to be mean firing rates. A fundamental result is that for slow interactions, there is good agreement between the two models (on an appropriately defined timescale). Various examples of desynchronization in the strong coupling regime are presented. First, a globally coupled network of identical neurons with strong inhibitory coupling is shown to exhibit oscillator death in which some of the neurons suppress the activity of others. However, the stability of the synchronous state persists for very large networks and fast synapses. Second, an asymmetric network with a mixture of excitation and inhibition is shown to exhibit periodic bursting patterns. Finally, a one-dimensional network of neurons with long-range interactions is shown to desynchronize to a state with a spatially periodic pattern of mean firing rates across the network. This is modulated by deterministic fluctuations of the instantaneous firing rate whose size is an increasing function of the speed of synaptic response.     

Patterns of synchrony in neural networks with spike adaptation

We study the emergence of synchronized burst activity in networks of neurons with spike adaptation. We show that networks of tonically firing adapting excitatory neurons can evolve to a state where the neurons burst in a synchronized manner. The mechanism leading to this burst activity is analyzed in a network of integrate-and-fire neurons with spike adaptation. The dependence of this state on the different network parameters is investigated, and it is shown that this mechanism is robust against inhomogeneities, sparseness of the connectivity, and noise. In networks of two populations, one excitatory and one inhibitory, we show that decreasing the inhibitory feedback can cause the network to switch from a tonically active, asynchronous state to the synchronized bursting state. Finally, we show that the same mechanism also causes synchronized burst activity in networks of more realistic conductance-based model neurons.     

Synchrony and desynchrony in integrate-and-fire oscillators.

Due to many experimental reports of synchronous neural activity in the brain, there is much interest in understanding synchronization in networks of neural oscillators and its potential for computing perceptual organization. Contrary to Hopfield and Herz (1995), we find that networks of locally coupled integrate-and-fire oscillators can quickly synchronize. Furthermore, we examine the time needed to synchronize such networks. We observe that these networks synchronize at times proportional to the logarithm of their size, and we give the parameters used to control the rate of synchronization. Inspired by locally excitatory globally inhibitory oscillator network (LEGION) dynamics with relaxation oscillators (Terman & Wang, 1995), we find that global inhibition can play a similar role of desynchronization in a network of integrate-and-fire oscillators. We illustrate that a LEGION architecture with integrate-and-fire oscillators can be similarly used to address image analysis.     

Analysing neurobiological models using communicating automata

Two important issues in computational modelling in cognitive neuroscience are: first, how to formally describe neuronal networks (i.e. biologically plausible models of the central nervous system), and second, how to analyse complex models, in particular, their dynamics and capacity to learn. We make progress towards these goals by presenting a communicating automata perspective on neuronal networks. Specifically, we describe neuronal networks and their biological mechanisms using Data-rich Communicating Automata, which extend classic automata theory with rich data types and communication. We use two case studies to illustrate our approach. In the first case study, we model a number of learning frameworks, which vary in respect of their biological detail, for instance the Backpropagation (BP) and the Generalized Recirculation (GeneRec) learning algorithms. We then used the SPIN model checker to investigate a number of behavioral properties of the neural learning algorithms. SPIN is a well-known model checker for reactive distributed systems, which has been successfully applied to many non-trivial problems. The verification results show that the biologically plausible GeneRec learning is less stable than BP learning. In the second case study, we presented a large scale (cognitive-level) neuronal network, which models an attentional spotlight mechanism in the visual system. A set of properties of this model was verified using Uppaal, a popular real-time model checker. The results show that the asynchronous processing supported by concurrency theory is not only a more biologically plausible way to model neural systems, but also provides a better performance in cognitive modelling of the brain than conventional artificial neural networks that use synchronous updates. Finally, we compared our approach with several other related theories that apply formal methods to cognitive modelling. In addition, the practical implications of the approach are discussed in the context of neuronal network based controllers.     

A generalized linear integrate-and-fire neural model produces diverse spiking behaviors

For simulations of neural networks, there is a trade-off between the size of the network that can be simulated and the complexity of the model used for individual neurons. In this study, we describe a generalization of the leaky integrate-and-fire model that produces a wide variety of spiking behaviors while still being analytically solvable between firings. For different parameter values, the model produces spiking or bursting, tonic, phasic or adapting responses, depolarizing or hyperpolarizing after potentials and so forth. The model consists of a diagonalizable set of linear differential equations describing the time evolution of membrane potential, a variable threshold, and an arbitrary number of firing-induced currents. Each of these variables is modified by an update rule when the potential reaches threshold. The variables used are intuitive and have biological significance. The model's rich behavior does not come from the differential equations, which are linear, but rather from complex update rules. This single-neuron model can be implemented using algorithms similar to the standard integrate-and-fire model. It is a natural match with event-driven algorithms for which the firing times are obtained as a solution of a polynomial equation.     

Gamma oscillations in a nonlinear regime: a minimal model approach using heterogeneous integrate-and-fire networks

Fast oscillations and in particular gamma-band oscillation (20-80 Hz) are commonly observed during brain function and are at the center of several neural processing theories. In many cases, mathematical analysis of fast oscillations in neural networks has been focused on the transition between irregular and oscillatory firing viewed as an instability of the asynchronous activity. But in fact, brain slice experiments as well as detailed simulations of biological neural networks have produced a large corpus of results concerning the properties of fully developed oscillations that are far from this transition point. We propose here a mathematical approach to deal with nonlinear oscillations in a network of heterogeneous or noisy integrate-and-fire neurons connected by strong inhibition. This approach involves limited mathematical complexity and gives a good sense of the oscillation mechanism, making it an interesting tool to understand fast rhythmic activity in simulated or biological neural networks. A surprising result of our approach is that under some conditions, a change of the strength of inhibition only weakly influences the period of the oscillation. This is in contrast to standard theoretical and experimental models of interneuron network gamma oscillations (ING), where frequency tightly depends on inhibition strength, but it is similar to observations made in some in vitro preparations in the hippocampus and the olfactory bulb and in some detailed network models. This result is explained by the phenomenon of suppression that is known to occur in strongly coupled oscillating inhibitory networks but had not yet been related to the behavior of oscillation frequency.     

局部耦合的双层忆阻神经元网络的嵌合态和同步

嵌合态被发现存在于神经系统并且可能在神经元节律、大脑的睡眠和记忆等诸多神经过程中发挥重要作用.本文考虑神经元交互中的电磁感应现象,建立了以Hindmarsh Rose神经元为节点的局部耦合的双层忆阻神经元网络,研究其嵌合态时空动力学模式及产生机理.结果发现,改变层内、层间突触耦合强度会使网络产生移动和不完美移动嵌合态等多种类型的嵌合模式,其中不完美移动嵌合态中不相干的区域会扩展到网络的相干域.特别地,在特定耦合强度下,存在一种新的嵌合态活动模式,即一部分神经元处于嵌合态,另一部分神经元处于移动嵌合态.考虑神经元突触的忆阻特性,发现忆阻参数的增加能够使处于嵌合态的神经元网络转变为同步态,且耦合强度越大,达到同步态所需要的忆阻参数值越小.进一步探究双层网络的同步性,发现层间耦合强度和忆阻参数的增大有助于网络达到更好的同步.研究结果表明神经元之间的相互作用可以激发双层神经元网络产生多种嵌合态模式,电磁感应可以促进网络由嵌合态向同步态转迁,这些结果有助于理解人脑中复杂的神经放电过程和信息处理机制,并为可能的类脑装置应用提供参考.     

Time-series prediction with single integrate-and-fire neuron

In this paper, a learning algorithm for a single integrate-and-fire neuron (IFN) is proposed and tested for various applications in which a multilayer perceptron neural network is conventionally used. It is found that a single IFN is sufficient for the applications that require a number of neurons in different hidden layers of a conventional neural network. Several benchmark and real-life problems of classification and time-series prediction have been illustrated. It is observed that the inclusion of some more biological phenomenon in an artificial neural network can make it more powerful.     

耦合Hindmarsh-Rose神经元的同步放电模式及转迁

通过数值模拟和分岔分析,探究了具有不同放电模式的两电耦合Hindmarsh-Rose神经元的几乎完全同步,并研究了同步放电模式对单个个体的放电模式的依赖性.研究结果有助于我们更好地理解神经元放电模式转迁的动力学机理和生物学意义.