Computational properties of networks of synchronous groups of spiking neurons

We demonstrate a model in which synchronously firing ensembles of neurons are networked to produce computational results. Each ensemble is a group of biological integrate-and-fire spiking neurons, with probabilistic interconnections between groups. An analogy is drawn in which each individual processing unit of an artificial neural network corresponds to a neuronal group in a biological model. The activation value of a unit in the artificial neural network corresponds to the fraction of active neurons, synchronously firing, in a biological neuronal group. Weights of the artificial neural network correspond to the product of the interconnection density between groups, the group size of the presynaptic group, and the postsynaptic potential heights in the synchronous group model. All three of these parameters can modulate connection strengths between neuronal groups in the synchronous group models. We give an example of nonlinear classification (XOR) and a function approximation example in which the capability of the artificial neural network can be captured by a neural network model with biological integrate-and-fire neurons configured as a network of synchronously firing ensembles of such neurons. We point out that the general function approximation capability proven for feedforward artificial neural networks appears to be approximated by networks of neuronal groups that fire in synchrony, where the groups comprise integrate-and-fire neurons. We discuss the advantages of this type of model for biological systems, its possible learning mechanisms, and the associated timing relationships.     

Higher-order statistics of input ensembles and the response of simple model neurons

Pairwise correlations among spike trains recorded in vivo have been frequently reported. It has been argued that correlated activity could play an important role in the brain, because it efficiently modulates the response of a postsynaptic neuron. We show here that a neuron's output firing rate critically depends on the higher-order statistics of the input ensemble. We constructed two statistical models of populations of spiking neurons that fired with the same rates and had identical pairwise correlations, but differed with regard to the higher-order interactions within the population. The first ensemble was characterized by clusters of spikes synchronized over the whole population. In the second ensemble, the size of spike clusters was, on average, proportional to the pairwise correlation. For both input models, we assessed the role of the size of the population, the firing rate, and the pairwise correlation on the output rate of two simple model neurons: a continuous firing-rate model and a conductance-based leaky integrate-and-fire neuron. An approximation to the mean output rate of the firing-rate neuron could be derived analytically with the help of shot noise theory. Interestingly, the essential features of the mean response of the two neuron models were similar. For both neuron models, the three input parameters played radically different roles with respect to the postsynaptic firing rate, depending on the interaction structure of the input. For instance, in the case of an ensemble with small and distributed spike clusters, the output firing rate was efficiently controlled by the size of the input population. In addition to the interaction structure, the ratio of inhibition to excitation was found to strongly modulate the effect of correlation on the postsynaptic firing rate.     

考虑树突整合效应的神经元网络的放电和同步特性

生物神经网络的同步被认为在大脑神经信息的处理过程中发挥了重要作用.本文在Hodgkin-Huxley(HH)神经元网络模型中考虑树突整合效应,得到修正后的DHH(Dendritic-integration-rule-based HH)神经元网络模型,研究了网络的放电和同步特性.首先以三个抑制性神经元构成的耦合系统为例,发现树突整合效应的加入提高了神经元的放电阈值;然后分别建立全局耦合的抑制性和兴奋性神经元网络,发现大的耦合强度能够诱导抑制性和兴奋性神经元网络达到几乎完全同步的状态,并且对神经元的放电幅值有较大的影响;更有趣的是,当树突整合系数为某一值时,抑制性神经元网络的同步达到最高,而兴奋性神经网络的同步达到最低.     

Classification of stationary neuronal activity according to its information rate

We propose a measure of the information rate of a single stationary neuronal activity with respect to the state of null information. The measure is based on the Kullback-Leibler distance between two interspike interval distributions. The selected activity is compared with the Poisson model with the same mean firing frequency. We show that the approach is related to the notion of specific information and that the method allows us to judge the relative encoding efficiency. Two classes of neuronal activity models are classified according to their information rate: the renewal process models and the first-order Markov chain models. It has been proven that information can be transmitted changing neither the spike rate nor the coefficient of variation and that the increase in serial correlation does not necessarily increase the information gain. We employ the simple, but powerful, Vasicek's estimator of differential entropy to illustrate an application on the experimental data coming from olfactory sensory neurons of rats.     

How noise affects the synchronization properties of recurrent networks of inhibitory neurons

GABAergic interneurons play a major role in the emergence of various types of synchronous oscillatory patterns of activity in the central nervous system. Motivated by these experimental facts, modeling studies have investigated mechanisms for the emergence of coherent activity in networks of inhibitory neurons. However, most of these studies have focused either when the noise in the network is absent or weak or in the opposite situation when it is strong. Hence, a full picture of how noise affects the dynamics of such systems is still lacking. The aim of this letter is to provide a more comprehensive understanding of the mechanisms by which the asynchronous states in large, fully connected networks of inhibitory neurons are destabilized as a function of the noise level. Three types of single neuron models are considered: the leaky integrate-and-fire (LIF) model, the exponential integrate-and-fire (EIF), model and conductance-based models involving sodium and potassium Hodgkin-Huxley (HH) currents. We show that in all models, the instabilities of the asynchronous state can be classified in two classes. The first one consists of clustering instabilities, which exist in a restricted range of noise. These instabilities lead to synchronous patterns in which the population of neurons is broken into clusters of synchronously firing neurons. The irregularity of the firing patterns of the neurons is weak. The second class of instabilities, termed oscillatory firing rate instabilities, exists at any value of noise. They lead to cluster state at low noise. As the noise is increased, the instability occurs at larger coupling, and the pattern of firing that emerges becomes more irregular. In the regime of high noise and strong coupling, these instabilities lead to stochastic oscillations in which neurons fire in an approximately Poisson way with a common instantaneous probability of firing that oscillates in time.     

神经元膜电位对信息的编码

神经元膜电位的放电活动是神经编码的基础。然而,目前对于神经元电活动对神经信息的编码方式,至今尚未形成一个完整的认识。传统的编码理论认为神经系统以离散的动作电位放电序列进行信息的表达和传递,主要研究动作电位的发放频率和放电活动的时间模式。基于该理论,对神经元放电序列所携带的信息已经出现了一些定量的计算方法,但这些方法还很难应用到大规模神经元网络的计算当中。本研究以神经元的膜电位为研究对象,展示了如何量化膜电位序列所携带的信息,并将该计算结果与传统放电序列方沣的计算结果进行了对比分析,其结果取得了很好的一致性。本研究为神经活动信息量的定量计算提供了一种新的思路和方法。     

The relationship between synchronization among neuronal populations and their mean activity levels.

In the past decade the importance of synchronized dynamics in the brain has emerged from both empirical and theoretical perspectives. Fast dynamic synchronous interactions of an oscillatory or nonoscillatory nature may constitute a form of temporal coding that underlies feature binding and perceptual synthesis. The relationship between synchronization among neuronal populations and the population firing rates addresses two important issues: the distinction between rate coding and synchronization coding models of neuronal interactions and the degree to which empirical measurements of population activity, such as those employed by neuroimaging, are sensitive to changes in synchronization. We examined the relationship between mean population activity and synchronization using biologically plausible simulations. In this article, we focus on continuous stationary dynamics. (In a companion article, Chawla (forthcoming), we address the same issue using stimulus-evoked transients.) By manipulation parameters such as extrinsic input, intrinsic noise, synaptic efficacy, density of extrinsic connections, the voltage-sensitive nature of postsynaptic mechanisms, the number of neurons, and the laminar structure within the populations, we were able to introduce variations in both mean activity and synchronization under a variety of simulated neuronal architectures. Analyses of the simulated spike trains and local field potentials showed that in nearly every domain of the model's parameter space, mean activity and synchronization were tightly coupled. This coupling appears to be mediated by an increase in synchronous gain when effective membrane time constants are lowered by increased activity. These observations show that under the assumptions implicit in our models, rate coding and synchrony coding in neural systems with reciprocal interconnections are two perspectives on the same underlying dynamic. This suggests that in the absence of specific mechanisms decoupling changes in synchronization from firing levels, indexes of brain activity that are based purely on synaptic activity (e.g., functional magnetic resonance imaging) may also be sensitive to changes in synchronous coupling.     

Estimating spiking irregularities under changing environments

We considered a gamma distribution of interspike intervals as a statistical model for neuronal spike generation. A gamma distribution is a natural extension of the Poisson process taking the effect of a refractory period into account. The model is specified by two parameters: a time-dependent firing rate and a shape parameter that characterizes spiking irregularities of individual neurons. Because the environment changes over time, observed data are generated from a model with a time-dependent firing rate, which is an unknown function. A statistical model with an unknown function is called a semiparametric model and is generally very difficult to solve. We used a novel method of estimating functions in information geometry to estimate the shape parameter without estimating the unknown function. We obtained an optimal estimating function analytically for the shape parameter independent of the functional form of the firing rate. This estimation is efficient without Fisher information loss and better than maximum likelihood estimation. We suggest a measure of spiking irregularity based on the estimating function, which may be useful for characterizing individual neurons in changing environments.     

Population density methods for large-scale modelling of neuronal networks with realistic synaptic kinetics: cutting the dimension down to size

Population density methods provide promising time-saving alternatives to direct Monte Carlo simulations of neuronal network activity, in which one tracks the state of thousands of individual neurons and synapses. A population density method has been found to be roughly a hundred times faster than direct simulation for various test networks of integrate-and-fire model neurons with instantaneous excitatory and inhibitory post-synaptic conductances. In this method, neurons are grouped into large populations of similar neurons. For each population, one calculates the evolution of a probability density function (PDF) which describes the distribution of neurons over state space. The population firing rate is then given by the total flux of probability across the threshold voltage for firing an action potential. Extending the method beyond instantaneous synapses is necessary for obtaining accurate results, because synaptic kinetics play an important role in network dynamics. Embellishments incorporating more realistic synaptic kinetics for the underlying neuron model increase the dimension of the PDF, which was one-dimensional in the instantaneous synapse case. This increase in dimension causes a substantial increase in computation time to find the exact PDF, decreasing the computational speed advantage of the population density method over direct Monte Carlo simulation. We report here on a one-dimensional model of the PDF for neurons with arbitrary synaptic kinetics. The method is more accurate than the mean-field method in the steady state, where the mean-field approximation works best, and also under dynamic-stimulus conditions. The method is much faster than direct simulations. Limitations of the method are demonstrated, and possible improvements are discussed.     

Minimal models of adapted neuronal response to in vivo-like input currents

Rate models are often used to study the behavior of large networks of spiking neurons. Here we propose a procedure to derive rate models that take into account the fluctuations of the input current and firing-rate adaptation, two ubiquitous features in the central nervous system that have been previously overlooked in constructing rate models. The procedure is general and applies to any model of firing unit. As examples, we apply it to the leaky integrate-and-fire (IF) neuron, the leaky IF neuron with reversal potentials, and to the quadratic IF neuron. Two mechanisms of adaptation are considered, one due to an afterhyperpolarization current and the other to an adapting threshold for spike emission. The parameters of these simple models can be tuned to match experimental data obtained from neocortical pyramidal neurons. Finally, we show how the stationary model can be used to predict the time-varying activity of a large population of adapting neurons.     

Energy function and energy evolution on neuronal populations.

Based on the principle of energy coding, an energy function of a variety of electric potentials of a neural population in cerebral cortex is formulated. The energy function is used to describe the energy evolution of the neuronal population with time and the coupled relationship between neurons at the subthreshold and the suprathreshold states. The Hamiltonian motion equation with the membrane potential is obtained from the neuroelectrophysiological data contaminated by Gaussian white noise. The results of this research show that the mean membrane potential is the exact solution of the motion equation of the membrane potential developed in a previously published paper. It also shows that the Hamiltonian energy function derived in this brief is not only correct but also effective. Particularly, based on the principle of energy coding, an interesting finding is that in some subsets of neurons, firing action potentials at the suprathreshold and some others simultaneously perform activities at the subthreshold level in neural ensembles. Notably, this kind of coupling has not been found in other models of biological neural networks.     

Theory of input spike auto- and cross-correlations and their effect on the response of spiking neurons

Spike correlations between neurons are ubiquitous in the cortex, but their role is not understood. Here we describe the firing response of a leaky integrate-and-fire neuron (LIF) when it receives a temporarily correlated input generated by presynaptic correlated neuronal populations. Input correlations are characterized in terms of the firing rates, Fano factors, correlation coefficients, and correlation timescale of the neurons driving the target neuron. We show that the sum of the presynaptic spike trains cannot be well described by a Poisson process. In fact, the total input current has a nontrivial two-point correlation function described by two main parameters: the correlation timescale (how precise the input correlations are in time) and the correlation magnitude (how strong they are). Therefore, the total current generated by the input spike trains is not well described by a white noise gaussian process. Instead, we model the total current as a colored gaussian process with the same mean and two-point correlation function, leading to the formulation of the problem in terms of a Fokker-Planck equation. Solutions of the output firing rate are found in the limit of short and long correlation timescales. The solutions described here expand and improve on our previous results (Moreno, de la Rocha, Renart, & Parga, 2002) by presenting new analytical expressions for the output firing rate for general IF neurons, extending the validity of the results for arbitrarily large correlation magnitude, and by describing the differential effect of correlations on the mean-driven or noise-dominated firing regimes. Also the details of this novel formalism are given here for the first time. We employ numerical simulations to confirm the analytical solutions and study the firing response to sudden changes in the input correlations. We expect this formalism to be useful for the study of correlations in neuronal networks and their role in neural processing and information transmission.     

Optimal neural rate coding leads to bimodal firing rate distributions

Many experimental studies concerning the neuronal code are based on graded responses of neurons, given by the emitted number of spikes measured in a certain time window. Correspondingly, a large body of neural network theory deals with analogue neuron models and discusses their potential use for computation or function approximation. All physical signals, however, are of limited precision, and neuronal firing rates in cortex are relatively low. Here, we investigate the relevance of analogue signal processing with spikes in terms of optimal stimulus reconstruction and information theory. In particular, we derive optimal tuning functions taking the biological constraint of limited firing rates into account. It turns out that depending on the available decoding time T, optimal encoding undergoes a phase transition from discrete binary coding for small T towards analogue or quasi-analogue encoding for large T. The corresponding firing rate distributions are bimodal for all relevant T, in particular in the case of population coding.     

Implications of neuronal diversity on population coding

In many cortical and subcortical areas, neurons are known to modulate their average firing rate in response to certain external stimulus features. It is widely believed that information about the stimulus features is coded by a weighted average of the neural responses. Recent theoretical studies have shown that the information capacity of such a coding scheme is very limited in the presence of the experimentally observed pairwise correlations. However, central to the analysis of these studies was the assumption of a homogeneous population of neurons. Experimental findings show a considerable measure of heterogeneity in the response properties of different neurons. In this study, we investigate the effect of neuronal heterogeneity on the information capacity of a correlated population of neurons. We show that information capacity of a heterogeneous network is not limited by the correlated noise, but scales linearly with the number of cells in the population. This information cannot be extracted by the population vector readout, whose accuracy is greatly suppressed by the correlated noise. On the other hand, we show that an optimal linear readout that takes into account the neuronal heterogeneity can extract most of this information. We study analytically the nature of the dependence of the optimal linear readout weights on the neuronal diversity. We show that simple online learning can generate readout weights with the appropriate dependence on the neuronal diversity, thereby yielding efficient readout.     

Gaussian processes and neuronal modeling

This work is a contribution towards the understanding of certain features of mathematical models of single neurons. Emphasis is set on neuronal firing, for which the first passage time (FPT) problem bears a fundamental relevance. We focus the attention on modeling the change of the neuron membrane potential between two consecutive spikes by Gaussian stochastic processes, both of Markov and of non-Markov types. Methods to solve the FPT problems, both of a theoretical and of a computational nature, are sketched, including the case of random initial values. Significant similarities or diversities between computational and theoretical results are pointed out, disclosing the role played by the correlation time that has been used to characterize the neuronal activity. It is highlighted that any conclusion on this matter is strongly model-dependent. In conclusion, an outline of the asymptotic behavior of FPT densities is provided, which is particularly useful to discuss neuronal firing under certain slow activity conditions.     

Synchronous firing and higher-order interactions in neuron pool

The stochastic mechanism of synchronous firing in a population of neurons is studied from the point of view of information geometry. Higher-order interactions of neurons, which cannot be reduced to pairwise correlations, are proved to exist in synchronous firing. In a neuron pool where each neuron fires stochastically, the probability distribution q(r) of the activity r, which is the fraction of firing neurons in the pool, is studied. When q(r) has a widespread distribution, in particular, when q(r) has two peaks, the neurons fire synchronously at one time and are quiescent at other times. The mechanism of generating such a probability distribution is interesting because the activity r is concentrated on its mean value when each neuron fires independently, because of the law of large numbers. Even when pairwise interactions, or third-order interactions, exist, the concentration is not resolved. This shows that higher-order interactions are necessary to generate widespread activity distributions. We analyze a simple model in which neurons receive common overlapping inputs and prove that such a model can have a widespread distribution of activity, generating higher-order stochastic interactions.     

An integrated fuzzy regression-data envelopment analysis algorithm for optimum oil consumption estimation with ambiguous data

This study introduces an integrated fuzzy regression (FR) data envelopment analysis (DEA) algorithm for oil consumption estimation and optimization with uncertain and ambiguous data. This is quite important as oil consumption estimations deals with several uncertainties due to social, economic factors. Furthermore, DEA is integrated with FR because there is no clear cut as to which FR approach is superior for oil consumption estimation. The standard indicators used in this paper are population, cost of crude oil, gross domestic production (GDP) and annual oil production. Fifteen popular and most cited FR models are considered in the algorithm. Each FR model has different approach and advantages. The input data is divided into train and test data. The FR models have been tuned for all their parameters according to the train data, and the best coefficients are identified. Center of Average Method for defuzzification output process is applied. For determining the rate of error of FR models estimations, the rate of defuzzified output of each model is compared with its actual rate consumption in test data. The efficiency of 15 FR models is examined by the output-oriented Data Envelopment Analysis (DEA) model without inputs by considering three types of relative error: RMSE, MAE and MAPE. The applicability and superiority of the proposed algorithm is shown for monthly oil consumption of Canada, United States, Japan and Australia from 1990 to 2005.     

Replicating receptive fields of simple and complex cells in primary visual cortex in a neuronal network model with temporal and population sparseness and reliability

We propose a new principle for replicating receptive field properties of neurons in the primary visual cortex. We derive a learning rule for a feedforward network, which maintains a low firing rate for the output neurons (resulting in temporal sparseness) and allows only a small subset of the neurons in the network to fire at any given time (resulting in population sparseness). Our learning rule also sets the firing rates of the output neurons at each time step to near-maximum or near-minimum levels, resulting in neuronal reliability. The learning rule is simple enough to be written in spatially and temporally local forms. After the learning stage is performed using input image patches of natural scenes, output neurons in the model network are found to exhibit simple-cell-like receptive field properties. When the output of these simple-cell-like neurons are input to another model layer using the same learning rule, the second-layer output neurons after learning become less sensitive to the phase of gratings than the simple-cell-like input neurons. In particular, some of the second-layer output neurons become completely phase invariant, owing to the convergence of the connections from first-layer neurons with similar orientation selectivity to second-layer neurons in the model network. We examine the parameter dependencies of the receptive field properties of the model neurons after learning and discuss their biological implications. We also show that the localized learning rule is consistent with experimental results concerning neuronal plasticity and can replicate the receptive fields of simple and complex cells.     

A population density approach that facilitates large-scale modeling of neural networks: extension to slow inhibitory synapses

A previously developed method for efficiently simulating complex networks of integrate-and-fire neurons was specialized to the case in which the neurons have fast unitary postsynaptic conductances. However, inhibitory synaptic conductances are often slower than excitatory ones for cortical neurons, and this difference can have a profound effect on network dynamics that cannot be captured with neurons that have only fast synapses. We thus extend the model to include slow inhibitory synapses. In this model, neurons are grouped into large populations of similar neurons. For each population, we calculate the evolution of a probability density function (PDF), which describes the distribution of neurons over state-space. The population firing rate is given by the flux of probability across the threshold voltage for firing an action potential. In the case of fast synaptic conductances, the PDF was one-dimensional, as the state of a neuron was completely determined by its transmembrane voltage. An exact extension to slow inhibitory synapses increases the dimension of the PDF to two or three, as the state of a neuron now includes the state of its inhibitory synaptic conductance. However, by assuming that the expected value of a neuron's inhibitory conductance is independent of its voltage, we derive a reduction to a one-dimensional PDF and avoid increasing the computational complexity of the problem. We demonstrate that although this assumption is not strictly valid, the results of the reduced model are surprisingly accurate.