On embedding synfire chains in a balanced network

We investigate the formation of synfire waves in a balanced network of integrate-and-fire neurons. The synaptic connectivity of this network embodies synfire chains within a sparse random connectivity. This network can exhibit global oscillations but can also operate in an asynchronous activity mode. We analyze the correlations of two neurons in a pool as convenient indicators for the state of the network. We find, using different models, that these indicators depend on a scaling variable. Beyond a critical point, strong correlations and large network oscillations are obtained. We looked for the conditions under which a synfire wave could be propagated on top of an otherwise asynchronous state of the network. This condition was found to be highly restrictive, requiring a large number of neurons for its implementation in our network. The results are based on analytic derivations and simulations.     

Patterns of synchrony in neural networks with spike adaptation

We study the emergence of synchronized burst activity in networks of neurons with spike adaptation. We show that networks of tonically firing adapting excitatory neurons can evolve to a state where the neurons burst in a synchronized manner. The mechanism leading to this burst activity is analyzed in a network of integrate-and-fire neurons with spike adaptation. The dependence of this state on the different network parameters is investigated, and it is shown that this mechanism is robust against inhomogeneities, sparseness of the connectivity, and noise. In networks of two populations, one excitatory and one inhibitory, we show that decreasing the inhibitory feedback can cause the network to switch from a tonically active, asynchronous state to the synchronized bursting state. Finally, we show that the same mechanism also causes synchronized burst activity in networks of more realistic conductance-based model neurons.     

The number of synaptic inputs and the synchrony of large, sparse neuronal networks

The prevalence of coherent oscillations in various frequency ranges in the central nervous system raises the question of the mechanisms that synchronize large populations of neurons. We study synchronization in models of large networks of spiking neurons with random sparse connectivity. Synchrony occurs only when the average number of synapses, M, that a cell receives is larger than a critical value, Mc. Below Mc, the system is in an asynchronous state. In the limit of weak coupling, assuming identical neurons, we reduce the model to a system of phase oscillators that are coupled via an effective interaction, gamma. In this framework, we develop an approximate theory for sparse networks of identical neurons to estimate Mc analytically from the Fourier coefficients of gamma. Our approach relies on the assumption that the dynamics of a neuron depend mainly on the number of cells that are presynaptic to it. We apply this theory to compute Mc for a model of inhibitory networks of integrate-and-fire (I&F) neurons as a function of the intrinsic neuronal properties (e.g., the refractory period Tr), the synaptic time constants, and the strength of the external stimulus, Iext. The number Mc is found to be nonmonotonous with the strength of Iext. For Tr = 0, we estimate the minimum value of Mc over all the parameters of the model to be 363.8. Above Mc, the neurons tend to fire in smeared one-cluster states at high firing rates and smeared two-or-more-cluster states at low firing rates. Refractoriness decreases Mc at intermediate and high firing rates. These results are compared to numerical simulations. We show numerically that systems with different sizes, N, behave in the same way provided the connectivity, M, is such that 1/Meff = 1/M - 1/N remains constant when N varies. This allows extrapolating the large N behavior of a network from numerical simulations of networks of relatively small sizes (N = 800 in our case). We find that our theory predicts with remarkable accuracy the value of Mc and the patterns of synchrony above Mc, provided the synaptic coupling is not too large. We also study the strong coupling regime of inhibitory sparse networks. All of our simulations demonstrate that increasing the coupling strength reduces the level of synchrony of the neuronal activity. Above a critical coupling strength, the network activity is asynchronous. We point out a fundamental limitation for the mechanisms of synchrony relying on inhibition alone, if heterogeneities in the intrinsic properties of the neurons and spatial fluctuations in the external input are also taken into account.     

The high-conductance state of cortical networks

We studied the dynamics of large networks of spiking neurons with conductance-based (nonlinear) synapses and compared them to networks with current-based (linear) synapses. For systems with sparse and inhibition-dominated recurrent connectivity, weak external inputs induced asynchronous irregular firing at low rates. Membrane potentials fluctuated a few millivolts below threshold, and membrane conductances were increased by a factor 2 to 5 with respect to the resting state. This combination of parameters characterizes the ongoing spiking activity typically recorded in the cortex in vivo. Many aspects of the asynchronous irregular state in conductance-based networks could be sufficiently well characterized with a simple numerical mean field approach. In particular, it correctly predicted an intriguing property of conductance-based networks that does not appear to be shared by current-based models: they exhibit states of low-rate asynchronous irregular activity that persist for some period of time even in the absence of external inputs and without cortical pacemakers. Simulations of larger networks (up to 350,000 neurons) demonstrated that the survival time of self-sustained activity increases exponentially with network size.     

Spike-timing-dependent plasticity in balanced random networks

The balanced random network model attracts considerable interest because it explains the irregular spiking activity at low rates and large membrane potential fluctuations exhibited by cortical neurons in vivo. In this article, we investigate to what extent this model is also compatible with the experimentally observed phenomenon of spike-timing-dependent plasticity (STDP). Confronted with the plethora of theoretical models for STDP available, we reexamine the experimental data. On this basis, we propose a novel STDP update rule, with a multiplicative dependence on the synaptic weight for depression, and a power law dependence for potentiation. We show that this rule, when implemented in large, balanced networks of realistic connectivity and sparseness, is compatible with the asynchronous irregular activity regime. The resultant equilibrium weight distribution is unimodal with fluctuating individual weight trajectories and does not exhibit development of structure. We investigate the robustness of our results with respect to the relative strength of depression. We introduce synchronous stimulation to a group of neurons and demonstrate that the decoupling of this group from the rest of the network is so severe that it cannot effectively control the spiking of other neurons, even those with the highest convergence from this group.     

A systematic approach to constructing incremental topology control algorithms using graph transformation

Communication networks form the backbone of our society. Topology control algorithms optimize the topology of such communication networks. Due to the importance of communication networks, a topology control algorithm should guarantee certain required consistency properties (e.g., connectivity of the topology), while achieving desired optimization properties (e.g., a bounded number of neighbors). Real-world topologies are dynamic (e.g., because nodes join, leave, or move within the network), which requires topology control algorithms to operate in an incremental way, i.e., based on the recently introduced modifications of a topology. Visual programming and specification languages are a proven means for specifying the structure as well as consistency and optimization properties of topologies. In this paper, we present a novel methodology, based on a visual graph transformation and graph constraint language, for developing incremental topology control algorithms that are guaranteed to fulfill a set of specified consistency and optimization constraints. More specifically, we model the possible modifications of a topology control algorithm and the environment using graph transformation rules, and we describe consistency and optimization properties using graph constraints. On this basis, we apply and extend a well-known constructive approach to derive refined graph transformation rules that preserve these graph constraints. We apply our methodology to re-engineer an established topology control algorithm, kTC, and evaluate it in a network simulation study to show the practical applicability of our approach.     

Event-driven simulation scheme for spiking neural networks using lookup tables to characterize neuronal dynamics

Nearly all neuronal information processing and interneuronal communication in the brain involves action potentials, or spikes, which drive the short-term synaptic dynamics of neurons, but also their long-term dynamics, via synaptic plasticity. In many brain structures, action potential activity is considered to be sparse. This sparseness of activity has been exploited to reduce the computational cost of large-scale network simulations, through the development of event-driven simulation schemes. However, existing event-driven simulations schemes use extremely simplified neuronal models. Here, we implement and evaluate critically an event-driven algorithm (ED-LUT) that uses precalculated look-up tables to characterize synaptic and neuronal dynamics. This approach enables the use of more complex (and realistic) neuronal models or data in representing the neurons, while retaining the advantage of high-speed simulation. We demonstrate the method's application for neurons containing exponential synaptic conductances, thereby implementing shunting inhibition, a phenomenon that is critical to cellular computation. We also introduce an improved two-stage event-queue algorithm, which allows the simulations to scale efficiently to highly connected networks with arbitrary propagation delays. Finally, the scheme readily accommodates implementation of synaptic plasticity mechanisms that depend on spike timing, enabling future simulations to explore issues of long-term learning and adaptation in large-scale networks.     

Interactions of excitatory and inhibitory feedback topologies in facilitating pattern separation and retrieval

Within the brain, the interplay between connectivity patterns of neurons and their spatiotemporal dynamics is believed to be intricately linked to the bases of behavior, such as the process of storing, consolidating, and retrieving memory traces. Memory is believed to be stored in the synaptic patterns of anatomical circuitry in the form of increased connectivity densities within subpopulations of neurons. At the same time, memory recall is thought to correspond to activation of discrete areas of the brain corresponding to those memories. Such regional subpopulations can selectively activate during memory recall or retrieval, signifying the process of accessing a single memory or concept. It has been shown previously that recovery of single memory activity patterns is mediated by global neuromodulation signifying transition into different cognitive states such as sleep or awake exploration. We examine how underlying topology can affect memory awake activation and sleep reactivation when such memories share increasing proportions of neurons. The results show that while single memory activation is diminished with increased overlap, pattern separation can be recovered by offsetting excitatory associations between two memories with targeted and heterogeneous inhibitory feedback. Such findings point to the importance of excitatory-to-inhibitory current balance at both the global and local levels in the context of memory retrieval and replay, and highlight the role of network topology in memory management processes.     

Spike-timing-dependent plasticity in small-world networks

Biologically plausible excitatory neural networks develop a persistent synchronized pattern of activity depending on spontaneous activity and synaptic refractoriness (short term depression). By fixed synaptic weights synchronous bursts of oscillatory activity are stable and involve the whole network. In our modeling study we investigate the effect of a dynamic Hebbian-like learning mechanism, spike-timing-dependent plasticity (STDP), on the changes of synaptic weights depending on synchronous activity and network connection strategies (small-world topology). We show that STDP modifies the weights of synaptic connections in such a way that synchronization of neuronal activity is considerably weakened. Networks with a higher proportion of long connections can sustain a higher level of synchronization in spite of STDP influence. The resulting distribution of the synaptic weights in single neurons depends both on the global statistics of firing dynamics and on the number of incoming and outgoing connections.     

Delay learning and polychronization for reservoir computing

We propose a multi-timescale learning rule for spiking neuron networks, in the line of the recently emerging field of reservoir computing. The reservoir is a network model of spiking neurons, with random topology and driven by STDP (spike-time-dependent plasticity), a temporal Hebbian unsupervised learning mode, biologically observed. The model is further driven by a supervised learning algorithm, based on a margin criterion, that affects the synaptic delays linking the network to the readout neurons, with classification as a goal task. The network processing and the resulting performance can be explained by the concept of polychronization, proposed by Izhikevich [Polychronization: computation with spikes, Neural Comput. 18(2) (2006) 245–282], on physiological grounds. The model emphasizes that polychronization can be used as a tool for exploiting the computational power of synaptic delays and for monitoring the topology and activity of a spiking neuron network.     

Correlations and population dynamics in cortical networks

The function of cortical networks depends on the collective interplay between neurons and neuronal populations, which is reflected in the correlation of signals that can be recorded at different levels. To correctly interpret these observations it is important to understand the origin of neuronal correlations. Here we study how cells in large recurrent networks of excitatory and inhibitory neurons interact and how the associated correlations affect stationary states of idle network activity. We demonstrate that the structure of the connectivity matrix of such networks induces considerable correlations between synaptic currents as well as between subthreshold membrane potentials, provided Dale's principle is respected. If, in contrast, synaptic weights are randomly distributed, input correlations can vanish, even for densely connected networks. Although correlations are strongly attenuated when proceeding from membrane potentials to action potentials (spikes), the resulting weak correlations in the spike output can cause substantial fluctuations in the population activity, even in highly diluted networks. We show that simple mean-field models that take the structure of the coupling matrix into account can adequately describe the power spectra of the population activity. The consequences of Dale's principle on correlations and rate fluctuations are discussed in the light of recent experimental findings.     

Design of continuous attractor networks with monotonic tuning using a symmetry principle

Neurons that sustain elevated firing in the absence of stimuli have been found in many neural systems. In graded persistent activity, neurons can sustain firing at many levels, suggesting a widely found type of network dynamics in which networks can relax to any one of a continuum of stationary states. The reproduction of these findings in model networks of nonlinear neurons has turned out to be nontrivial. A particularly insightful model has been the "bump attractor," in which a continuous attractor emerges through an underlying symmetry in the network connectivity matrix. This model, however, cannot account for data in which the persistent firing of neurons is a monotonic -- rather than a bell-shaped -- function of a stored variable. Here, we show that the symmetry used in the bump attractor network can be employed to create a whole family of continuous attractor networks, including those with monotonic tuning. Our design is based on tuning the external inputs to networks that have a connectivity matrix with Toeplitz symmetry. In particular, we provide a complete analytical solution of a line attractor network with monotonic tuning and show that for many other networks, the numerical tuning of synaptic weights reduces to the computation of a single parameter.     

Bursting without slow kinetics: a role for a small world?

Bursting, a dynamical phenomenon whereby episodes of neural action potentials are punctuated by periodic episodes of inactivity, is ubiquitous in neural systems. Examples include components of the respiratory rhythm generating circuitry in the brain stem, spontaneous activity in the neonatal rat spinal cord, and developing neural networks in the retina of the immature ferret. Bursting can also manifest itself in single neurons. Bursting dynamics require one or more kinetic processes slower than the timescale of the action potentials. Such processes usually manifest themselves in intrinsic ion channel properties, such as slow voltage-dependent gating or calcium-dependent processes, or synaptic mechanisms, such as synaptic depression. In this note, we show rhythmic bursting in a simulated neural network where no such slow processes exist at the cellular or synaptic level. Rather, the existence of rhythmic bursting is critically dependent on the connectivity of the network and manifests itself only when connectivity is characterized as small world. The slow process underlying the timescale of bursting manifests itself as a progressive synchronization of the network within each burst.     

Rapid temporal modulation of synchrony by competition in cortical interneuron networks

The synchrony of neurons in extrastriate visual cortex is modulated by selective attention even when there are only small changes in firing rate (Fries, Reynolds, Rorie, & Desimone, 2001). We used Hodgkin-Huxley type models of cortical neurons to investigate the mechanism by which the degree of synchrony can be modulated independently of changes in firing rates. The synchrony of local networks of model cortical interneurons interacting through GABA(A) synapses was modulated on a fast timescale by selectively activating a fraction of the interneurons. The activated interneurons became rapidly synchronized and suppressed the activity of the other neurons in the network but only if the network was in a restricted range of balanced synaptic background activity. During stronger background activity, the network did not synchronize, and for weaker background activity, the network synchronized but did not return to an asynchronous state after synchronizing. The inhibitory output of the network blocked the activity of pyramidal neurons during asynchronous network activity, and during synchronous network activity, it enhanced the impact of the stimulus-related activity of pyramidal cells on receiving cortical areas (Salinas & Sejnowski, 2001). Synchrony by competition provides a mechanism for controlling synchrony with minor alterations in rate, which could be useful for information processing. Because traditional methods such as cross-correlation and the spike field coherence require several hundred milliseconds of recordings and cannot measure rapid changes in the degree of synchrony, we introduced a new method to detect rapid changes in the degree of coincidence and precision of spike timing.     

Potential Fields for Maintaining Connectivity of Mobile Networks

The control of mobile networks of multiple agents raises fundamental and novel problems in controlling the structure of the resulting dynamic graphs. In this paper, we consider the problem of controlling a network of agents so that the resulting motion always preserves the connectivity property of the network. In particular, the connectivity condition is translated to differentiable constraints on individual agent motion by considering the dynamics of the Laplacian matrix and its spectral properties. Artificial potential fields are then used to drive the agents to configurations away from the undesired space of disconnected networks while avoiding collisions with each other. We conclude by illustrating a class of interesting problems that can be achieved while preserving connectivity constraints.     

A framework for simulating and estimating the state and functional topology of complex dynamic geometric networks

We introduce a framework for simulating signal propagation in geometric networks (networks that can be mapped to geometric graphs in some space) and developing algorithms that estimate (i.e., map) the state and functional topology of complex dynamic geometric networks. Within the framework, we define the key features typically present in such networks and of particular relevance to biological cellular neural networks: dynamics, signaling, observation, and control. The framework is particularly well suited for estimating functional connectivity in cellular neural networks from experimentally observable data and has been implemented using graphics processing unit high-performance computing. Computationally, the framework can simulate cellular network signaling close to or faster than real time. We further propose a standard test set of networks to measure performance and compare different mapping algorithms.     

GeoSpray: A geographic routing protocol for vehicular delay-tolerant networks

Vehicular networks are characterized by a highly dynamic network topology, and disruptive and intermittent connectivity. In such network environments, a complete path from source to destination does not exist on the most part of the time. Vehicular delay-tolerant network (VDTN) architecture was introduced to deal with these connectivity constraints. VDTN assumes asynchronous, bundle-oriented communication, and a store-carry-and-forward routing paradigm. A routing protocol for VDTNs should make the best use of the tight resources available in network nodes to create a multi-hop path that exists over time. This paper proposes a VDTN routing protocol, called GeoSpray, which takes routing decisions based on geographical location data, and combines a hybrid approach between multiple-copy and single-copy schemes. First, it starts with a multiple-copy scheme, spreading a limited number of bundle copies, in order to exploit alternative paths. Then, it switches to a forwarding scheme, which takes advantage of additional contact opportunities. In order to improve resources utilization, it clears delivered bundles across the network nodes. It is shown that GeoSpray improves significantly the delivery probability and reduces the delivery delay, compared to traditional location and non location-based single-copy and multiple-copy routing protocols.     

Advancing the boundaries of high-connectivity network simulation with distributed computing

The availability of efficient and reliable simulation tools is one of the mission-critical technologies in the fast-moving field of computational neuroscience. Research indicates that higher brain functions emerge from large and complex cortical networks and their interactions. The large number of elements (neurons) combined with the high connectivity (synapses) of the biological network and the specific type of interactions impose severe constraints on the explorable system size that previously have been hard to overcome. Here we present a collection of new techniques combined to a coherent simulation tool removing the fundamental obstacle in the computational study of biological neural networks: the enormous number of synaptic contacts per neuron. Distributing an individual simulation over multiple computers enables the investigation of networks orders of magnitude larger than previously possible. The software scales excellently on a wide range of tested hardware, so it can be used in an interactive and iterative fashion for the development of ideas, and results can be produced quickly even for very large networks. In contrast to earlier approaches, a wide class of neuron models and synaptic dynamics can be represented.     

Modeling alternation to synchrony with inhibitory coupling: a neuromorphic VLSI approach

We developed an analog very large-scale integrated system of two mutually inhibitory silicon neurons that display several different stable oscillations. For example, oscillations can be synchronous with weak inhibitory coupling and alternating with relatively strong inhibitory coupling. All oscillations observed experimentally were predicted by bifurcation analysis of a corresponding mathematical model. The synchronous oscillations do not require special synaptic properties and are apparently robust enough to survive the variability and constraints inherent in this physical system. In biological experiments with oscillatory neuronal networks, blockade of inhibitory synaptic coupling can sometimes lead to synchronous oscillations. An example of this phenomenon is the transition from alternating to synchronous bursting in the swimming central pattern generator of lamprey when synaptic inhibition is blocked by strychnine. Our results suggest a simple explanation for the observed oscillatory transitions in the lamprey central pattern generator network: that inhibitory connectivity alone is sufficient to produce the observed transition.