Exact subthreshold integration with continuous spike times in discrete-time neural network simulations

Very large networks of spiking neurons can be simulated efficiently in parallel under the constraint that spike times are bound to an equidistant time grid. Within this scheme, the subthreshold dynamics of a wide class of integrate-and-fire-type neuron models can be integrated exactly from one grid point to the next. However, the loss in accuracy caused by restricting spike times to the grid can have undesirable consequences, which has led to interest in interpolating spike times between the grid points to retrieve an adequate representation of network dynamics. We demonstrate that the exact integration scheme can be combined naturally with off-grid spike events found by interpolation. We show that by exploiting the existence of a minimal synaptic propagation delay, the need for a central event queue is removed, so that the precision of event-driven simulation on the level of single neurons is combined with the efficiency of time-driven global scheduling. Further, for neuron models with linear subthreshold dynamics, even local event queuing can be avoided, resulting in much greater efficiency on the single-neuron level. These ideas are exemplified by two implementations of a widely used neuron model. We present a measure for the efficiency of network simulations in terms of their integration error and show that for a wide range of input spike rates, the novel techniques we present are both more accurate and faster than standard techniques.     

Cluster integration method in Lagrangian particle dynamics

An efficient and robust approach is proposed in order to conduct numerical simulations of collisional particle dynamics in the Lagrangian framework. Clusters of particles are made of particles that interact or may interact during the next global time-step. Potential collision partners are found by performing a test move, that follows the patterns of a hard-sphere model. The clusters are integrated separately and the collisional forces between particles are given by a soft-sphere collision model. However, the present approach also allows longer range inter-particle forces. The integration of the clusters can be done by any one-step ordinary differential equation solver, but for dilute particle systems, the variable step-size Runge-Kutta solvers as the Dormand and Prince scheme [J. Comput. Appl. Math. 6 (1980) 19] are superior. The cluster integration method is applied on sedimentation of 5000 particles in a two-dimensional box. A significant speed-up is achieved. Compared to a traditional discrete element method with the forward Euler scheme, a speed-up factor of three orders of magnitude in the dilute regime and two orders of magnitude in the dense regime were observed. As long as the particles are dilute, the Dormand and Prince scheme is ten times faster than the classical fourth-order Runge-Kutta solver with fixed step size.     

Time-accurate Navier-Stokes simulation of vortex convection using an unstructured dynamic mesh procedure

A two-dimensional Navier-Stokes flow solver is developed for the simulation of unsteady flows on unstructured adaptive meshes. The solver is based on a second-order accurate implicit time integration using a point Gauss-Seidel relaxation scheme and a dual time-step subiteration. A vertex-centered, finite-volume discretization is used in conjunction with Roe’s flux-difference splitting. The Spalart-Allmaras one equation model is employed for the simulation of turbulence. An unsteady solution-adaptive dynamic mesh scheme is used by adding and deleting mesh points to take account of spatial and temporal variations of the flowfield. Unsteady viscous flow for a traveling vortex in a free stream is simulated to validate the accuracy of the dynamic mesh adaptation procedure. Flow around a circular cylinder and two blade-vortex interaction problems are investigated for demonstration of the present method. Computed results show good agreement with existing experimental and computational results. It was found that unsteady time-accurate viscous flows can be accurately simulated using the present unstructured dynamic mesh adaptation procedure.     

Analytical integrate-and-fire neuron models with conductance-based dynamics for event-driven simulation strategies

Event-driven simulation strategies were proposed recently to simulate integrate-and-fire (IF) type neuronal models. These strategies can lead to computationally efficient algorithms for simulating large-scale networks of neurons; most important, such approaches are more precise than traditional clock-driven numerical integration approaches because the timing of spikes is treated exactly. The drawback of such event-driven methods is that in order to be efficient, the membrane equations must be solvable analytically, or at least provide simple analytic approximations for the state variables describing the system. This requirement prevents, in general, the use of conductance-based synaptic interactions within the framework of event-driven simulations and, thus, the investigation of network paradigms where synaptic conductances are important. We propose here a number of extensions of the classical leaky IF neuron model involving approximations of the membrane equation with conductance-based synaptic current, which lead to simple analytic expressions for the membrane state, and therefore can be used in the event-driven framework. These conductance-based IF (gIF) models are compared to commonly used models, such as the leaky IF model or biophysical models in which conductances are explicitly integrated. All models are compared with respect to various spiking response properties in the presence of synaptic activity, such as the spontaneous discharge statistics, the temporal precision in resolving synaptic inputs, and gain modulation under in vivo-like synaptic bombardment. Being based on the passive membrane equation with fixed-threshold spike generation, the proposed gIF models are situated in between leaky IF and biophysical models but are much closer to the latter with respect to their dynamic behavior and response characteristics, while still being nearly as computationally efficient as simple IF neuron models. gIF models should therefore provide a useful tool for efficient and precise simulation of large-scale neuronal networks with realistic, conductance-based synaptic interactions.     

On the performance of voltage stepping for the simulation of adaptive, nonlinear integrate-and-fire neuronal networks

In traditional event-driven strategies, spike timings are analytically given or calculated with arbitrary precision (up to machine precision). Exact computation is possible only for simplified neuron models, mainly the leaky integrate-and-fire model. In a recent paper, Zheng, Tonnelier, and Martinez (2009) introduced an approximate event-driven strategy, named voltage stepping, that allows the generic simulation of nonlinear spiking neurons. Promising results were achieved in the simulation of single quadratic integrate-and-fire neurons. Here, we assess the performance of voltage stepping in network simulations by considering more complex neurons (quadratic integrate-and-fire neurons with adaptation) coupled with multiple synapses. To handle the discrete nature of synaptic interactions, we recast voltage stepping in a general framework, the discrete event system specification. The efficiency of the method is assessed through simulations and comparisons with a modified time-stepping scheme of the Runge-Kutta type. We demonstrated numerically that the original order of voltage stepping is preserved when simulating connected spiking neurons, independent of the network activity and connectivity.     

A generalized linear integrate-and-fire neural model produces diverse spiking behaviors

For simulations of neural networks, there is a trade-off between the size of the network that can be simulated and the complexity of the model used for individual neurons. In this study, we describe a generalization of the leaky integrate-and-fire model that produces a wide variety of spiking behaviors while still being analytically solvable between firings. For different parameter values, the model produces spiking or bursting, tonic, phasic or adapting responses, depolarizing or hyperpolarizing after potentials and so forth. The model consists of a diagonalizable set of linear differential equations describing the time evolution of membrane potential, a variable threshold, and an arbitrary number of firing-induced currents. Each of these variables is modified by an update rule when the potential reaches threshold. The variables used are intuitive and have biological significance. The model's rich behavior does not come from the differential equations, which are linear, but rather from complex update rules. This single-neuron model can be implemented using algorithms similar to the standard integrate-and-fire model. It is a natural match with event-driven algorithms for which the firing times are obtained as a solution of a polynomial equation.     

A comparative evaluation of DRE integration algorithms for real-time simulation of biologically activated sludge process

An activated sludge process is considered in this work for comparative tests of new integration algorithms. Based on the configuration of the process and on the process kinetics for heterotrophic bacterial growth, the mathematical model of the considered process has been derived in the form of a state ordinary differential equation system. The state ordinary differential equation system describing the considered process may be both stiff and non-stiff for operator's control changes of the oxygen feeding flow rate. In the work, new discrete response equivalent (DRE) integration algorithms are proposed for simulation runs with a fixed integration step size, which is independent of the process dynamics (this possibility is due to self-adaptive features of the algorithms). The proposed algorithms have been compared with most other frequently used integration algorithms. The comparative tests show that, among the compared algorithms, only the DRE integration algorithms may be used with a fixed, arbitrarily chosen integration step size for simulation of the state ordinary differential equation system which may be both stiff and non-stiff during simulation.     

Spike timing precision and neural error correction: local behavior

The effects of spike timing precision and dynamical behavior on error correction in spiking neurons were investigated. Stationary discharges-phase locked, quasiperiodic, or chaotic-were induced in a simulated neuron by presenting pacemaker presynaptic spike trains across a model of a prototypical inhibitory synapse. Reduced timing precision was modeled by jittering presynaptic spike times. Aftereffects of errors-in this communication, missed presynaptic spikes-were determined by comparing postsynaptic spike times between simulations identical except for the presence or absence of errors. Results show that the effects of an error vary greatly depending on the ongoing dynamical behavior. In the case of phase lockings, a high degree of presynaptic spike timing precision can provide significantly faster error recovery. For nonlocked behaviors, isolated missed spikes can have little or no discernible aftereffects (or even serve to paradoxically reduce uncertainty in postsynaptic spike timing), regardless of presynaptic imprecision. This suggests two possible categories of error correction: high-precision locking with rapid recovery and low-precision nonlocked with error immunity.     

Spiking神经元输入脉冲扰动敏感性研究

脉冲神经网络是一种基于生物的网络模型,它的输入输出为具有时间特性的脉冲序列,其运行机制相比其他传统人工神经网络更加接近于生物神经网络。神经元之间通过脉冲序列传递信息,这些信息通过脉冲的激发时间编码能够更有效地发挥网络的学习性能。脉冲神经元的时间特性导致了其工作机制较为复杂,而spiking神经元的敏感性反映了当神经元输入发生扰动时输出的spike的变化情况,可以作为研究神经元内部工作机制的工具。不同于传统的神经网络,spiking神经元敏感性定义为输出脉冲的变化时刻个数与运行时间长度的比值,能直接反映出输入扰动对输出的影响程度。通过对不同形式的输入扰动敏感性的分析,可以看出spiking神经元的敏感性较为复杂,当全体突触发生扰动时,神经元为定值,而当部分突触发生扰动时,不同突触的扰动会导致不同大小的神经元敏感性。     

改进变步长算法在实时飞行仿真中的应用

飞行模拟机是民航训练飞行员的重要装备,飞行仿真系统是飞行模拟机的重要系统之一.飞行仿真要建立飞机的全量运动方程,利用数值算法解算运动方程达到仿真飞机飞行状况的目的.采用四元数法表示的飞机欧拉方程能够克服普通欧拉方程奇异性,但用定步长方法解算会产生较大累积误差,所以必须采用变步长方法.实时飞行仿真要求较高的实时性与逼真度,通过仿真试验分析,确定了采用一种改进变步长2阶Runge-Kutta法,该方法具有迭代次数少,解算精度高,实时性强的优点.利用该改进变步长算法,编写了实时飞行系统仿真软件,软件中使用相应算法解决了变步长算法选择步长与系统迭代定时时间不匹配的矛盾,实现了精确的实时仿真.     

Independent variable time-step integration of individual neurons for network simulations

Realistic neural networks involve the coexistence of stiff, coupled, continuous differential equations arising from the integrations of individual neurons, with the discrete events with delays used for modeling synaptic connections. We present here an integration method, the local variable time-step method (lvardt), that uses separate variable-step integrators for individual neurons in the network. Cells that are undergoing excitation tend to have small time steps, and cells that are at rest with little synaptic input tend to have large time steps. A synaptic input to a cell causes reinitialization of only that cell's integrator without affecting the integration of other cells. We illustrated the use of lvardt on three models: a worst-case synchronizing mutual-inhibition model, a best-case synfire chain model, and a more realistic thalamocortical network model.     

基于精细积分法的无偏非齐次灰色模型构建

传统无偏灰色预测模型的参数估计和序列模拟都是通过白化方程的离散时间响应函数求取相应的估计值和模拟值.基于精细积分法,提出一种无需求离散时间响应函数的无偏非齐次灰色模型.该模型通过引入新变量,将白化方程转化为齐次矩阵微分方程,利用指数矩阵求得递推关系,进而推导参数无偏估计公式,并采用精细积分法直接计算灰色模型的模拟(预测)值,从而减少舍入误差,提高计算精度.同时,还证明该建模方法具有非齐次指数规律重合性和伸缩变换一致性.严格非齐次指数序列、近似非齐次指数序列、不同类型的单调序列以及汽车保有量预测实例的结果进一步表明,所构建的模型能严格拟合非齐次指数序列,验证了该模型的有效性和实用性,提高了拟合(预测)精度.     

On the contact detection for contact-impact analysis in multibody systems

One of the most important and complex parts of the simulation of multibody systems with contact-impact involves the detection of the precise instant of impact. In general, the periods of contact are very small and, therefore, the selection of the time step for the integration of the time derivatives of the state variables plays a crucial role in the dynamics of multibody systems. The conservative approach is to use very small time steps throughout the analysis. However, this solution is not efficient from the computational view point. When variable time-step integration algorithms are used and the preimpact dynamics does not involve high-frequencies, the integration algorithms may use larger time steps and the contact between two surfaces may start with initial penetrations that are artificially high. This fact leads either to a stall of the integration algorithm or to contact forces that are physically impossible which, in turn, lead to post-impact dynamics that is unrelated to the physical problem. The main purpose of this work is to present a general and comprehensive approach to automatically adjust the time step, in variable time-step integration algorithms, in the vicinity of contact of multibody systems. The proposed methodology ensures that for any impact in a multibody system the time step of the integration is such that any initial penetration is below any prescribed threshold. In the case of the start of contact, and after a time step is complete, the numerical error control of the selected integration algorithm is forced to handle the physical criteria to accept/reject time steps in equal terms with the numerical error control that it normally uses. The main features of this approach are the simplicity of its computational implementation, its good computational efficiency, and its ability to deal with the transitions between non-contact and contact situations in multibody dynamics. A demonstration case provides the results that support the discussion and show the validity of the proposed methodology.     

Analytical integrate-and-fire neuron models with conductance-based dynamics and realistic postsynaptic potential time course for event-driven simulation strategies

In a previous paper (Rudolph & Destexhe, 2006), we proposed various models, the gIF neuron models, of analytical integrate-and-fire (IF) neurons with conductance-based (COBA) dynamics for use in event-driven simulations. These models are based on an analytical approximation of the differential equation describing the IF neuron with exponential synaptic conductances and were successfully tested with respect to their response to random and oscillating inputs. Because they are analytical and mathematically simple, the gIF models are best suited for fast event-driven simulation strategies. However, the drawback of such models is they rely on a nonrealistic postsynaptic potential (PSP) time course, consisting of a discontinuous jump followed by a decay governed by the membrane time constant. Here, we address this limitation by conceiving an analytical approximation of the COBA IF neuron model with the full PSP time course. The subthreshold and suprathreshold response of this gIF4 model reproduces remarkably well the postsynaptic responses of the numerically solved passive membrane equation subject to conductance noise, while gaining at least two orders of magnitude in computational performance. Although the analytical structure of the gIF4 model is more complex than that of its predecessors due to the necessity of calculating future spike times, a simple and fast algorithmic implementation for use in large-scale neural network simulations is proposed.     

Adaptive finite differences and IMEX time-stepping to price options under Bates model

In this paper, we consider numerical pricing of European and American options under the Bates model, a model which gives rise to a partial-integro differential equation. This equation is discretized in space using adaptive finite differences while an IMEX scheme is employed in time. The sparse linear systems of equations in each time-step are solved using an LU-decomposition and an operator splitting technique is employed for the linear complementarity problems arising for American options. The integral part of the equation is treated explicitly in time which means that we have to perform matrix-vector multiplications each time-step with a matrix with dense blocks. These multiplications are accomplished through fast Fourier transforms. The great performance of the method is demonstrated through numerical experiments.     

New schemes for the coupled nonlinear Schrödinger equation

In this paper, we present three new schemes for the coupled nonlinear Schrödinger equation. The three new schemes are multi-symplectic schemes that preserve the intrinsic geometry property of the equation. The three new schemes are also semi-explicit in the sense that they need not solve linear algebraic equations every time-step, which is usually the most expensive in numerical simulation of partial differential equations. Many numerical experiments on collisions of solitons are presented to show the efficiency of the new multi-symplectic schemes.     

Error and timing analysis of multiple time-step integration methods for molecular dynamics

Molecular dynamics simulations of biomolecules performed using multiple time-step integration methods are hampered by resonance instabilities. We analyze the properties of a simple 1D linear system integrated with the symplectic reference system propagator MTS (r-RESPA) technique following earlier work by others. A closed form expression for the time step dependent Hamiltonian which corresponds to r-RESPA integration of the model is derived. This permits us to present an analytic formula for the dependence of the integration accuracy on short-range force cutoff range. A detailed analysis of the force decomposition for the standard Ewald summation method is then given as the Ewald method is a good candidate to achieve high scaling on modern massively parallel machines. We test the new analysis on a realistic system, a protein in water. Under Langevin dynamics with a weak friction coefficient (ζ=1 ps⁻¹) to maintain temperature control and using the SHAKE algorithm to freeze out high frequency vibrations, we show that the 5 fs resonance barrier present when all degrees of freedom are unconstrained is postponed to ≈12 fs. An iso-error boundary with respect to the short-range cutoff range and multiple time step size agrees well with the analytical results which are valid due to dominance of the high frequency modes in determining integrator accuracy. Using r-RESPA to treat the long range interactions results in a 6× increase in efficiency for the decomposition described in the text.     

脉冲视觉研究进展

视频是视觉信息处理的基础概念,传统视频的帧率只有几十Hz,不能记录光的高速变化过程,成为限制机器视觉速度的天花板,其根本原因在于视频概念脱胎于胶片成像,未能发挥电子和数字技术的潜力。脉冲视觉模型通过感光器件捕获光子,累积能量达到约定阈值时产生脉冲,形成脉冲的时间越长,表明收到的光信号越弱,反之光信号越强,据此可估计任意时刻的光强,从而实现连续成像。采用普通器件,研制了比影视视频快千倍的超高速成像芯片和相机,进而基于脉冲神经网络实现了超高速目标检测、跟踪和识别,打破了机器视觉提速依赖算力线性增长的传统范式。本文从脉冲视觉模型表达视觉信息的生物学基础和物理原理出发,介绍了脉冲视觉原理的软件模拟器及其模拟真实世界光子传播的计算过程,描述了基于脉冲视觉原理的高灵敏光电传感器件及芯片的工作机理和结构设计、基于脉冲视觉的影像重建原理以及脉冲视觉信号与普通图像信号融合的计算摄像算法与计算摄像系统,介绍了基于脉冲神经网络的超高速运动目标检测、跟踪与识别,通过对比国际国内相关研究内容和发展现状,展望了脉冲视觉的发展与演进方向。脉冲视觉芯片和系统在工业(高铁、电力和轮机等不停机监测,智能制造高速监视等)、民用(高速相机、智能交通、辅助驾驶、司法取证和体育判罚等)以及国防(高速对抗)等领域都具有巨大应用潜力,是未来值得重点关注和研究的一个重要方向。     

Minimizing the effect of process mismatch in a neuromorphic system using spike-timing-dependent adaptation.

This paper investigates whether spike-timing-dependent plasticity (STDP) can minimize the effect of mismatch within the context of a depth-from-motion algorithm. To improve noise rejection, this algorithm contains a spike prediction element, whose performance is degraded by analog very large scale integration (VLSI) mismatch. The error between the actual spike arrival time and the prediction is used as the input to an STDP circuit, to improve future predictions. Before STDP adaptation, the error reflects the degree of mismatch within the prediction circuitry. After STDP adaptation, the error indicates to what extent the adaptive circuitry can minimize the effect of transistor mismatch. The circuitry is tested with static and varying prediction times and chip results are presented. The effect of noisy spikes is also investigated. Under all conditions the STDP adaptation is shown to improve performance.     

A continuous Hopfield network equilibrium points algorithm

The continuous Hopfield network (CHN) is a classical neural network model. It can be used to solve some classification and optimization problems in the sense that the equilibrium points of a differential equation system associated to the CHN is the solution to those problems. The Euler method is the most widespread algorithm to obtain these CHN equilibrium points, since it is the simplest and quickest method to simulate complex differential equation systems. However, this method is highly sensitive with respect to initial conditions and it requires a lot of CPU time for medium or greater size CHN instances. In order to avoid these shortcomings, a new algorithm which obtains one equilibrium point for the CHN is introduced in this paper. It is a variable time-step method with the property that the convergence time is shortened; moreover, its robustness with respect to initial conditions will be proven and some computational experiences will be shown in order to compare it with the Euler method.