Continuous attractors of a class of recurrent neural networks

Recurrent neural networks (RNNs) may possess continuous attractors, a property that many brain theories have implicated in learning and memory. There is good evidence for continuous stimuli, such as orientation, moving direction, and the spatial location of objects could be encoded as continuous attractors in neural networks. The dynamical behaviors of continuous attractors are interesting properties of RNNs. This paper proposes studying the continuous attractors for a class of RNNs. In this network, the inhibition among neurons is realized through a kind of subtractive mechanism. It shows that if the synaptic connections are in Gaussian shape and other parameters are appropriately selected, the network can exactly realize continuous attractor dynamics. Conditions are derived to guarantee the validity of the selected parameters. Simulations are employed for illustration.     

Effects of delay on the type and velocity of travelling pulses in neuronal networks with spatially decaying connectivity

We study a one-dimensional model of integrate-and-fire neurons that are allowed to fire only one spike, and are coupled by excitatory synapses with delay. At small delay values, this model describes a disinhibited cortical slice. At large delay values, the model is a reduction of a model of thalamic networks composed of excitatory and inhibitory neurons, in which the excitatory neurons show the post-inhibitory rebound mechanism. The velocity and stability of propagating continuous pulses are calculated analytically. Two pulses with different velocities exist if the synaptic coupling is larger than a minimal value; the pulse with the lower velocity is always unstable. Above a certain critical value of the constant delay, continuous pulses lose stability via a Hopf bifurcation, and lurching pulses emerge. The parameter regime for which lurching occurs is strongly affected by the synaptic footprint (connectivity) shape. A bistable regime, in which both continuous and lurching pulses can propagate. may occur with square or Gaussian footprint shapes but not with an exponential footprint shape. A perturbation calculation is used in order to calculate the spatial lurching period and the velocity of lurching pulses at large delay values. For strong synaptic coupling, the velocity of the lurching pulse is governed by the tail of the synaptic footprint shape. Moreover, the velocities of continuous and lurching pulses have the same functional dependencies on the strength of the synaptic coupling strength gsyn: they increase logarithmically with gsyn for an exponential footprint shape, they scale like (In gsyn)1/2 for a Gaussian footprint shape, and they are bounded for a square footprint shape or any shape with a finite support. We find analytically how the axonal propagation velocity reduces the velocity of continuous pulses; it does not affect the critical delay. We conclude that the differences in velocity and shape between the front of thalamic spindle waves in vitro and cortical paroxysmal discharges stem from their different effective delays.     

Dynamics and computation of continuous attractors

Continuous attractor is a promising model for describing the encoding of continuous stimuli in neural systems. In a continuous attractor, the stationary states of the neural system form a continuous parameter space, on which the system is neutrally stable. This property enables the neutral system to track time-varying stimuli smoothly, but it also degrades the accuracy of information retrieval, since these stationary states are easily disturbed by external noise. In this work, based on a simple model, we systematically investigate the dynamics and the computational properties of continuous attractors. In order to analyze the dynamics of a large-size network, which is otherwise extremely complicated, we develop a strategy to reduce its dimensionality by utilizing the fact that a continuous attractor can eliminate the noise components perpendicular to the attractor space very quickly. We therefore project the network dynamics onto the tangent of the attractor space and simplify it successfully as a one-dimensional Ornstein-Uhlenbeck process. Based on this simplified model, we investigate (1) the decoding error of a continuous attractor under the driving of external noisy inputs, (2) the tracking speed of a continuous attractor when external stimulus experiences abrupt changes, (3) the neural correlation structure associated with the specific dynamics of a continuous attractor, and (4) the consequence of asymmetric neural correlation on statistical population decoding. The potential implications of these results on our understanding of neural information processing are also discussed.     

A network of pyramidal neurons is sensitive to the timing of its excitatory inputs

The transmission of excitatory inputs by a network of coupled pyramidal cells is investigated by means of numerical simulations. The pyramidal cell models are coupled by excitatory synapses and each one receives an excitatory pulse at a random time extracted from a Gaussian distribution. Moreover, each cell model is injected with a noisy current. It was found that the excitatory coupling promotes the transmission of the synaptic inputs on a time scale of a few ms.     

含滞迟缓冲器的螺栓法兰连接结构稳态随机响应

基于螺栓法兰连接结构简化动力学模型,针对上部结构内包含滞迟缓冲机构的情况,通过蒙特卡洛数值模拟研究了高斯白噪声纵向载荷作用下的结构随机响应特性.在简化动力学模型中,通过拉压不同刚度的非线性弹簧表征螺栓法兰连接,上部缓冲机构用Yar-Hammond双线性滞迟系统描述,以便用于航天器对应舱段结构动力学响应的快速计算.在数值模拟之前对此系统进行了无量纲化,而后通过数值模拟分析了不同参数条件下的稳态响应概率密度函数,讨论了结构稳态随机响应的演化规律.     

摄像机简化模型对三维重构的影响--分析与实验

讨论了摄像机简化模型对三维重构的影响．主要结论有：当摄像机在两幅图像间的运动为纯平移运动时,从理论上证明了使用摄像机简化模型重构空间点与实际空间点之间满足仿射变换;当摄像机在两幅图像间的运动为一般刚体运动时,使用简化模型的重构只有在一定条件下才能较好地保持原物体的形状;在简化模型下,基于Kruppa方程的方法所估计的焦距精度不能满足三维重构的要求．实验结果表明：在三维重构中不能盲目地使用简化模型,必须对摄像机内参数进行全面标定．     

Computing with continuous attractors: stability and online aspects

Two issues concerning the application of continuous attractors in neural systems are investigated: the computational robustness of continuous attractors with respect to input noises and the implementation of Bayesian online decoding. In a perfect mathematical model for continuous attractors, decoding results for stimuli are highly sensitive to input noises, and this sensitivity is the inevitable consequence of the system's neutral stability. To overcome this shortcoming, we modify the conventional network model by including extra dynamical interactions between neurons. These interactions vary according to the biologically plausible Hebbian learning rule and have the computational role of memorizing and propagating stimulus information accumulated with time. As a result, the new network model responds to the history of external inputs over a period of time, and hence becomes insensitive to short-term fluctuations. Also, since dynamical interactions provide a mechanism to convey the prior knowledge of stimulus, that is, the information of the stimulus presented previously, the network effectively implements online Bayesian inference. This study also reveals some interesting behavior in neural population coding, such as the trade-off between decoding stability and the speed of tracking time-varying stimuli, and the relationship between neural tuning width and the tracking speed.     

Computing and stability in cortical networks

Cortical neurons are predominantly excitatory and highly interconnected. In spite of this, the cortex is remarkably stable: normal brains do not exhibit the kind of runaway excitation one might expect of such a system. How does the cortex maintain stability in the face of this massive excitatory feedback? More importantly, how does it do so during computations, which necessarily involve elevated firing rates? Here we address these questions in the context of attractor networks-networks that exhibit multiple stable states, or memories. We find that such networks can be stabilized at the relatively low firing rates observed in vivo if two conditions are met: (1) the background state, where all neurons are firing at low rates, is inhibition dominated, and (2) the fraction of neurons involved in a memory is above some threshold, so that there is sufficient coupling between the memory neurons and the background. This allows "dynamical stabilization" of the attractors, meaning feedback from the pool of background neurons stabilizes what would otherwise be an unstable state. We suggest that dynamical stabilization may be a strategy used for a broad range of computations, not just those involving attractors.     

Thin structure segmentation and visualization in three-dimensional biomedical images: a shape-based approach

This paper presents a shape-based approach in extracting thin structures, such as lines and sheets, from three-dimensional (3D) biomedical images. Of particular interest is the capability to recover cellular structures, such as microtubule spindle fibers and plasma membranes, from laser scanning confocal microscopic (LSCM) data. Hessian-based shape methods are reviewed. A synthesized linear structure is used to evaluate the sensitivity of the multiscale filtering approach in extracting closely positioned fibers. We find that the multiscale approach tends to fuse lines together, which makes it unsuitable for visualizing mouse egg spindle fibers. Single-scale Gaussian filters, balanced between sensitivity and noise resistance, are adopted instead. In addition, through an ellipsoidal Gaussian model, the eigenvalues of the Hessian matrix are quantitatively associated with the standard deviations of the Gaussian model. Existing shape filters are simplified and applied to LSCM data. A significant improvement in extracting closely positioned thin lines is demonstrated by the resultant images. Further, the direct association of shape models and eigenvalues makes the processed images more understandable qualitatively and quantitatively.     

Neuron function: the mystery persists

There are currently three primary models of how neurons function, each with its uses and variations, according to James McClelland, a professor of psychology and computer science at Carnegie Mellon University and codirector of the Center for Neural Basis of Cognition. The first and simplest is the integrate-and-fire model, which is based on the idea that the neuron adds and subtracts excitatory and inhibitory inputs until it reaches a threshold, at which point it fires a single impulse or action potential. Another model is the sigmoid transfer function, in which the neuron adds up excitatory and inhibitory inputs (as in the integrate-and-fire model) but treats the output as a continuous quantity. Finally, in the sigma-pi unit model, a neuron's output is equal to the sum of many products, each consisting of a multiplication of several inputs.     

基于形状索引的DoG特征结合GPRT的人脸关键点检测算法

针对传统人脸对齐算法效率较低的问题,提出一种基于形状索引的高斯差分(DoG)特征与高斯过程回归树(GPRT)的人脸关键点检测算法。首先,由高斯过程回归树的内核测量两个输入之间的相似性,并表示为两个输入进入相同叶子的树木数。然后基于高斯过程回归树模型提取形状索引DoG特征,并进一步完成GPRT的特征设计。最后从局部视网膜模式中采集滤波回应来增加稳定性,实现对抗几何差异的鲁棒性。在LFPW人脸数据库上验证结果表明该方法能够取得良好的性能表现,证明了基于形状索引的DoG特征与GPRT的人脸关键点检测算法的有效性。     

折线模糊神经网络对模糊函数的通用逼近

基于折线模糊数间的模糊算术以及一个新的扩展原理建立了一种新的模糊神经网络模型,证明了当输入为负模糊数时,相应的前向三层折线模糊网络可以作为连续模糊函数的通用逼近器,并给出了此时连续模糊函数所需满足的等价条件,最后给出了一个仿真实例。     

Burst synchrony patterns in hippocampal pyramidal cell model networks

Types of. mechanisms for and stability of synchrony are discussed in the context of two-compartment CA3 pyramidal cell and interneuron model networks. We show how the strength and timing of inhibitory and excitatory synaptic inputs work together to produce either perfectly synchronized or nearly synchronized oscillations, across different burst or spiking modes of firing. The analysis shows how excitatory inputs tend to desynchronize cells, and how common, slowly decaying inhibition can be used to synchronize them. We also introduce the concept of 'equivalent networks' in which networks with different architectures and synaptic connections display identical firing patterns.     

Dynamic neural cortical ensembles model

One-layered model of cortical neurons as a set of overlapping ensembles, each with a structure similar to Hopfield network, is proposed. Ensemble equilibrium equation is solved and formulas for connections weights calculation for given set of attractors are obtained. Concept of dynamic attractors that consists of consequent recalling of stored patterns with moving activity through the network is introduced. Role of dynamic attractors in long-term memory is discussed and mechanism for memory recovery after destruction of some neurons is proposed. Results of experiments on associative memory recovery after partial removal of neurons are shown.     

永磁同步电动机的混沌模型及其模糊建模

推导出永磁同步电动机 (PMSM)的数学模型, 讨论了常输入电压、常外部转矩条件下系统的稳态特性. 该模型在适当的参数选择和外部输入下, 可以呈现出非常复杂的极限环或混沌行为. 基于Takagi_Sugeno模糊建模方法, 给出了永磁同步电动机的TS模糊模型, 这为进一步研究模糊和混沌理论的内在联系, 及利用基于模糊模型的控制方法控制混沌现象提供了一条途径. 计算机仿真结果表明TS模糊系统的吸引子与原系统的混沌吸引子是拓扑等价的.     

Rewiring-induced chaos in pulse-coupled neural networks

The dependence of the dynamics of pulse-coupled neural networks on random rewiring of excitatory and inhibitory connections is examined. When both excitatory and inhibitory connections are rewired, periodic synchronization emerges with a Hopf-like bifurcation and a subsequent period-doubling bifurcation; chaotic synchronization is also observed. When only excitatory connections are rewired, periodic synchronization emerges with a saddle node-like bifurcation, and chaotic synchronization is also observed. This result suggests that randomness in the system does not necessarily contaminate the system, and sometimes it even introduces rich dynamics to the system such as chaos.     

An amplitude equation approach to contextual effects in visual cortex

A mathematical theory of interacting hypercolumns in primary visual cortex (V1) is presented that incorporates details concerning the anisotropic nature of long-range lateral connections. Each hypercolumn is modeled as a ring of interacting excitatory and inhibitory neural populations with orientation preferences over the range 0 to 180 degrees. Analytical methods from bifurcation theory are used to derive nonlinear equations for the amplitude and phase of the population tuning curves in which the effective lateral interactions are linear in the amplitudes. These amplitude equations describe how mutual interactions between hypercolumns via lateral connections modify the response of each hypercolumn to modulated inputs from the lateral geniculate nucleus; such interactions form the basis of contextual effects. The coupled ring model is shown to reproduce a number of orientation-dependent and contrast-dependent features observed in center-surround experiments. A major prediction of the model is that the anisotropy in lateral connections results in a nonuniform modulatory effect of the surround that is correlated with the orientation of the center.     

Training integrate-and-fire neurons with the Informax principle II

For pt I see J. Phys. A, vol. 35, p. 2379-94 (2002).We develop neuron learning rules using the Informax principle together with the input-output relationship of the integrate-and-fire (IF) model with Poisson inputs. The learning rule is then tested with constant inputs, time-varying inputs and images. For constant inputs, it is found that, under the Informax principle, a network of IF models with initially all positive weights tends to disconnect some connections between neurons. For time-varying inputs and images, we perform signal separation tasks called independent component analysis. Numerical simulations indicate that some number of inhibitory inputs improves the performance of the system in both biological and engineering senses.     

Analysis of Continuous Attractors for 2-D Linear Threshold Neural Networks

This brief investigates continuous attractors of the well-developed model in visual cortex, i.e., the linear threshold (LT) neural networks, based on a parameterized 2-D model. On the basis of existing results on nondegenerate equilibria in mathematics, we further discuss degenerate equilibria for such networks and present properties and distributions of the equilibria, which enables us to draw the coexistence conditions of nondegenerate and degenerate equilibria (e.g., singular lines). Our theoretical results provide a useful framework for precise tuning on the network parameters, e.g., the feedbacks and visual inputs. Simulations are also presented to illustrate the theoretical findings.