Spike-driven synaptic plasticity: theory, simulation, VLSI implementation

We present a model for spike-driven dynamics of a plastic synapse, suited for aVLSI implementation. The synaptic device behaves as a capacitor on short timescales and preserves the memory of two stable states (efficacies) on long timescales. The transitions (LTP/LTD) are stochastic because both the number and the distribution of neural spikes in any finite (stimulation) interval fluctuate, even at fixed pre- and postsynaptic spike rates. The dynamics of the single synapse is studied analytically by extending the solution to a classic problem in queuing theory (Takacs process). The model of the synapse is implemented in aVLSI and consists of only 18 transistors. It is also directly simulated. The simulations indicate that LTP/LTD probabilities versus rates are robust to fluctuations of the electronic parameters in a wide range of rates. The solutions for these probabilities are in very good agreement with both the simulations and measurements. Moreover, the probabilities are readily manipulable by variations of the chip's parameters, even in ranges where they are very small. The tests of the electronic device cover the range from spontaneous activity (3-4 Hz) to stimulus-driven rates (50 Hz). Low transition probabilities can be maintained in all ranges, even though the intrinsic time constants of the device are short (approximately 100 ms). Synaptic transitions are triggered by elevated presynaptic rates: for low presynaptic rates, there are essentially no transitions. The synaptic device can preserve its memory for years in the absence of stimulation. Stochasticity of learning is a result of the variability of interspike intervals; noise is a feature of the distributed dynamics of the network. The fact that the synapse is binary on long timescales solves the stability problem of synaptic efficacies in the absence of stimulation. Yet stochastic learning theory ensures that it does not affect the collective behavior of the network, if the transition probabilities are low and LTP is balanced against LTD.     

Recovery of hidden information through synaptic dynamics

The role of synaptic dynamics in processing neural information is investigated in a neural information channel with realistic model neurons having chaotic intrinsic dynamics. Our neuron models are realized in simple analogue circuits, and our synaptic connections are realized both in analogue circuits and through a dynamic clamp program. The information which is input to the first chaotic neuron in the channel emerges partially absent and partially 'hidden'. Part is absent because of the dynamical effects of the chaotic oscillation that effectively acts as a noisy channel. The 'hidden' part is recoverable. We show that synaptic parameters, most significantly receptor binding time constants, can be tuned to enhance the information transmission by the combination of a neuron plus a synapse. We discuss how the dynamics of the synapse can be used to recover 'hidden' information using average mutual information as a measure of the quality of information transport.     

Memory processes of flight situation awareness: interactive roles of working memory capacity, long-term working memory, and expertise

This research examined the role of working memory (WM) capacity and long-term working memory (LT-WM) in flight situation awareness (SA). We developed spatial and verbal measures of WM capacity and LT-WM skill and then determined the ability of these measures to predict pilot performance on SA tasks. Although both spatial measures of WM capacity and LT-WM skills were important predictors of SA performance, their importance varied as a function of pilot expertise. Spatial WM capacity was most predictive of SA performance for novices, whereas spatial LT-WM skill based on configurations of control flight elements (attitude and power) was most predictive for experts. Furthermore, evidence for an interactive role of WM and LT-WM mechanisms was indicated. Actual or potential applications of this research include cognitive analysis of pilot expertise and aviation training.     

Neuronal regulation versus synaptic unlearning in memory maintenance mechanisms

Hebbian learning, the paradigm of memory formation, needs further mechanisms to guarantee creation and maintenance of a viable memory system. One such proposed mechanism is Hebbian unlearning, a process hypothesized to occur during sleep. It can remove spurious states and eliminate global correlations in the memory system. However, the problem of spurious states is unimportant in the biologically interesting case of memories that are sparsely coded on excitatory neurons. Moreover, if some memories are anomalously strong and have to be weakened to guarantee proper functioning of the network, we show that it is advantageous to do that by neuronal regulation (NR) rather than synaptic unlearning. Neuronal regulation can account for dynamical maintenance of memory systems that undergo continuous synaptic turnover. This neuronal-based mechanism, regulating all excitatory synapses according to neuronal average activity, has recently gained strong experimental support. NR achieves synaptic maintenance over short time scales by preserving the average neuronal input field. On longer time scales it acts to maintain memories by letting the stronger synapses grow to their upper bounds. In ageing, these bounds are increased to allow stronger values of remaining synapses to overcome the loss of synapses that have perished.     

Visuospatial working memory in learning from multimedia systems

Abstract  Multimedia systems involve the association of various types of information: verbal information presented visually or auditorily, static or dynamic pictorial information, and sound information. In a cognitive approach, integrating this information involves complex processes constrained by properties of the learner's cognitive system, and especially by the capacity of working memory. This paper, reports on an experiment focused on the integration of verbal and pictorial information when students learn a series of physics concepts. The involvement of the visuo-spatial working memory was investigated by means of a dual-task paradigm. Results show that pictorial information enhances the learning process. They also suggest that the visual and the spatial components of visuospatial working memory should be considered separately. Finally, they emphasise the need to consider the limitations in cognitive resources available to the learner.     

Characterizing and reproducing working set size dynamics

A characterization of program referencing dynamics based on the temporal behavior of memory demand (as represented by the working set size of a program for a given window size) is proposed. A deterministic generative model which produces a references string having a given dynamic characterization is then presented, and its practical implementation is discussed. An experimental study of the accuracy and the viability of such a model is performed, together with a theoretical and empirical investigation of the feasibility of constructing a synthetic program which produces an approximation to that artificial string, thereby exhibiting the given dynamic behavior. The results for working-set-like environments with window sizes larger than or equal to that used in the generation of the artificial string are satisfactory, but those for other types of memory policies reveal that improvements to the string generation algorithm or different characterizations are needed.     

Storage capacity diverges with synaptic efficiency in an associative memory model with synaptic delay and pruning

It is known that storage capacity per synapse increases by synaptic pruning in the case of a correlation-type associative memory model. However, the storage capacity of the entire network then decreases. To overcome this difficulty, we propose decreasing the connectivity while keeping the total number of synapses constant by introducing delayed synapses. In this paper, a discrete synchronous-type model with both delayed synapses and their prunings is discussed as a concrete example of the proposal. First, we explain the Yanai-Kim theory by employing statistical neurodynamics. This theory involves macrodynamical equations for the dynamics of a network with serial delay elements. Next, considering the translational symmetry of the explained equations, we rederive macroscopic steady-state equations of the model by using the discrete Fourier transformation. The storage capacities are analyzed quantitatively. Furthermore, two types of synaptic prunings are treated analytically: random pruning and systematic pruning. As a result, it becomes clear that in both prunings, the storage capacity increases as the length of delay increases and the connectivity of the synapses decreases when the total number of synapses is constant. Moreover, an interesting fact becomes clear: the storage capacity asymptotically approaches 2//spl pi/ due to random pruning. In contrast, the storage capacity diverges in proportion to the logarithm of the length of delay by systematic pruning and the proportion constant is 4//spl pi/. These results theoretically support the significance of pruning following an overgrowth of synapses in the brain and may suggest that the brain prefers to store dynamic attractors such as sequences and limit cycles rather than equilibrium states.     

The role of working memory on graphical information processing

This research extends previous graphics research by examining how individual differences in working memory (WM) capacity and changes in graphic design influence graphical information processing. An experiment compared decision accuracy of two graphic decision aids and an unaided group for a task at two levels of complexity. There were no accuracy differences for the low complexity task. At high levels of task complexity, accuracy depended upon WM capacity and how the graphic aid influenced perception. Eye movement data show information processing differences also are contingent upon graphic design features and WM capacity. We postulate that graphs reduce cognitive overhead by shifting some of the cognitive burden to our visual perception system. More efficient graphical perceptual will improve decision performance only if our cognitive resources are capacity constrained and those cognitive resources are used elsewhere in the problem solving process.     

A working memory model based on fast Hebbian learning

Recent models of the oculomotor delayed response task have been based on the assumption that working memory is stored as a persistent activity state (a 'bump' state). The delay activity is maintained by a finely tuned synaptic weight matrix producing a line attractor. Here we present an alternative hypothesis, that fast Hebbian synaptic plasticity is the mechanism underlying working memory. A computational model demonstrates a working memory function that is more resistant to distractors and network inhomogeneity compared to previous models, and that is also capable of storing multiple memories.     

The role of working memory on graphical information processing

This research extends previous graphics research by examining how individual differences in working memory (WM) capacity and changes in graphic design influence graphical information processing. An experiment compared decision accuracy of two graphic decision aids and an unaided group for a task at two levels of complexity. There were no accuracy differences for the low complexity task. At high levels of task complexity, accuracy depended upon WM capacity and how the graphic aid influenced perception. Eye movement data show information processing differences also are contingent upon graphic design features and WM capacity. We postulate that graphs reduce cognitive overhead by shifting some of the cognitive burden to our visual perception system. More efficient graphical perceptual will improve decision performance only if our cognitive resources are capacity constrained and those cognitive resources are used elsewhere in the problem solving process.     

Auto-associative memory with two-stage dynamics of nonmonotonicneurons

Dynamical properties of a neural auto-associative memory with two-stage neurons are investigated theoretically. The two-stage neuron is a model whose output is determined by a two-stage nonlinear function of the internal field of the neuron (internal field is a weighted sum of outputs of the other neurons). The model is general, including nonmonotonic neurons as well as monotonic ones. Recent studies on associative memory revealed superiority of nonmonotonic neurons to monotonic ones. The present paper supplies theoretical verification on the high performance of nonmonotonic neurons and proves that the capacity of the auto-associative memory with two-stage neurons is O(n/ radicallog n), in contrast to O(n/log n) of simple threshold neurons. There is also a discussion of recall processes, where the radius of basin of attraction of memorized patterns is clarified. An intuitive explanation on why the performance is improved by nonmonotonic neurons is also provided by showing the correspondence of the recall processes of the two-stage-neuron net and orthogonal learning.     

New bistable organic photochromic compounds for bitwise working optical memory

The results of our spectral and kinetic studies for some photochromic compounds (58 diarylethenes and 9 fulgimides) are presented. Aiming at targeted synthesis of new photochromic compounds, the structure-photochromic behavior relationship (SPBR) for the synthesized compounds has been analyzed. The perspectives for application of these compounds in development of recording media for use in optical memory devices are outlined. The text was submitted by the authors in English.     

Activity clusters in dynamical model of the working memory system

A mathematical model of working memory is proposed in the form of a network of neuron-like units interacting via global inhibitory feedback. This network is capable of storing information items in the form of clusters of periodical spiking activity. Several sequentially excited clusters can coexist simultaneously, corresponding to several items stored in the memory. The capacity of the memory is studied as the function of the system parameters.     

Properties of synaptic transmission and the global stability of delayed activity states

The influence of synaptic channel properties on the stability of delayed activity maintained by recurrent neural networks is studied. The duration of excitatory post-synaptic current (EPSC) is shown to be essential for the global stability of the delayed response. The NMDA receptor channel is a much more reliable mediator of the reverberating activity than the AMPA receptor, due to a longer EPSC. This allows one to interpret the deterioration of the working memory observed in NMDA channel blockade experiments. The key mechanism leading to the decay of the delayed activity originates in the unreliability of synaptic transmission. The optimum fluctuation of the synaptic currents leading to the decay is identified. The decay time is calculated analytically and the result is confirmed computationally.     

Background synaptic activity as a switch between dynamical states in a network

A bright red light may trigger a sudden motor action in a driver crossing an intersection: stepping at once on the brakes. The same red light, however, may be entirely inconsequential if it appears, say, inside a movie theater. Clearly, context determines whether a particular stimulus will trigger a motor response, but what is the neural correlate of this? How does the nervous system enable or disable whole networks so that they are responsive or not to a given sensory signal? Using theoretical models and computer simulations, I show that networks of neurons have a built-in capacity to switch between two types of dynamic state: one in which activity is low and approximately equal for all units, and another in which different activity distributions are possible and may even change dynamically. This property allows whole circuits to be turned on or off by weak, unstructured inputs. These results are illustrated using networks of integrate-and-fire neurons with diverse architectures. In agreement with the analytic calculations, a uniform background input may determine whether a random network has one or two stable firing levels; it may give rise to randomly alternating firing episodes in a circuit with reciprocal inhibition; and it may regulate the capacity of a center-surround circuit to produce either self-sustained activity or traveling waves. Thus, the functional properties of a network may be drastically modified by a simple, weak signal. This mechanism works as long as the network is able to exhibit stable firing states, or attractors.     

Exact associative neural memory dynamics utilizing Boolean matrices

The exact dynamics of shallow loaded associative neural memories are generated and characterized. The Boolean matrix analysis approach is employed for the efficient generation of all possible state transition trajectories for parallel updated binary-state dynamic associative memories (DAMs). General expressions for the size of the basin of attraction of fundamental and oscillatory memories and the number of oscillatory and stable states are derived for discrete synchronous Hopfield DAMs loaded with one, two, or three even-dimensionality bipolar memory vectors having the same mutual Hamming distances between them. Spurious memories are shown to occur only if the number of stored patterns exceeds two in an even-dimensionality Hopfield memory. The effects of odd- versus even-dimensionality memory vectors on DAM dynamics and the effects of memory pattern encoding on DAM performance are tested. An extension of the Boolean matrix dynamics characterization technique to other, more complex DAMs is presented.     

A competitive associative memory model and its dynamics

Conventional associative memory networks perform "noncompetitive recognition" or "competitive recognition in distance". In this paper a "competitive recognition" associative memory model is introduced which simulates the competitive persistence of biological species. Unlike most of the conventional networks, the proposed model takes only the prototype patterns as its equilibrium points, so that the spurious points are effectively excluded. Furthermore, it is shown that, as the competitive parameters vary, the network has a unique stable equilibrium point corresponding to the winner competitive parameter and, in this case, the unique stable equilibrium state can be recalled from any initial key.     

Localization of two-particle quantum walk on glued-tree and its application in generating Bell states

Studies on two-particle quantum walks show that the spatial interaction between walkers will dynamically generate complex entanglement. However, those entanglement states are usually on a large state space and their evolutions are complex. It makes the entanglement states generated by quantum walk difficult to be applied directly in many applications of quantum information, such as quantum teleportation and quantum cryptography. In this paper, we firstly analyse a localization phenomena of two-particle quantum walk and then introduce how to use it to generate a Bell state. We will show that one special superposition component of the walkers’ state is localized on the root vertex if a certain interaction exists between walkers. This localization is interesting because it is contrary to our knowledge that quantum walk spreads faster than its classical counterpart. More interestingly, the localized component is a Bell state in the coin space of two walkers. By this method, we can obtain a Bell state easily from the quantum walk with spatial interaction by a local measurement, which is required in many applications. Through simulations, we verify that this method is able to generate the Bell state \(\frac{1}{\sqrt{2}}(|A \rangle _1|A\rangle _2 \pm |B\rangle _1|B\rangle _2)\) in the coin space of two walkers with fidelity greater than \(99.99999\,\%\) in theory, and we have at least a \(50\,\%\) probability to obtain the expected Bell state after a proper local measurement.