Storage of linguistic information in a continuous classifying associative memory

In this article, we analyze the use of the continuous classifying associative memory (CCLAM) to store linguistic information. Freedom in the choice of the functions that control the operation of the CCLAM equip this memory with the capacity to adapt to different information storage and recovery needs. We begin with the problem of storing linguistic terms by memorizing the patterns formed by the degrees of compatibility with these terms. After that, the problem of storing linguistic rules is discussed. Let us remark that in these cases not a single CCLAM is used, but rather a set of them connected in suitable structured ways. © 2002 Wiley Periodicals, Inc.     

模拟反馈联想记忆的吸引域和指数收敛速度的估计及其应用

本文得到了若干关于模拟反馈联想记忆各记忆模式的吸引域及其中每一点趋向相应记忆模式的指数收敛速度的估计结果，它们可用于高效模拟反馈联想记忆的性能评价以及综合过程．     

多值指数式多向联想记忆模型

多向联想记忆MDAM(multidirectional associative memory)模型是Kosko双向联想记忆模型BAM(bidirectional associative memory)的一个直接推广,它可应用于数据融合及维数分裂,使模型能处理大维数输入问题.目前所提出的若干种多向模型均局限于二值输入/输出模式对,但如在图象处理等的实际应用中,所处理的模式均是多值的.本文的目的就是提出一个多值指数式多向联想记忆模型MVeMDAM(multivalued exponential multidi     

A Bidirectional Hetero-Associative Memory for True-Color Patterns

Classical bidirectional associative memories (BAM) have poor memory storage capacity, are sensitive to noise, are subject to spurious steady states during recall, and can only recall bipolar patterns. In this paper, we introduce a new bidirectional hetero-associative memory model for true-color patterns that uses the associative model with dynamical synapses recently introduced in Vazquez and Sossa (Neural Process Lett, Submitted, 2008). Synapses of the associative memory could be adjusted even after the training phase as a response to an input stimulus. Propositions that guarantee perfect and robust recall of the fundamental set of associations are provided. In addition, we describe the behavior of the proposed associative model under noisy versions of the patterns. At last, we present some experiments aimed to show the accuracy of the proposed model with a benchmark of true-color patterns.     

Effective neuronal learning with ineffective Hebbian learning rules

In this article we revisit the classical neuroscience paradigm of Hebbian learning. We find that it is difficult to achieve effective associative memory storage by Hebbian synaptic learning, since it requires network-level information at the synaptic level or sparse coding level. Effective learning can yet be achieved even with nonsparse patterns by a neuronal process that maintains a zero sum of the incoming synaptic efficacies. This weight correction improves the memory capacity of associative networks from an essentially bounded one to a memory capacity that scales linearly with network size. It also enables the effective storage of patterns with multiple levels of activity within a single network. Such neuronal weight correction can be successfully carried out by activity-dependent homeostasis of the neuron's synaptic efficacies, which was recently observed in cortical tissue. Thus, our findings suggest that associative learning by Hebbian synaptic learning should be accompanied by continuous remodeling of neuronally driven regulatory processes in the brain.     

Recurrent correlation associative memories

A model for a class of high-capacity associative memories is presented. Since they are based on two-layer recurrent neural networks and their operations depend on the correlation measure, these associative memories are called recurrent correlation associative memories (RCAMs). The RCAMs are shown to be asymptotically stable in both synchronous and asynchronous (sequential) update modes as long as their weighting functions are continuous and monotone nondecreasing. In particular, a high-capacity RCAM named the exponential correlation associative memory (ECAM) is proposed. The asymptotic storage capacity of the ECAM scales exponentially with the length of memory patterns, and it meets the ultimate upper bound for the capacity of associative memories. The asymptotic storage capacity of the ECAM with limited dynamic range in its exponentiation nodes is found to be proportional to that dynamic range. Design and fabrication of a 3-mm CMOS ECAM chip is reported. The prototype chip can store 32 24-bit memory patterns, and its speed is higher than one associative recall operation every 3 mus. An application of the ECAM chip to vector quantization is also described.     

基于模式关联的一对多联想记忆方法

在现有的多模块一对多联想记忆模型中,由于所处理的记忆模式集合本身的特点以及记忆模式之间的关联被忽视,使得构造出来的模型结构复杂,难以实际应用.针对这一不足,提出一种基于模式关联的实现方法.以该方法构造出的多模块一对多联想记忆模型结构简单,易于硬件实现,使得多模块一对多联想记忆模型具有了实际应用的可能.     

Double-pattern associative memory neural network with pattern loop

A double-pattern associative memory neural network with “pattern loop” is proposed. It can store 2N bit bipolar binary patterns up to the order of 2^2N , retrieve part or all of the stored patterns which all have the minimum Hamming distance with input pattern, completely eliminate spurious patterns, and has higher storing efficiency and reliability than conventional associative memory. The length of a pattern stored in this associative memory can be easily extended from 2N to κN.     

Part 3: Brain science, information science and associative memory model

I review recent progress on the associative memory model, which is a kind of neural network model. First, I introduce this model and a mathematical theory called statistical neurodynamics describing its properties. Next, I discuss an associative memory model with hierarchically correlated memory patterns. Initially, in this model, the state approaches a mixed state that is a superposition of memory patterns. After that, it diverges from the mixed state, and finally converges to a memory pattern. I show that this retrieval dynamics can qualitatively replicate the temporal dynamics of face-responsive neurons in the inferior temporal cortex, which is considered to be the final stage of visual perception in the brain. Finally, I show an unexpected link between associative memory and mobile phones (CDMA). The mathematical structure of the CDMA multi-user detection problem resembles that of the associative memory model. It enables us to apply a theoretical framework of the associative memory model to CDMA.     

Behavioural study of median associative memory under true-colour image patterns

Median associative memories (MED-AMs) are a special type of associative memory that substitutes the maximum and minimum operators of a morphological associative memory with the median operator. This associative model has been applied to restore grey scale images and provided a better performance than morphological associative memories when the patterns are altered with mixed noise. Despite their power, MED-AMs have not been adopted in problems related with true-colour patterns. In this paper, we describe how MED-AMs can be applied to problems involving true-colour patterns. Furthermore, a complete study of the behaviour of this associative model in the restoration of true-colour images is performed using a benchmark of 16,000 images altered by different noise types.     

Weakly pulse-coupled oscillators, FM interactions, synchronization,and oscillatory associative memory

We study pulse-coupled neural networks that satisfy only two assumptions: each isolated neuron fires periodically, and the neurons are weakly connected. Each such network can be transformed by a piece-wise continuous change of variables into a phase model, whose synchronization behavior and oscillatory associative properties are easier to analyze and understand. Using the phase model, we can predict whether a given pulse-coupled network has oscillatory associative memory, or what minimal adjustments should be made so that it can acquire memory. In the search for such minimal adjustments we obtain a large class of simple pulse-coupled neural networks that ran memorize and reproduce synchronized temporal patterns the same way a Hopfield network does with static patterns. The learning occurs via modification of synaptic weights and/or synaptic transmission delays.     

A New Associative Model with Dynamical Synapses

The brain is not a huge fixed neural network, but a dynamic, changing neural network that continuously adapts to meet the demands of communication and computational needs. In classical neural networks approaches, particularly associative memory models, synapses are only adjusted during the training phase. After this phase, synapses are no longer adjusted. In this paper we describe a new dynamical model where synapses of the associative memory could be adjusted even after the training phase as a response to an input stimulus. We provide some propositions that guarantee perfect and robust recall of the fundamental set of associations. In addition, we describe the behavior of the proposed associative model under noisy versions of the patterns. At last, we present some experiments aimed to show the accuracy of the proposed model.     

Some experiments on human memory and a new neural model

An associative memory with parallel architecture is presented. The neurons are modelled by perceptrons having only binary, rather than continuous valued input. To store m elements each having n features, m neurons each with n connections are needed. The n features are coded as an n-bit binary vector. The weights of the n connections that store the n features of an element has only two values -1 and 1 corresponding to the absence or presence of a feature. This makes the learning very simple and straightforward. For an input corrupted by binary noise, the associative memory indicates the element that is closest (in terms of Hamming distance) to the noisy input. In the case where the noisy input is equidistant from two or more stored vectors, the associative memory indicates two or more elements simultaneously. From some simple experiments performed on the human memory and also on the associative memory, it can be concluded that the associative memory presented in this paper is in some respect more akin to a human memory than a Hopfield model.     

非线性连续联想记忆神经网络的分析和优化设计

研究非线性连续联想记忆神经网络的渐近稳定性,得出几个定理.在此基础上,提出了一 种优化设计方法,并给出了理论证明.目前已有的若干结论是本文所得定理的特例.     

基于线性规划的联想记忆神经网络模型

提出一种基于优化线性函数的神经网络联想记忆器,打破了将待识别模式作为网络起始点的常规,它能保证渐近稳定的平衡点集与样本点集相同,吸引域分布合理,不渐近稳定的平衡点恰为实际的拒识模式,并且电路实现容易,对拒识模式有清楚的解释。理论分析和计算机模拟都表明本文的模型是理想的联想记忆器,还可降低对硬件的精度要求。     

The modulation of learning state in a biological associative memory: an in vitro,in vivo,and in computo study of object recognition in mammalian olfactory cortex

During learning of overlapping input patterns in an associative memory, recall of previously stored patterns can interfere with the learning of new patterns. Most associative memory models avoid this difficulty by ignoring the effect of previously modified connections during learning, by clamping network activity to the patterns to be learned. Through the interaction of experimental and modeling techniques, we now have evidence to suggest that a somewhat analogous approach may have been taken by biology within the olfactory cerebral cortex. Specifically we have recently discovered that the naturally occurring neuromodulator acetylcholine produces a variety of effects on cortical cells and circuits which, when taken together, can prevent memory interference in a biologically realistic memory model. Further, it has been demonstrated that these biological mechanisms can actually improve the memory storage performance of previously published abstract neural network associative memory models.     

基于认知的联想记忆仿真研究

在对联想记忆神经网络的研究中,为提高现有联想记忆网络的存储能力以及相似模式和多值模式的联想成功率,提出了一种新的联想记忆网络。样本模式信息存储在动态权值矩阵中,网络根据不同的输入模式可自适应地调节当前权值矩阵。与传统联想网络相比,输入模式的信息不仅给出了联想记忆的初值,且在联想记忆过程中起到启发式搜索的作用,使网络的存储能力和联想成功率得到较好的改善。尤其可以有效地实现相似模式以及多值模式的联想记忆功能。仿真结果验证了方法的有效性。     

Design of GBSB neural associative memories using semidefiniteprogramming

This paper concerns reliable search for the optimally performing GBSB (generalized brain-state-in-a-box) neural associative memory given a set of prototype patterns to be stored as stable equilibrium points. First, we observe some new qualitative properties of the GBSB model. Next, we formulate the synthesis of GBSB neural associative memories as a constrained optimization problem. Finally, we convert the optimization problem into a semidefinite program (SDP), which can be solved efficiently by recently developed interior point methods. The validity of this approach is illustrated by a design example.     

Quantum Pattern Recognition

I review and expand the model of quantum associative memory that I have recently proposed. In this model binary patterns of n bits are stored in the quantum superposition of the appropriate subset of the computational basis of n qbits. Information can be retrieved by performing an input-dependent rotation of the memory quantum state within this subset and measuring the resulting state. The amplitudes of this rotated memory state are peaked on those stored patterns which are closest in Hamming distance to the input, resulting in a high probability of measuring a memory pattern very similar to it. The accuracy of pattern recall can be tuned by adjusting a parameter playing the role of an effective temperature. This model solves the well-known capacity shortage problem of classical associative memories, providing a large improvement in capacity. PACS: 03.67.-a     

Adaptive associative memories capable of pattern segmentation

This paper presents an adaptive type of associative memory (AAM) that can separate patterns from composite inputs which might be degraded by deficiency or noise and that can recover incomplete or noisy single patterns. The behavior of AAM is analyzed in terms of stability, giving the stable solutions (results of recall), and the recall of spurious memories (the undesired solutions) is shown to be greatly reduced compared with earlier types of associative memory that can perform pattern segmentation. Two conditions that guarantee the nonexistence of undesired solutions are also given. Results of computer experiments show that the performance of AAM is much better than that of the earlier types of associative memory in terms of pattern segmentation and pattern recovery.