A new design method for the complex-valued multistate Hopfield associative memory

A method to store each element of an integral memory set M subset {1,2,...,K}/sup n/ as a fixed point into a complex-valued multistate Hopfield network is introduced. The method employs a set of inequalities to render each memory pattern as a strict local minimum of a quadratic energy landscape. Based on the solution of this system, it gives a recurrent network of n multistate neurons with complex and symmetric synaptic weights, which operates on the finite state space {1,2,...,K}/sup n/ to minimize this quadratic functional. Maximum number of integral vectors that can be embedded into the energy landscape of the network by this method is investigated by computer experiments. This paper also enlightens the performance of the proposed method in reconstructing noisy gray-scale images.     

A new neural associative memory model

We present a new associative memory model based on the Hamming memory, but where the winner-take-all network part is replaced by a layer of nodes with somewhat complex node functions. This new memory can produce output vectors with individual “don't know” bits. the simulations demonstrate that this memory model works appropriately. © 1992 John Wiley & Sons, Inc.     

Memristor-based chaotic neural networks for associative memory

In chaotic neural networks, the rich dynamic behaviors are generated from the contributions of spatio-temporal summation, continuous output function, and refractoriness. However, a large number of spatio-temporal summations in turn make the physical implementation of a chaotic neural network impractical. This paper proposes and investigates a memristor-based chaotic neural network model, which adequately utilizes the memristor with unique memory ability to realize the spatio-temporal summations in a simple way. Furthermore, the associative memory capabilities of the proposed memristor-based chaotic neural network have been demonstrated by conventional methods, including separation of superimposed pattern, many-to-many associations, and successive learning. Thanks to the nanometer scale size and automatic memory ability of the memristors, the proposed scheme is expected to greatly simplify the structure of chaotic neural network and promote the hardware implementation of chaotic neural networks.     

Exact associative neural memory dynamics utilizing Boolean matrices

The exact dynamics of shallow loaded associative neural memories are generated and characterized. The Boolean matrix analysis approach is employed for the efficient generation of all possible state transition trajectories for parallel updated binary-state dynamic associative memories (DAMs). General expressions for the size of the basin of attraction of fundamental and oscillatory memories and the number of oscillatory and stable states are derived for discrete synchronous Hopfield DAMs loaded with one, two, or three even-dimensionality bipolar memory vectors having the same mutual Hamming distances between them. Spurious memories are shown to occur only if the number of stored patterns exceeds two in an even-dimensionality Hopfield memory. The effects of odd- versus even-dimensionality memory vectors on DAM dynamics and the effects of memory pattern encoding on DAM performance are tested. An extension of the Boolean matrix dynamics characterization technique to other, more complex DAMs is presented.     

CMOS current-mode neural associative memory design with on-chiplearning

Based on the Grossberg mathematical model called the outstar, a modular neural net with on-chip learning and memory is designed and analyzed. The outstar is the minimal anatomy that can interpret the classical conditioning or associative memory. It can also be served as a general-purpose pattern learning device. To realize the outstar, CMOS (complimentary metal-oxide semiconductor) current-mode analog dividers are developed to implement the special memory called the ratio-type memory. Furthermore, a CMOS current-mode analog multiplier is used to implement the correlation. The implemented CMOS outstar can on-chip store the relative ratio values of the trained weights for a long time. It can also be modularized to construct general neural nets. HSPICE (a circuit simulator of Meta Software, Inc.) simulation results of the CMOS outstar circuits as associative memory and pattern learner have successfully verified their functions. The measured results of the fabricated CMOS outstar circuits have also successfully confirmed the ratio memory and on-chip learning capability of the circuits. Furthermore, it has been shown that the storage time of the ratio memory can be as long as five minutes without refreshment. Also the outstar can enhance the contrast of the stored pattern within a long period. This makes the outstar circuits quite feasible in many applications.     

Double-pattern associative memory neural network with pattern loop

A double-pattern associative memory neural network with “pattern loop” is proposed. It can store 2N bit bipolar binary patterns up to the order of 2^2N , retrieve part or all of the stored patterns which all have the minimum Hamming distance with input pattern, completely eliminate spurious patterns, and has higher storing efficiency and reliability than conventional associative memory. The length of a pattern stored in this associative memory can be easily extended from 2N to κN.     

Periodic oscillation of discrete-time bidirectional associative memory neural networks

In this paper, a discrete-time bidirectional associative memory neural networks model is considered. By employing the theory of coincidence degree and using Halanay-type inequality technique we give some sufficient conditions ensuring the existence and globally exponential stability of periodic solutions for the discrete-time bidirectional neural networks. An example with the numerical simulations is provided to show the correctness of our analysis.     

自适应联想记忆细胞神经网络的优化设计

针对训练自适应联想记忆细胞神经网络(AM-CNN)过程收敛慢,设计出的网络抗噪性能不高的特点,通过融合蚁群优化算法和粒子群算法的思想,提出以目标网络对噪声模式的输出误差为目标函数,在目标函数的一个阈值分成的两个区间内,分别采取局部搜索和全局搜索策略,训练出AM-CNN的克隆模板的设计方法。数字模拟表明,与以往的设计方法相比,该算法能在细胞神经网络4~6次的迭代过程中稳定输出期望模式,收敛速度更快,设计出的AM-CNN性能比较稳定,并对噪声鲁棒,对高斯噪声N(0,0.8)准确率达到90%左右。     

Evolving neural networks with iterative learning scheme for associative memory

A locally iterative learning (LIL) rule is adapted to a model of the associative memory based on the evolving recurrent-type neural networks composed of growing neurons. There exist extremely different scale parameters of time, the individual learning time and the generation in evolution. This model allows us definite investigation on the interaction between learning and evolution. And the reinforcement of the robustness against the noise is also achieved in the evolutional scheme.     

多维联想记忆神经网络在图像回忆中的应用

多维联想记忆神经网络可以用来回忆灰度图像。投影算法是回忆算法中的一类。采用不规则凸多边形的笛卡儿积构成的凸集代替正多边形的笛卡儿积构成的凸集,前者比后者更紧凑。数值实例表明,应用前者回忆灰度图像要比应用后者回忆灰度图像得到的图像更清晰,回忆所花时间更短。     

小世界神经网络中的联想记忆研究

本文研究了基于小世界结构的神经网络中的联想记忆模型.网络恢复存储模式的行为其实是无序参数为一有限值时的相位变化.越是规则的网络越是难以恢复记忆模式,且容易变成混合状态.另外,在无序参数的值适中时,对于一定数量的存储模式,最终得到恢复的效果可以达到最大.     

An evolutionary approach to associative memory in recurrent neural networks

In this paper, we investigate the associative memory in recurrent neural networks, based on the model of evolving neural networks proposed by Nolfi, Miglino and Parisi.Experimentally developed network has highly asymmetric synaptic weights and dilute connections, quite different from those of the Hopfield model.Some results on the effect of learning efficiency on the evolution are also presented.     

神经网络关联存储器的设计及应用

神经网络的存储能力一直是一个重大的缺陷,其存储主要体现在权重系数上,因此参数量一多,训练起来就十分困难。给神经网络设计一个外部关联存储器,能有效对神经网络的输入进行关联查询,并将查询的结果作为辅助输入传入到神经网络中去。此外,设计了自然语言语句的向量嵌入模型,并将模型和关联存储器集合起来形成一个自动关联语句语义向量的关联存储系统,其性能指标达到了设计要求。     

Recurrent neural nets as dynamical Boolean systems with applicationto associative memory

Discrete-time/discrete-state recurrent neural networks are analyzed from a dynamical Boolean systems point of view in order to devise new analytic and design methods for the class of both single and multilayer recurrent artificial neural networks. With the proposed dynamical Boolean systems analysis, we are able to formulate necessary and sufficient conditions for network stability which are more general than the well-known but restrictive conditions for the class of single layer networks: (1) symmetric weight matrix with (2) positive diagonal and (3) asynchronous update. In terms of design, we use a dynamical Boolean systems analysis to construct a high performance associative memory. With this Boolean memory, we can guarantee that all fundamental memories are stored, and also guarantee the size of the basin of attraction for each fundamental memory.     

A property of neural networks of associative memory with replacing units

A memory capacity exists for artificial neural networks of associative memory. The addition of new memories beyond the capacity overloads the network system and makes all learned memories irretrievable (catastrophic forgetting) unless there is a provision for forgetting old memories. This article describes a property of associative memory networks in which a number of units are replaced when networks learn. In our network, every time the network learns a new item or pattern, a number of units are erased and the same number of units are added. It is shown that the memory capacity of the network depends on the number of replaced units, and that there exists a optimal number of replaced units in which the memory capacity is maximized. The optimal number of replaced units is small, and seems to be independent of the network size. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007     

On the synthesis of brain-state-in-a-box neural models with application to associative memory

In this article we present techniques for designing associative memories to be implemented by a class of synchronous discrete-time neural networks based on a generalization of the brain-state-in-a-box neural model. First, we address the local qualitative properties and global qualitative aspects of the class of neural networks considered. Our approach to the stability analysis of the equilibrium points of the network gives insight into the extent of the domain of attraction for the patterns to be stored as asymptotically stable equilibrium points and is useful in the analysis of the retrieval performance of the network and also for design purposes. By making use of the analysis results as constraints, the design for associative memory is performed by solving a constraint optimization problem whereby each of the stored patterns is guaranteed a substantial domain of attraction. The performance of the designed network is illustrated by means of three specific examples.     

不确定双向联想记忆神经网络的稳定性分析

对双向联想记忆神经网络研究了平衡点的鲁棒稳定性.该网络的参数不确定,并且有时变时滞.当神经网络的激励函数满足Lipschitz连续性条件时,通过选取合适的Lyapunov-Krasovskii函数,建立了两个全局鲁棒稳定判据.由于这些判据考虑了神经元激励作用和抑制作用对网络的影响,他们和时变时滞的数值无关,并且易于使用内点算法进行检验.在注释中和已有的结果进行了对比.两个数值例子展示了所得结果的有效性.     

相空间压缩法实现混沌神经网络联想记忆

给出了利用相空间压缩法控制混沌神经网络,使得网络能够收敛于存储的目标模式的充分条件和必要条件.通过数学分析,得到了相空间压缩控制方法中对应参数的上下限;并通过对仿真结果的分析,提出了通过改变相空间压缩控制方法中对应的参数来实现混沌神经网络联想记忆的新方法.以上结果均通过仿真得到验证.     

Global stability of bidirectional associative memory neural networks with continuously distributed delays

Global asymptotic stability of the equilibrium point of bidirectional associative memory (BAM) neural networks with continuously distributed delays is studied. Under two mild assumptions on the activation functions, two sufficient conditions ensuring global stability of such networks are derived by utilizing Lyapunov functional and some inequality analysis technique. The results here extend some previous results. A numerical example is given showing the validity of our method.     

Extension from a linear associative memory to a linguistic linear associative memory

We present a linguistic extension from a crisp model by using a codification model that allows us to implement a fuzzy system on a discrete decision model. The paper begins with an introduction to the representation of fuzzy information, followed by a discussion of the codification method and the extension of a linear associative memory to a linguistic linear associative memory. Finally, we enumerate the advantages and disadvantages of the obtained linguistic linear associative memory. © 1998 John Wiley & Sons, Inc.13: 41–57, 1998