A feedforward bidirectional associative memory

In contrast to conventional feedback bidirectional associative memory (BAM) network models, a feedforward BAM network is developed based on a one-shot design algorithm of O(p(2)(n+m)) computational complexity, where p is the number of prototype pairs and n, m are the dimensions of the input/output bipolar vectors. The feedforward BAM is an n-p-m three-layer network of McCulloch-Pitts neurons with storage capacity 2(min{m,n}) and guaranteed perfect bidirectional recall. The overall network design procedure is fully scalable in the sense that any number p=/<2(min{m,n}) of bidirectional associations can be implemented. The prototype patterns may be arbitrarily correlated. With respect to inference performance, it is shown that the Hamming attractive radius of each prototype reaches the maximum possible value. Simulation studies and comparisons illustrate and support these theoretical developments.     

An energy function-based design method for discrete hopfield associative memory with attractive fixed points

An energy function-based autoassociative memory design method to store a given set of unipolar binary memory vectors as attractive fixed points of an asynchronous discrete Hopfield network (DHN) is presented. The discrete quadratic energy function whose local minima correspond to the attractive fixed points of the network is constructed via solving a system of linear inequalities derived from the strict local minimality conditions. The weights and the thresholds are then calculated using this energy function. If the inequality system is infeasible, we conclude that no such asynchronous DHN exists, and extend the method to design a discrete piecewise quadratic energy function, which can be minimized by a generalized version of the conventional DHN, also proposed herein. In spite of its computational complexity, computer simulations indicate that the original method performs better than the conventional design methods in the sense that the memory can store, and provide the attractiveness for almost all memory sets whose cardinality is less than or equal to the dimension of its elements. The overall method, together with its extension, guarantees the storage of an arbitrary collection of memory vectors, which are mutually at least two Hamming distances away from each other, in the resulting network.     

A study on neural networks

The Hopfield neural network is a mathematical model in which each neuron performs a threshold logic function. an important property of the model is that a neural network always converges to a stable state when operating in a serial mode. This property is the basis of potential applications of neural networks such as associative memory devices, computational models, etc. This article reviews some of the known properties of the model and presents some new results regarding its possible applications. the principal contributions which are developed in this article are:

• (1) Showing that a very large class of mappings are not feasible by neural nets, in particular mappings which contain spheres, e.g., Hamming codes.
• (2) Showing that the neural network model can be designed to perform a local search algorithm for the Directed Min Cut problem.
• (3) Exploring the term “capacity of the neural network model” and criticizing some results known in the literature.
• (4) Showing the limitations of the model for its use as a pattern recognizer by proving that all images with a single black point can be recognized by the network iff the network is fully connected.

     

Double-pattern associative memory neural network with pattern loop

A double-pattern associative memory neural network with “pattern loop” is proposed. It can store 2N bit bipolar binary patterns up to the order of 2^2N , retrieve part or all of the stored patterns which all have the minimum Hamming distance with input pattern, completely eliminate spurious patterns, and has higher storing efficiency and reliability than conventional associative memory. The length of a pattern stored in this associative memory can be easily extended from 2N to κN.     

Dynamics of Winner-Take-All Competition in Recurrent Neural Networks With Lateral Inhibition

This paper studies the behavior of recurrent neural networks with lateral inhibition. Such network architecture is important in biological neural systems. General conditions determining the existence, number, and stability of network equilibria are derived. The manner in which these features depend upon steepness of neuronal activation functions and the strength of lateral inhibition is demonstrated for a broad range of nondecreasing activation functions including the discontinuous threshold function which represents the infinite gain limit. For uniform lateral inhibitory networks, the lateral inhibition is shown to sharpen neuron output patterns by increasing separation of suprathreshold activity levels of competing neurons. This results in the tendency of one neuron's output to dominate those of the others which can afford a "winner-take-all" (WTA) mechanism. Importantly, multiple stable equilibria may exist and shifts in inputs levels may yield network state transitions that exhibit hysteresis. A limitation of using lateral inhibition to implement WTA is further demonstrated. The possible significance of these identified network dynamics to physiology and pathophysiology of the striatum (particularly in Parkinsonian rest tremor) is discussed     

双向联想记忆学习与回忆的研究

本文对双向联想记忆(BAM)的学习与回忆过程进行了详细的分析。在学习过程中,先是运用自适应非对称BAM算法进行学习,进而采用设置印象门限的反复记忆算法进行学习,本文从理论上证明了印象门限与样本吸引域之间的关系,指出反复记忆方法的理论依据。回忆过程中,采用非零阈值函数的运行方程,提出了阈值学习方法,并且从理论上证明了非零阈值函数的运行方程的采用,可进一步扩大吸引域。为了进一步扩大网络的信息存储量,本文引入了并联的BAM结构。本文方法的采纳,使得BAM网络的信息存储量、误差校正能力等得到很大程度的提高。     

基于约束区域的BSB联想记忆模型

提出一种根据联想记忆点设计基于约束区域的ＢＳＢ（Ｂｒａｉｎ－Ｓｔａｔｅ－ｉｎｍ－ａ－Ｂｏｘ）神经网络模型,它保证了渐近稳定的平衡点集与样本点集相同,不渐近稳定的平衡点恰为实际的拒识状态,并且吸引域分布合理,从而将ＥＳＢ完善为理想的联想记忆器。     

The convergence of Hamming memory networks

The convergence properties of Hamming memory networks are studied. It is shown how to construct the network so that it probably converges to an appropriate result, and a tight bound is given on the convergence time. The bound on the convergence time is largest when several stored vectors are at the minimum distance from the input vector. For random binary vectors, the probability for such situations to occur is not small. With a specific choice of parameter values, the worst-case convergence time is on the order of p ln (pn), where p is the memory capacity and n is the vector length. By allowing the connection weights to change during the computation, the convergence time can be decreased considerably.     

Domains of attraction in autoassociative memory networks

An autoassociative memory network is constructed by storing reference pattern vectors whose components consist of a small positive number ∈ and 1-∈. Although its connection weights can not be determined only by this storing condition, it is proved that the output function of the network becomes a contraction mapping in a region around each stored pattern if ∈ is sufficiently small. This implies that the region is a domain of attraction in the network. The shape of the region is clarified in our analysis. Domains of attraction larger than this region are also found. Any noisy pattern vector in such domains, which may have real valued components, can be recognized as one of the stored patterns. We propose a method for determining connection weights of the network, which uses the shape of the domains of attraction. The model obtained by this method has symmetric connection weights and is successfully applied to character pattern recognition.     

Generalizations of the Hamming Associative Memory

This Letter reviews four models of associative memory which generalize the operation of the Hamming associative memory: the grounded Hamming memory, the cellular Hamming memory, the decoupled Hamming memory, and the two-level decoupled Hamming memory. These memory models offer high performance and allow for a more practical hardware realization than the Hamming net and other fully interconnected neural net architectures.     

一种新型双向联想记忆神经网络

提出了一种新型双向联想记忆神经网络,此网络将两个相互关联的模式以模式对的形式存储在由N个连接构成的模式环中,记忆容量为22N数量级,完全消除了假模式对、能够全部或部分地回忆出与输入模式对具有最小Hamming距的被记忆的模式对,同时具有较高的记忆效率和可靠性。连接由“连接状态”和“禁止路径”组成,前者直接存储二进制模式对向量的分量,后者用于消除假模式;此神经网络具有正向联想、逆向联想和自联想方式,使得网络能更灵活有效地满足不同的回忆要求。     

A new neural associative memory model

We present a new associative memory model based on the Hamming memory, but where the winner-take-all network part is replaced by a layer of nodes with somewhat complex node functions. This new memory can produce output vectors with individual “don't know” bits. the simulations demonstrate that this memory model works appropriately. © 1992 John Wiley & Sons, Inc.     

A novel continuous-time neural network for realizing associativememory

A novel neural network is proposed in this paper for realizing associative memory. The main advantage of the neural network is that each prototype pattern is stored if and only if as an asymptotically stable equilibrium point. Furthermore, the basin of attraction of each desired memory pattern is distributed reasonably (in the Hamming distance sense), and an equilibrium point that is not asymptotically stable is really the state that cannot be recognized. The proposed network also has a high storage as well as the capability of learning and forgetting, and all its components can be implemented. The network considered is a very simple linear system with a projection on a closed convex set spanned by the prototype patterns. The advanced performance of the proposed network is demonstrated by means of simulation of a numerical example.     

快速分类神经网络在字符识别中的应用

本文以海明神经网络与自适应谐振理论(ART)模型学习算法为基础,从理论上分析了海明网络学习算法的缺陷,利用ART网络的思想,提出了一种快速分类的神经元网络的算法,命名为Improved Hamming算法(简称Im-H算法)。此算法主要优点在于阈值更新及引入了经验迭代次数。将此算法用于字符模式识别,大量的计算机实验结果表明了Im-H网络学习算法的有效性、快速性。     

The modulation of learning state in a biological associative memory: an in vitro,in vivo,and in computo study of object recognition in mammalian olfactory cortex

During learning of overlapping input patterns in an associative memory, recall of previously stored patterns can interfere with the learning of new patterns. Most associative memory models avoid this difficulty by ignoring the effect of previously modified connections during learning, by clamping network activity to the patterns to be learned. Through the interaction of experimental and modeling techniques, we now have evidence to suggest that a somewhat analogous approach may have been taken by biology within the olfactory cerebral cortex. Specifically we have recently discovered that the naturally occurring neuromodulator acetylcholine produces a variety of effects on cortical cells and circuits which, when taken together, can prevent memory interference in a biologically realistic memory model. Further, it has been demonstrated that these biological mechanisms can actually improve the memory storage performance of previously published abstract neural network associative memory models.     

二层非耦合汉明联想存储器的分析及实现

二层非耦合汉明联想存储器是一种新型的联想存储器.它既具有基本汉明联想存储器容量大,容错性好的优点,同时又克服了基本汉明联想存储器回收慢,硬件难以实现的缺点.整个二层非耦合汉明联想存储器的硬件实现是基于FPGA的设计.它的非耦合结构使得硬件得以实现,并提高了执行效率,本文就上述讨论进行了分析,并给出了具体实现.     

Alien attractors and memory annihilation of structured sets inHopfield networks

This paper considers the encoding of structured sets into Hopfield associative memories. A structured set is a set of vectors with equal Hamming distance h from one another, and its centroid is an external vector that has distance h/2 from every vector of the set. Structured sets having centroids are not infrequent. When such a set is encoded into a noiseless Hopfield associative memory using a bipolar outer-product connection matrix, and the network operates with synchronous neuronal update, the memory of all encoded vectors is annihilated even for sets with as few as three vectors in dimension n>5 (four for n=5). In such self-annihilating structured sets, the centroid emerges as a stable attractor. We call it an alien attractor. For canonical structured sets, self-annihilation takes place only if h相似文献   

An efficient learning algorithm for associative memories

Associative memories (AMs) can be implemented using networks with or without feedback. We utilize a two-layer feedforward neural network and propose a learning algorithm that efficiently implements the association rule of a bipolar AM. The hidden layer of the network employs p neurons where p is the number of prototype patterns. In the first layer, the input pattern activates at most one hidden layer neuron or "winner". In the second layer, the "winner" associates the input pattern to the corresponding prototype pattern. The underlying association principle is minimum Hamming distance and the proposed scheme can be viewed also as an approximately minimum Hamming distance decoder. Theoretical analysis supported by simulations indicates that, in comparison with other suboptimum minimum Hamming distance association schemes, the proposed structure exhibits the following favorable characteristics: 1) it operates in one-shot which implies no convergence-time requirements; 2) it does not require any feedback; and 3) our case studies show that it exhibits superior performance to the popular linear system in a saturated mode. The network also exhibits 4) exponential capacity and 5) easy performance assessment (no asymptotic analysis is necessary). Finally, since it does not require any hidden layer interconnections or tree-search operations, it exhibits low structural as well as operational complexity.     

Neural simulation of acquired and progressive language disorders

This paper describes a novel approach to the simulation of language disorders, based upon the notion of a multi-network architecture —a set of autonomous neural networks which have been linked in some manner to perform a complex function that cannot readily be performed by any one network alone. The merits of this approach have been assessed by mapping a neuropsychological model of single-word language processing onto a multi-network architecture. Language disorders may be simulated by damaging, or lesioning, one or more component networks. Our attempts to simulate two specific language disorders, semantic dementia and deep dysphasia, are described. The relative success of our simulation work is encouraging, and leads us to conclude that a multi-network approach to the simulation of cognitive function and dysfunction offers a valid alternative to the traditional single-network based perspective.     

Quantum Pattern Recognition

I review and expand the model of quantum associative memory that I have recently proposed. In this model binary patterns of n bits are stored in the quantum superposition of the appropriate subset of the computational basis of n qbits. Information can be retrieved by performing an input-dependent rotation of the memory quantum state within this subset and measuring the resulting state. The amplitudes of this rotated memory state are peaked on those stored patterns which are closest in Hamming distance to the input, resulting in a high probability of measuring a memory pattern very similar to it. The accuracy of pattern recall can be tuned by adjusting a parameter playing the role of an effective temperature. This model solves the well-known capacity shortage problem of classical associative memories, providing a large improvement in capacity. PACS: 03.67.-a