Alpha–Beta bidirectional associative memories: theory and applications

In this work a new Bidirectional Associative Memory model, surpassing every other past and current model, is presented. This new model is based on Alpha–Beta associative memories, from whom it inherits its name. The main and most important characteristic of Alpha–Beta bidirectional associative memories is that they exhibit perfect recall of all patterns in the fundamental set, without requiring the fulfillment of any condition. The capacity they show is 2^min(n,m), being n and m the input and output patterns dimensions, respectively. Design and functioning of this model are mathematically founded, thus demonstrating that pattern recall is always perfect, with no regard to the trained pattern characteristics, such as linear independency, orthogonality, or Hamming distance. Two applications illustrating the optimal functioning of the model are shown: a translator and a fingerprint identifier.     

A Bidirectional Hetero-Associative Memory for True-Color Patterns

Classical bidirectional associative memories (BAM) have poor memory storage capacity, are sensitive to noise, are subject to spurious steady states during recall, and can only recall bipolar patterns. In this paper, we introduce a new bidirectional hetero-associative memory model for true-color patterns that uses the associative model with dynamical synapses recently introduced in Vazquez and Sossa (Neural Process Lett, Submitted, 2008). Synapses of the associative memory could be adjusted even after the training phase as a response to an input stimulus. Propositions that guarantee perfect and robust recall of the fundamental set of associations are provided. In addition, we describe the behavior of the proposed associative model under noisy versions of the patterns. At last, we present some experiments aimed to show the accuracy of the proposed model with a benchmark of true-color patterns.     

Dynamics of selective recall in an associative memory model with one-to-many associations.

The dynamics of selective recall in an associative memory model are analyzed in the scenario of one-to-many association. The present model, which can deal with one-to-many association, consists of a heteroassociative network and an autoassociative network. In the heteroassociative network, a mixture of associative items in one-to-many association is recalled by a key item. In the autoassociative network, the selective recall of one of the associative items is examined by providing a seed of a target item either to the heteroassociative network (Model 1) or to the autoassociative network (Model 2). We show that the critical similarity of Model 2 is not sensitive to the change in the dimension ratio of key vectors to associative vectors, and it has smaller critical similarity than Model 1 for a large initial overlap. On the other hand, we show that Model 1 has smaller critical similarity for a small initial overlap. We also show that unreachable equilibrium states exist in the proposed model.     

Storage Capacity of the Exponential Correlation Associative Memory

In this paper we analyze the pattern storage capacity of the exponential correlation associative memory (ECAM). This architecture was first studied by Chiueh and Goodman [3] who concluded that, under certain conditions on the input patterns, the memory has a storage capacity that was exponential in the length of the bit-patterns. A recent analysis by Pelillo and Hancock [9], using the Kanerva picture of recall, concluded that the storage capacity was limited by 2 ^N–1/N ² patterns. Both of these analyses can be criticised on the basis that they overlook the role of initial bit-errors in the recall process and deal only with the capacity for perfect pattern recall. In other words, they fail to model the effect of presenting corrupted patterns to the memory. This can be expected to lead to a more pessimistic limit. Here we model the performance of the ECAM when presented with corrupted input patterns. Our model leads to an expression for the storage capacity of the ECAM both in terms of the length of the bit-patterns and the probability of bit-corruption in the original input patterns. These storage capacities agree closely with simulation. In addition, our results show that slightly superior performance can be obtained by selecting an optimal value of the exponential constant.     

Adaptive associative memories capable of pattern segmentation

This paper presents an adaptive type of associative memory (AAM) that can separate patterns from composite inputs which might be degraded by deficiency or noise and that can recover incomplete or noisy single patterns. The behavior of AAM is analyzed in terms of stability, giving the stable solutions (results of recall), and the recall of spurious memories (the undesired solutions) is shown to be greatly reduced compared with earlier types of associative memory that can perform pattern segmentation. Two conditions that guarantee the nonexistence of undesired solutions are also given. Results of computer experiments show that the performance of AAM is much better than that of the earlier types of associative memory in terms of pattern segmentation and pattern recovery.     

Recurrent correlation associative memories

A model for a class of high-capacity associative memories is presented. Since they are based on two-layer recurrent neural networks and their operations depend on the correlation measure, these associative memories are called recurrent correlation associative memories (RCAMs). The RCAMs are shown to be asymptotically stable in both synchronous and asynchronous (sequential) update modes as long as their weighting functions are continuous and monotone nondecreasing. In particular, a high-capacity RCAM named the exponential correlation associative memory (ECAM) is proposed. The asymptotic storage capacity of the ECAM scales exponentially with the length of memory patterns, and it meets the ultimate upper bound for the capacity of associative memories. The asymptotic storage capacity of the ECAM with limited dynamic range in its exponentiation nodes is found to be proportional to that dynamic range. Design and fabrication of a 3-mm CMOS ECAM chip is reported. The prototype chip can store 32 24-bit memory patterns, and its speed is higher than one associative recall operation every 3 mus. An application of the ECAM chip to vector quantization is also described.     

CLAM: A new model of associative memory

We present a new associative memory model that stores arbitrary bipolar patterns without the problems we can find in other models like BAM or LAM. After identifying those problems we show the new memory topology and we explain its learning and recall stages. Mathematical demonstrations are provided to prove that the new memory model guarantees the perfect retrieval of every stored pattern and also to prove that whatever the input of the memory is, it operates as a nearest neighbor classifier. ©2000 John Wiley & Sons, Inc.     

Associative Gray Level Pattern Processing using Binary Decomposition and α β Memories

In this note we show how a binary memory can be used to recall gray-level patterns. We take as example the α β associative memories recently proposed in Yáñez, Associative Memories based on order Relations and Binary Operators(In Spanish), PhD Thesis, Center for computing Research, February of 2002, only useful in the binary case. Basically, the idea consists on that given a set of gray-level patterns to be first memorized: (1) Decompose them into their corresponding binary patterns, and (2) Build the corresponding binary associative memory (one memory for each binary layer) with each training pattern set (by layers). A given pattern or a distorted version of it, it is recalled in three steps: (1) Decomposition of the pattern by layers into its binary patterns, (2) Recalling of each one of its binary components, layer by layer also, and (3) Reconstruction of the pattern from the binary patterns already recalled in step 2. The proposed methodology operates at two phases: training and recalling. Conditions for perfect recall of a pattern either from the fundamental set or from a distorted version of one them are also given. Experiments where the efficiency of the proposal is tested are also given.     

Quantum Pattern Recognition

I review and expand the model of quantum associative memory that I have recently proposed. In this model binary patterns of n bits are stored in the quantum superposition of the appropriate subset of the computational basis of n qbits. Information can be retrieved by performing an input-dependent rotation of the memory quantum state within this subset and measuring the resulting state. The amplitudes of this rotated memory state are peaked on those stored patterns which are closest in Hamming distance to the input, resulting in a high probability of measuring a memory pattern very similar to it. The accuracy of pattern recall can be tuned by adjusting a parameter playing the role of an effective temperature. This model solves the well-known capacity shortage problem of classical associative memories, providing a large improvement in capacity. PACS: 03.67.-a     

A bidirectional heteroassociative memory for binary and grey-level patterns

Typical bidirectional associative memories (BAM) use an offline, one-shot learning rule, have poor memory storage capacity, are sensitive to noise, and are subject to spurious steady states during recall. Recent work on BAM has improved network performance in relation to noisy recall and the number of spurious attractors, but at the cost of an increase in BAM complexity. In all cases, the networks can only recall bipolar stimuli and, thus, are of limited use for grey-level pattern recall. In this paper, we introduce a new bidirectional heteroassociative memory model that uses a simple self-convergent iterative learning rule and a new nonlinear output function. As a result, the model can learn online without being subject to overlearning. Our simulation results show that this new model causes fewer spurious attractors when compared to others popular BAM networks, for a comparable performance in terms of tolerance to noise and storage capacity. In addition, the novel output function enables it to learn and recall grey-level patterns in a bidirectional way.     

A study of pattern recovery in recurrent correlation associative memories

In this paper, we analyze the recurrent correlation associative memory (RCAM) model of Chiueh and Goodman (1990, 1991). This is an associative memory in which stored binary memory patterns are recalled via an iterative update rule. The update of the individual pattern-bits is controlled by an excitation function, which takes as its argument the inner product between the stored memory patterns and the input patterns. Our contribution is to analyze the dynamics of pattern recall when the input patterns are corrupted by noise of a relatively unrestricted class. We show how to identify the excitation function which maximizes the separation (the Fisher discriminant) between the uncorrupted realization of the noisy input pattern and the remaining patterns residing in the memory. The excitation function which gives maximum separation is exponential when the input bit-errors follow a binomial distribution. We develop an expression for the expectation value of bit-error probability on the input pattern after one iteration. We show how to identify the excitation function which minimizes the bit-error probability. The relationship between the excitation functions which result from the two different approaches is examined for a binomial distribution of bit-errors. We develop a semiempirical approach to the modeling of the dynamics of the RCAM.     

Gray-scale morphological associative memories

Neural models of associative memories are usually concerned with the storage and the retrieval of binary or bipolar patterns. Thus far, the emphasis in research on morphological associative memory systems has been on binary models, although a number of notable features of autoassociative morphological memories (AMMs) such as optimal absolute storage capacity and one-step convergence have been shown to hold in the general, gray-scale setting. In this paper, we make extensive use of minimax algebra to analyze gray-scale autoassociative morphological memories. Specifically, we provide a complete characterization of the fixed points and basins of attractions which allows us to describe the storage and recall mechanisms of gray-scale AMMs. Computer simulations using gray-scale images illustrate our rigorous mathematical results on the storage capacity and the noise tolerance of gray-scale morphological associative memories (MAMs). Finally, we introduce a modified gray-scale AMM model that yields a fixed point which is closest to the input pattern with respect to the Chebyshev distance and show how gray-scale AMMs can be used as classifiers.     

混沌神经网络在分离叠加模式和多对多联想记忆中的应用

混沌是不含外加随机因素的完全确定性的系统表现出来的界于规则和随机之间的内秉随机行为。脑神经系统是由神经细胞组成的网络。类似于人脑思维的人工神经网络与冯·诺依曼计算机相比,在信息处理方面有很大的优越性。混沌和神经网络相互融合的研究是从90年代开始的,其主要的目标是通过分析大脑的混沌现象,建立含有混沌动力学的神经网络模型(即混沌神经网络模型),将混沌的遍历性、对初始值敏感等特点与神经网络的非线性、自适应、并行处理优势相结合,     

Second-order asymmetric BAM design with a maximal basin of attraction

Bidirectional associative memory (BAM) generalizes the associative memory (AM) to be capable of performing two-way recalling of pattern pairs. Asymmetric bidirectional associative memory (ABAM) is a variant of BAM relaxed with connection weight symmetry restriction and enjoys a much better performance than a conventional BAM structure. Higher-order associative memories (HOAMs) are reputed for their higher memory capacity than the first-order counterparts. The paper concerns the design of a second-order asymmetric bidirectional associative memory (SOABAM) with a maximal basin of attraction, whose extension to a HOABAM is possible and straightforward. First, a necessary and sufficient condition is derived for the connection weight matrix of SOABAM that can guarantee the recall of all prototype pattern pairs. A local training rule which is adaptive in the learning step size is formulated. Then derived is a theorem, designing a SOABAM further enlarging the quantities required to meet the complete recall theorem will enhance the capability of evolving a noisy pattern to converge to its association pattern vector without error. Based on this theorem, our algorithm is also modified to ensure each training pattern is stored with a basin of attraction as large as possible.     

A selective attention-based method for visual pattern recognitionwith application to handwritten digit recognition and face recognition

Parallel pattern recognition requires great computational resources; it is NP-complete. From an engineering point of view it is desirable to achieve good performance with limited resources. For this purpose, we develop a serial model for visual pattern recognition based on the primate selective attention mechanism. The idea in selective attention is that not all parts of an image give us information. If we can attend only to the relevant parts, we can recognize the image more quickly and using less resources. We simulate the primitive, bottom-up attentive level of the human visual system with a saliency scheme and the more complex, top-down, temporally sequential associative level with observable Markov models. In between, there is a neural network that analyses image parts and generates posterior probabilities as observations to the Markov model. We test our model first on a handwritten numeral recognition problem and then apply it to a more complex face recognition problem. Our results indicate the promise of this approach in complicated vision applications     

基于广义逆的指数型联想存储器模型

基于Ｋｏｈｏｎｅｎ的广义逆联想存储模型ＧＩＡＭ（ｇｅｎｅｒａｌｉｚｅｄｉｎｖｅｒｓｅａｓｏｃｉａｔｉｖｅｍｅｍｏｒｙ）和Ｍｕｒａｋａｍｉ的最小平方联想存储ＬＳＡＭ（ｌｅａｓｔｓｑｕａｒｅｓａｓｓｏｃｉａｔｉｖｅｍｅｍｏｒｙ）原理，本文提出了一个指数型联想存储器．该模型的存储性能经计算机模拟证实，远远优于ＧＩＡＭ和ＬＳＡＭ，通过适当地调节参数，几乎可达到完全的联想．对输入噪声方差，无需先验假设，同时还实现了一定程度的非线性映射特性.     

The restricted influence of sparseness of coding on the capacity of familiarity discrimination networks

Much evidence indicates that the perirhinal cortex is involved in the familiarity discrimination aspect of recognition memory. It has been previously shown under selective conditions that neural networks performing familiarity discrimination can achieve very high storage capacity, being able to deal with many times more stimuli than associative memory networks can in associative recall. The capacity of associative memories for recall has been shown to be highly dependent on the sparseness of coding. However, previous work on the networks of Bogacz et al, Norman and O'Reilly and Sohal and Hasselmo that model familiarity discrimination in the perirhinal cortex has not investigated the effects of the sparseness of encoding on capacity. This paper explores how sparseness of coding influences the capacity of each of these published models and establishes that sparse coding influences the capacity of the different models in different ways. The capacity of the Bogacz et al model can be made independent of the sparseness of coding. Capacity increases as coding becomes sparser for a simplified version of the neocortical part of the Norman and O'Reilly model, whereas capacity decreases as coding becomes sparser for a simplified version of the Sohal and Hasselmo model. Thus in general, and in contrast to associative memory networks, sparse encoding results in little or no advantage for the capacity of familiarity discrimination networks. Hence it may be less important for coding to be sparse in the perirhinal cortex than it is in the hippocampus. Additionally, it is established that the capacities of the networks are strongly dependent on the precise form of the learning rules (synaptic plasticity) used in the network. This finding indicates that the precise characteristics of synaptic plastic changes in the real brain are likely to have major influences on storage capacity.     

连续学习混沌神经网络的研究

近几年混沌神经网络在信息处理，特别是联想记忆中的应用得到了极大重视。本文提出了一个改进的连续学习混沌神经网络(MSLCNN)模型，它具有两个重要特征：(1)根据不同的输入，神经网络做出不同的响应，可从已知模式来识别未知模式；(2)可连续学习未知模式。计算机仿真表明我们的模型具有应用潜力。     

An autoassociative neural network model of paired-associate learning

Hebbian heteroassociative learning is inherently asymmetric. Storing a forward association, from item A to item B, enables recall of B (given A), but does not permit recall of A (given B). Recurrent networks can solve this problem by associating A to B and B back to A. In these recurrent networks, the forward and backward associations can be differentially weighted to account for asymmetries in recall performance. In the special case of equal strength forward and backward weights, these recurrent networks can be modeled as a single autoassociative network where A and B are two parts of a single, stored pattern. We analyze a general, recurrent neural network model of associative memory and examine its ability to fit a rich set of experimental data on human associative learning. The model fits the data significantly better when the forward and backward storage strengths are highly correlated than when they are less correlated. This network-based analysis of associative learning supports the view that associations between symbolic elements are better conceptualized as a blending of two ideas into a single unit than as separately modifiable forward and backward associations linking representations in memory.     

基于约束区域的连续时间联想记忆神经网络

传统的联想记忆神经网络模型是根据联想记忆点设计权值。文中提出一种根据联想记忆点设计基于约束区域的神经网络模型,它保证了渐近稳定的平衡点集与样要点集相同,不渐近稳定的平衡点恰为实际的拒识状态,并且吸引域分布合理。它具有学习和遗忘能力,还具有记忆容量大和电路可实现优点,是理想的联想记忆器。