Multivalued associative memories based on recurrent networks

A multivalued neural associative memory model based on a recurrent network structure is proposed. This model adopts the same principle proposed in the authors' previous work, the exponential correlation associative memories (ECAM). The model also has a very high storage capacity and strong error-correction capability. The major components of the new model include a weighted average process and some similarity-measure computation. As in ECAM, in order to enhance the differences among the weights and make the largest weights more overwhelming, the new model incorporates a nonlinear function in the calculation of weights. Several possible similarity measures suitable for this model are suggested. Simulation results of the performance of the new model with different measures show that, loaded with 500 64-component patterns, the model can sustain noise with power about one fifth to three fifths of the average signal power.     

Recurrent Correlation Associative Memories: A Feature Space Perspective

In this paper, we analyze a model of recurrent kernel associative memory (RKAM) recently proposed by Garcia and Moreno. We show that this model consists in a kernelization of the recurrent correlation associative memory (RCAM) of Chiueh and Goodman. In particular, using an exponential kernel, we obtain a generalization of the well-known exponential correlation associative memory (ECAM), while using a polynomial kernel, we obtain a generalization of higher order Hopfield networks with Hebbian weights. We show that the RKAM can outperform the aforementioned associative memory models, becoming equivalent to them when a dominance condition is fulfilled by the kernel matrix. To ascertain the dominance condition, we propose a statistical measure which can be easily computed from the probability distribution of the interpattern Hamming distance or directly estimated from the memory vectors. The RKAM can be used below saturation to realize associative memories with reduced dynamic range with respect to the ECAM and with reduced number of synaptic coefficients with respect to higher order Hopfield networks.     

Distance-Based Sparse Associative Memory Neural Network Algorithm for Pattern Recognition

A sparse two-Dimension distance weighted approach for improving the performance of exponential correlation associative memory (ECAM) and modified exponential correlation associative memory (MECAM) is presented in this paper. The approach is inspired by biological visual perception mechanism and extensively existing sparse small-world network phenomenon. By means of the approach, the two new associative memory neural networks, i.e., distance-based sparse ECAM (DBS-ECAM) and distance-based sparse MECAM (DBS-MECAM), are induced by introducing both the decaying two-Dimension distance factor and small-world architecture into ECAM and MECAM’s evolution rule for image processing application. Such a new configuration can reduce the connection complexity of conventional fully connected associative memories so that makes AM’ VLSI implementation easier. More importantly, the experiments performed on the binary visual images show DBS-ECAM and DBS-MECAM can learn and recognize patterns more effectively than ECAM and MECAM, respectively.     

A general model for bidirectional associative memories

This paper proposes a general model for bidirectional associative memories that associate patterns between the X-space and the Y-space. The general model does not require the usual assumption that the interconnection weight from a neuron in the X-space to a neuron in the Y-space is the same as the one from the Y-space to the X-space. We start by defining a supporting function to measure how well a state supports another state in a general bidirectional associative memory (GBAM). We then use the supporting function to formulate the associative recalling process as a dynamic system, explore its stability and asymptotic stability conditions, and develop an algorithm for learning the asymptotic stability conditions using the Rosenblatt perceptron rule. The effectiveness of the proposed model for recognition of noisy patterns and the performance of the model in terms of storage capacity, attraction, and spurious memories are demonstrated by some outstanding experimental results.     

DEMON-BASED ASSOCIATIVE MEMORIES

Associative memories are of two fundamental types, those that store representations of prototypical patterns (auto-associative memories)and those that store associations between pairs of arbitrary patterns (hetero-associative memories)Four network models of the latter type, each employing a single layer of linear threshold units are presented. Two of these models maintain fixed arrangements of their components. The other two are dynamically self-organizing. They employ feedback about performance to guide changes in the organization of their components. These models are evaluated in terms of storage capacity, error-tolerance, and storage space efficiency. One form of dynamic memory has the highest storage capacity of any known network model of associative memory. A discussion of models by Anderson and Hopfield and some implications of static and dynamic architectures conclude the paper.     

Storage Capacity of the Exponential Correlation Associative Memory

In this paper we analyze the pattern storage capacity of the exponential correlation associative memory (ECAM). This architecture was first studied by Chiueh and Goodman [3] who concluded that, under certain conditions on the input patterns, the memory has a storage capacity that was exponential in the length of the bit-patterns. A recent analysis by Pelillo and Hancock [9], using the Kanerva picture of recall, concluded that the storage capacity was limited by 2 ^N–1/N ² patterns. Both of these analyses can be criticised on the basis that they overlook the role of initial bit-errors in the recall process and deal only with the capacity for perfect pattern recall. In other words, they fail to model the effect of presenting corrupted patterns to the memory. This can be expected to lead to a more pessimistic limit. Here we model the performance of the ECAM when presented with corrupted input patterns. Our model leads to an expression for the storage capacity of the ECAM both in terms of the length of the bit-patterns and the probability of bit-corruption in the original input patterns. These storage capacities agree closely with simulation. In addition, our results show that slightly superior performance can be obtained by selecting an optimal value of the exponential constant.     

Design and Analysis of High-Capacity Associative Memories Based on a Class of Discrete-Time Recurrent Neural Networks

This paper presents a design method for synthesizing associative memories based on discrete-time recurrent neural networks. The proposed procedure enables both hetero- and autoassociative memories to be synthesized with high storage capacity and assured global asymptotic stability. The stored patterns are retrieved by feeding probes via external inputs rather than initial conditions. As typical representatives, discrete-time cellular neural networks (CNNs) designed with space-invariant cloning templates are examined in detail. In particular, it is shown that procedure herein can determine the input matrix of any CNN based on a space-invariant cloning template which involves only a few design parameters. Two specific examples and many experimental results are included to demonstrate the characteristics and performance of the designed associative memories.      

Gray-scale morphological associative memories

Neural models of associative memories are usually concerned with the storage and the retrieval of binary or bipolar patterns. Thus far, the emphasis in research on morphological associative memory systems has been on binary models, although a number of notable features of autoassociative morphological memories (AMMs) such as optimal absolute storage capacity and one-step convergence have been shown to hold in the general, gray-scale setting. In this paper, we make extensive use of minimax algebra to analyze gray-scale autoassociative morphological memories. Specifically, we provide a complete characterization of the fixed points and basins of attractions which allows us to describe the storage and recall mechanisms of gray-scale AMMs. Computer simulations using gray-scale images illustrate our rigorous mathematical results on the storage capacity and the noise tolerance of gray-scale morphological associative memories (MAMs). Finally, we introduce a modified gray-scale AMM model that yields a fixed point which is closest to the input pattern with respect to the Chebyshev distance and show how gray-scale AMMs can be used as classifiers.     

具有非线性映射能力的高性能联想记忆模型

通常的联想记忆模型的联想性能由于受到输入模式间交叉相关项的影响而有所下降，并且在输入与输出之间缺乏非线性映射能力。本文介绍一种高性能联想记忆模型，它将低维输入向量映射到一个高维的中间向量，从而提高了系统的联想能力，又使系统具有非线性映射能力，最后给出了几种推广。     

A Bidirectional Hetero-Associative Memory for True-Color Patterns

Classical bidirectional associative memories (BAM) have poor memory storage capacity, are sensitive to noise, are subject to spurious steady states during recall, and can only recall bipolar patterns. In this paper, we introduce a new bidirectional hetero-associative memory model for true-color patterns that uses the associative model with dynamical synapses recently introduced in Vazquez and Sossa (Neural Process Lett, Submitted, 2008). Synapses of the associative memory could be adjusted even after the training phase as a response to an input stimulus. Propositions that guarantee perfect and robust recall of the fundamental set of associations are provided. In addition, we describe the behavior of the proposed associative model under noisy versions of the patterns. At last, we present some experiments aimed to show the accuracy of the proposed model with a benchmark of true-color patterns.     

A 16-Kbit Θ-search associative memory

The design of a high-capacity Θ-search associative memory (Θ∈{<,>,⩽,⩾,=≠}) is presented. PSPICE simulation and layouts show that the proposed Θ-search associative memory chip consisting of 256 words, each 64-b long, can fit on a 13.5-mm×9.5-mm chip. It can perform maskable Θ-search operations over its contents in 110 ns     

The capacity of associative memories with malfunctioning neurons

Hopfield associative memories with αn malfunctioning neurons are considered. Using some facts from exchangeable events theory, the asymptotic storage capacity of such a network is derived as a function of the parameter α under stability and attractivity requirements. It is shown that the asymptotic storage capacity is (1-α)2n/(4 log n) under stability and (1-α)²(1-2ρ)²n/(4 log n) under attractivity requirements, respectively. Comparing these capacities with their maximum values corresponding to the case when there is no malfunctioning neurons, α=0, shows the robustness of the retrieval mechanism of Hopfield associative memories with respect to the existence of malfunctioning neurons. This result also supports the claim that neural networks are fault tolerant     

Exact associative neural memory dynamics utilizing Boolean matrices

The exact dynamics of shallow loaded associative neural memories are generated and characterized. The Boolean matrix analysis approach is employed for the efficient generation of all possible state transition trajectories for parallel updated binary-state dynamic associative memories (DAMs). General expressions for the size of the basin of attraction of fundamental and oscillatory memories and the number of oscillatory and stable states are derived for discrete synchronous Hopfield DAMs loaded with one, two, or three even-dimensionality bipolar memory vectors having the same mutual Hamming distances between them. Spurious memories are shown to occur only if the number of stored patterns exceeds two in an even-dimensionality Hopfield memory. The effects of odd- versus even-dimensionality memory vectors on DAM dynamics and the effects of memory pattern encoding on DAM performance are tested. An extension of the Boolean matrix dynamics characterization technique to other, more complex DAMs is presented.     

Implicative Fuzzy Associative Memories

Associative neural memories are models of biological phenomena that allow for the storage of pattern associations and the retrieval of the desired output pattern upon presentation of a possibly noisy or incomplete version of an input pattern. In this paper, we introduce implicative fuzzy associative memories (IFAMs), a class of associative neural memories based on fuzzy set theory. An IFAM consists of a network of completely interconnected Pedrycz logic neurons with threshold whose connection weights are determined by the minimum of implications of presynaptic and postsynaptic activations. We present a series of results for autoassociative models including one pass convergence, unlimited storage capacity and tolerance with respect to eroded patterns. Finally, we present some results on fixed points and discuss the relationship between implicative fuzzy associative memories and morphological associative memories     

Neural associative memory for brain modeling and information retrieval

This work concisely reviews and unifies the analysis of different variants of neural associative networks consisting of binary neurons and synapses (Willshaw model). We compute storage capacity, fault tolerance, and retrieval efficiency and point out problems of the classical Willshaw model such as limited fault tolerance and restriction to logarithmically sparse random patterns. Then we suggest possible solutions employing spiking neurons, compression of the memory structures, and additional cell layers. Finally, we discuss from a technical perspective whether distributed neural associative memories have any practical advantage over localized storage, e.g., in compressed look-up tables.     

Nonlinear quantization on Hebbian-type associative memories

Hebbian-type associative memory is characterized by its simple architecture. However, the hardware implementation of Hebbian-type associative memories is normally complicated when there are a huge number of patterns stored. To simplify the interconnection values of a network, a nonlinear quantization strategy is presented. The strategy takes into account the property that the interconnection values are Gaussian distributed, and divides the interconnection weight values into a small number of unequal ranges accordingly. Interconnection weight values in each range contain information equally and each range is quantized to a value.     

Hopfield网络中二元正交记忆模式的吸引域分析

在作为联想记忆的Hopfield网络中,二元正交记忆模式的分析对网络记忆容量的研究起着重要作用。文中提出了利用吸引指数的概念对各个二元正交记忆模式的吸引域进行估计的方法。理论分析和计算机仿真表明,当网络容量不超过0．33N时（比通常的0．15N要好）,每个二元正交记忆模式的吸引域至少包含一个汉明球。     

Multilayer associative neural networks (MANN''s): storage capacityversus perfect recall

The objective of this paper is to to resolve important issues in artificial neural nets-exact recall and capacity in multilayer associative memories. These problems have imposed restrictions on coding strategies. We propose the following triple-layered hybrid neural network: the first synapse is a one-shot associative memory using the modified Kohonen's adaptive learning algorithm with arbitrary input patterns; the second one is Kosko's bidirectional associative memory consisting of orthogonal input/output basis vectors such as Walsh series satisfying the strict continuity condition; and finally, the third one is a simple one-shot associative memory with arbitrary output images. A mathematical framework based on the relationship between energy local minima (capacity of the neural net) and noise-free recall is established. The robust capacity conditions of this multilayer associative neural network that lead to forming the local minima of the energy function at the exact training pairs are derived. The chosen strategy not only maximizes the total number of stored images but also completely relaxes any code-dependent conditions of the learning pairs.     

The Kosko Subsethood Fuzzy Associative Memory (KS-FAM): Mathematical Background and Applications in Computer Vision

Many well-known fuzzy associative memory (FAM) models can be viewed as (fuzzy) morphological neural networks (MNNs) because they perform an operation of (fuzzy) mathematical morphology at every node, possibly followed by the application of an activation function. The vast majority of these FAMs represent distributive models given by single-layer matrix memories. Although the Kosko subsethood FAM (KS-FAM) can also be classified as a fuzzy morphological associative memory (FMAM), the KS-FAM constitutes a two-layer non-distributive model. In this paper, we prove several theorems concerning the conditions of perfect recall, the absolute storage capacity, and the output patterns produced by the KS-FAM. In addition, we propose a normalization strategy for the training and recall phases of the KS-FAM. We employ this strategy to compare the error correction capabilities of the KS-FAM and other fuzzy and gray-scale associative memories in terms of some experimental results concerning gray-scale image reconstruction. Finally, we apply the KS-FAM to the task of vision-based self-localization in robotics.     

Behavioural study of median associative memory under true-colour image patterns

Median associative memories (MED-AMs) are a special type of associative memory that substitutes the maximum and minimum operators of a morphological associative memory with the median operator. This associative model has been applied to restore grey scale images and provided a better performance than morphological associative memories when the patterns are altered with mixed noise. Despite their power, MED-AMs have not been adopted in problems related with true-colour patterns. In this paper, we describe how MED-AMs can be applied to problems involving true-colour patterns. Furthermore, a complete study of the behaviour of this associative model in the restoration of true-colour images is performed using a benchmark of 16,000 images altered by different noise types.