Optimal bidirectional associative memories

This paper proposes a model for linear optimal bidirectional associative memories (BAMs), in which the design of the weight matrix is based on a least-squares cost functional. The minimization of the functional leads to a Lyapunov-type matrix equation. The mathematical properties of such an optimal bidirectional system are studied in detail. Simulations are conducted for the bidirectional association of pattern pairs with grey levels. It is shown that this optimal design of BAM is effective for memorizing patterns with grey levels.     

Representation of concept lattices by bidirectional associative memories

This article presents a concept interpretation of patterns for bidirectional associative memory (BAM) and a representation of hierarchical structures of concepts (concept lattices) by BAMs. The constructive representation theorem provides a storing rule for a training set that allows a concept interpretation. Examples demonstrating the theorems are presented.     

Orthogonal schemes for bidirectional associative memories

Two issues are addressed in this paper. Firstly, it investigates some important properties of bidirectional associative memories (BAM) and proposes an improved capacity estimate. Those properties are the encoding form of the input pattern pairs as sell as their decoding, the orthogonality of the pattern pairs, the similarity of associated patterns, and the density of the pattern pairs. Secondly, it proposes an implementation approach to improve the storage capacity. The approach embraces three proposed methods, i.e., the bipolar-orthogonal augmentation, the set partition, and the combined method. Along with those methods is the construction of the set of bipolar orthogonal patterns.     

A general model for bidirectional associative memories

This paper proposes a general model for bidirectional associative memories that associate patterns between the X-space and the Y-space. The general model does not require the usual assumption that the interconnection weight from a neuron in the X-space to a neuron in the Y-space is the same as the one from the Y-space to the X-space. We start by defining a supporting function to measure how well a state supports another state in a general bidirectional associative memory (GBAM). We then use the supporting function to formulate the associative recalling process as a dynamic system, explore its stability and asymptotic stability conditions, and develop an algorithm for learning the asymptotic stability conditions using the Rosenblatt perceptron rule. The effectiveness of the proposed model for recognition of noisy patterns and the performance of the model in terms of storage capacity, attraction, and spurious memories are demonstrated by some outstanding experimental results.     

Alpha–Beta bidirectional associative memories: theory and applications

In this work a new Bidirectional Associative Memory model, surpassing every other past and current model, is presented. This new model is based on Alpha–Beta associative memories, from whom it inherits its name. The main and most important characteristic of Alpha–Beta bidirectional associative memories is that they exhibit perfect recall of all patterns in the fundamental set, without requiring the fulfillment of any condition. The capacity they show is 2^min(n,m), being n and m the input and output patterns dimensions, respectively. Design and functioning of this model are mathematically founded, thus demonstrating that pattern recall is always perfect, with no regard to the trained pattern characteristics, such as linear independency, orthogonality, or Hamming distance. Two applications illustrating the optimal functioning of the model are shown: a translator and a fingerprint identifier.     

An efficient learning algorithm for associative memories

Associative memories (AMs) can be implemented using networks with or without feedback. We utilize a two-layer feedforward neural network and propose a learning algorithm that efficiently implements the association rule of a bipolar AM. The hidden layer of the network employs p neurons where p is the number of prototype patterns. In the first layer, the input pattern activates at most one hidden layer neuron or "winner". In the second layer, the "winner" associates the input pattern to the corresponding prototype pattern. The underlying association principle is minimum Hamming distance and the proposed scheme can be viewed also as an approximately minimum Hamming distance decoder. Theoretical analysis supported by simulations indicates that, in comparison with other suboptimum minimum Hamming distance association schemes, the proposed structure exhibits the following favorable characteristics: 1) it operates in one-shot which implies no convergence-time requirements; 2) it does not require any feedback; and 3) our case studies show that it exhibits superior performance to the popular linear system in a saturated mode. The network also exhibits 4) exponential capacity and 5) easy performance assessment (no asymptotic analysis is necessary). Finally, since it does not require any hidden layer interconnections or tree-search operations, it exhibits low structural as well as operational complexity.     

Fast access to concepts in concept lattices via bidirectional associative memories

Bidirectional associative memories (BAMs) are shown to be capable of precisely learning concept lattice structures by Radim Belohlávek. The focus of this letter is to show that the BAM, when set up with a concept lattice by setting up connection weights according to the rule proposed by Belohlávek, always returns the most specific or most generic concept containing the given set of objects or attributes when a set of objects or attributes is presented as input to the object or attribute layer. A proof of this property is given here, together with an example, and a brief application of the property is provided.     

Morphological associative memories

The theory of artificial neural networks has been successfully applied to a wide variety of pattern recognition problems. In this theory, the first step in computing the next state of a neuron or in performing the next layer neural network computation involves the linear operation of multiplying neural values by their synaptic strengths and adding the results. A nonlinear activation function usually follows the linear operation in order to provide for nonlinearity of the network and set the next state of the neuron. In this paper we introduce a novel class of artificial neural networks, called morphological neural networks, in which the operations of multiplication and addition are replaced by addition and maximum (or minimum), respectively. By taking the maximum (or minimum) of sums instead of the sum of products, morphological network computation is nonlinear before possible application of a nonlinear activation function. As a consequence, the properties of morphological neural networks are drastically different than those of traditional neural network models. The main emphasis of the research presented here is on morphological associative memories. We examine the computing and storage capabilities of morphological associative memories and discuss differences between morphological models and traditional semilinear models such as the Hopfield net.     

Gray-scale morphological associative memories

Neural models of associative memories are usually concerned with the storage and the retrieval of binary or bipolar patterns. Thus far, the emphasis in research on morphological associative memory systems has been on binary models, although a number of notable features of autoassociative morphological memories (AMMs) such as optimal absolute storage capacity and one-step convergence have been shown to hold in the general, gray-scale setting. In this paper, we make extensive use of minimax algebra to analyze gray-scale autoassociative morphological memories. Specifically, we provide a complete characterization of the fixed points and basins of attractions which allows us to describe the storage and recall mechanisms of gray-scale AMMs. Computer simulations using gray-scale images illustrate our rigorous mathematical results on the storage capacity and the noise tolerance of gray-scale morphological associative memories (MAMs). Finally, we introduce a modified gray-scale AMM model that yields a fixed point which is closest to the input pattern with respect to the Chebyshev distance and show how gray-scale AMMs can be used as classifiers.     

Bipolar spectral associative memories

Nonlinear spectral associative memories are proposed as quantized frequency domain formulations of nonlinear, recurrent associative memories in which volatile network attractors are instantiated by attractor waves. In contrast to conventional associative memories, attractors encoded in the frequency domain by convolution may be viewed as volatile online inputs, rather than nonvolatile, off-line parameters. Spectral memories hold several advantages over conventional associative memories, including decoder/attractor separability and linear scalability, which make them especially well suited for digital communications. Bit patterns may be transmitted over a noisy channel in a spectral attractor and recovered at the receiver by recurrent, spectral decoding. Massive nonlocal connectivity is realized virtually, maintaining high symbol-to-bit ratios while scaling linearly with pattern dimension. For n-bit patterns, autoassociative memories achieve the highest noise immunity, whereas heteroassociative memories offer the added flexibility of achieving various code rates, or degrees of extrinsic redundancy. Due to linear scalability, high noise immunity and use of conventional building blocks, spectral associative memories hold much promise for achieving robust communication systems. Simulations are provided showing bit error rates for various degrees of decoding time, computational oversampling, and signal-to-noise ratio.     

Recurrent correlation associative memories

A model for a class of high-capacity associative memories is presented. Since they are based on two-layer recurrent neural networks and their operations depend on the correlation measure, these associative memories are called recurrent correlation associative memories (RCAMs). The RCAMs are shown to be asymptotically stable in both synchronous and asynchronous (sequential) update modes as long as their weighting functions are continuous and monotone nondecreasing. In particular, a high-capacity RCAM named the exponential correlation associative memory (ECAM) is proposed. The asymptotic storage capacity of the ECAM scales exponentially with the length of memory patterns, and it meets the ultimate upper bound for the capacity of associative memories. The asymptotic storage capacity of the ECAM with limited dynamic range in its exponentiation nodes is found to be proportional to that dynamic range. Design and fabrication of a 3-mm CMOS ECAM chip is reported. The prototype chip can store 32 24-bit memory patterns, and its speed is higher than one associative recall operation every 3 mus. An application of the ECAM chip to vector quantization is also described.     

A feedforward bidirectional associative memory

In contrast to conventional feedback bidirectional associative memory (BAM) network models, a feedforward BAM network is developed based on a one-shot design algorithm of O(p(2)(n+m)) computational complexity, where p is the number of prototype pairs and n, m are the dimensions of the input/output bipolar vectors. The feedforward BAM is an n-p-m three-layer network of McCulloch-Pitts neurons with storage capacity 2(min{m,n}) and guaranteed perfect bidirectional recall. The overall network design procedure is fully scalable in the sense that any number p=/<2(min{m,n}) of bidirectional associations can be implemented. The prototype patterns may be arbitrarily correlated. With respect to inference performance, it is shown that the Hamming attractive radius of each prototype reaches the maximum possible value. Simulation studies and comparisons illustrate and support these theoretical developments.     

Volume quadric hologram-based associative memories

The class of associative memories based on volume nonlinearly recorded holograms is represented. It is shown that the quadratic nonlinearity of an amplitude response of a volume hologram recorded with a combined reference wave imparts to the hologram specific phase conjugate properties, which can be used for implementation of error-correcting auto- and heteroassociative memories. Within the quadric hologram approximation, we elaborate and demonstrate the architectures both for self-conjugate autoassociative reconstruction with a true tone rendering and for crosstalk-free heteroassociative reconstruction of complex optical singals accumulated at one carrier without interference.     

Noise performance of linear associative memories

The performance of two commonly used linear models of associative memories, generalized inverse (GI) and correlation matrix memory (CMM) is studied analytically in the presence of a new type of noise (training noise due to noisy training patterns). Theoretical expressions are determined for the S/N ratio gain of the GI and CMM memories in the auto-associative and hetero-associative modes of operation. It is found that the GI method performance degrades significantly in the presence of training noise while the CMM method is relatively unaffected by it. The theoretical expressions are plotted and compared with the results obtained from Monte Carlo simulations and the two are found to be in excellent agreement     

Nonlinear quantization on Hebbian-type associative memories

Hebbian-type associative memory is characterized by its simple architecture. However, the hardware implementation of Hebbian-type associative memories is normally complicated when there are a huge number of patterns stored. To simplify the interconnection values of a network, a nonlinear quantization strategy is presented. The strategy takes into account the property that the interconnection values are Gaussian distributed, and divides the interconnection weight values into a small number of unequal ranges accordingly. Interconnection weight values in each range contain information equally and each range is quantized to a value.     

On new fuzzy morphological associative memories

In this paper, the new fuzzy morphological associative memories (FMAMs) based on fuzzy operations (/spl and/,/spl middot/) and (/spl or/,/spl middot/) are presented. FMAM with (/spl or/,/spl middot/) is extremely robust for dilative noise and FMAM with (/spl and/,/spl middot/) is extremely robust for erosive noise. Autoassociative FMAM has the unlimited storage capability and can converge in one step. The convex autoassociative FMAM can be used to achieve a reasonable tradeoff for the mixed noise. Finally, comparisons between autoassociative FMAM and the famous FAM are discussed. FMAM, as another new encoding way of fuzzy rules, still has a multitude of open problems worthy to explore in the future.     

Permutation-based finite implicative fuzzy associative memories

Implicative fuzzy associative memories (IFAMs) are single layer feedforward fuzzy neural networks whose synaptic weights and threshold values are given by implicative fuzzy learning. Despite an excellent tolerance with respect to either pasitive or negative noise, IFAMs are not suited for patterns corrupted by mixed noise. This paper presents a solution to this problem. Precisely, we first introduce the class of finite IFAMs by replacing the unit interval by a finite chain L. Then, we generalize both finite IFAMs and their dual versions by means of a permutation on L. The resulting models are referred to as permutation-based finite IFAMs (π-IFAMs). We show that a π-IFAM can be viewed as a finite IFAM, but defined on an alternative lattice structure (L,?). Thus, π-IFAMs also exhibit optimal absolute storage capacity and one step convergence in the autoassociative case. Furthermore, computational experiments revealed that a certain π-IFAM, called Lukasiewicz π_μ-IFAM, outperformed several other associative memory models for the reconstruction of gray-scale patterns corrupted by salt and pepper noise.     

Delay-independent stability in bidirectional associative memorynetworks

It is shown that if the neuronal gains are small compared with the synaptic connection weights, then a bidirectional associative memory network with axonal signal transmission delays converges to the equilibria associated with exogenous inputs to the network. Both discrete and continuously distributed delays are considered; the asymptotic stability is global in the state space of neuronal activations and also is independent of the delays.     

Adaptive associative memories capable of pattern segmentation

This paper presents an adaptive type of associative memory (AAM) that can separate patterns from composite inputs which might be degraded by deficiency or noise and that can recover incomplete or noisy single patterns. The behavior of AAM is analyzed in terms of stability, giving the stable solutions (results of recall), and the recall of spurious memories (the undesired solutions) is shown to be greatly reduced compared with earlier types of associative memory that can perform pattern segmentation. Two conditions that guarantee the nonexistence of undesired solutions are also given. Results of computer experiments show that the performance of AAM is much better than that of the earlier types of associative memory in terms of pattern segmentation and pattern recovery.     

具有抗随机噪声能力的联想记忆网络

形态联想记忆网络具有良好的联想记忆功能和较强的抗膨胀或腐蚀噪声能力,但抗混合噪声的能力很弱.而在实际中,随机噪声往往是混合型的,既有膨胀又有腐蚀噪声,将尺度空间和形态联想记忆网络相结合,得到了一种新的联想记忆网络,它提高了形态自联想记忆网络的抗随机噪声能力.通过仿真实验验证了该方法具有良好的性能.