An optimization approach to design of generalized BSB neural associative memories

This article is concerned with the synthesis of the optimally performing GBSB (generalized brain-state-in-a-box) neural associative memory given a set of desired binary patterns to be stored as asymptotically stable equilibrium points. Based on some known qualitative properties and newly observed fundamental properties of the GBSB model, the synthesis problem is formulated as a constrained optimization problem. Next, we convert this problem into a quasi-convex optimization problem called GEVP (generalized eigenvalue problem). This conversion is particularly useful in practice, because GEVPs can be efficiently solved by recently developed interior point methods. Design examples are given to illustrate the proposed approach and to compare with existing synthesis methods.     

On the design of BSB neural associative memories using semidefinite programming

This article is concerned with the reliable search for optimally performing BSB (brain state in a box) neural associative memories given a set of prototype patterns to be stored as stable equilibrium points. By converting and/or modifying the nonlinear constraints of a known formulation for the synthesis of BSB-based associative memories into linear matrix inequalities, we recast the synthesis into semidefinite programming problems and solve them by recently developed interior point methods. The validity of this approach is illustrated by a design example.     

Multivalued associative memories based on recurrent networks

A multivalued neural associative memory model based on a recurrent network structure is proposed. This model adopts the same principle proposed in the authors' previous work, the exponential correlation associative memories (ECAM). The model also has a very high storage capacity and strong error-correction capability. The major components of the new model include a weighted average process and some similarity-measure computation. As in ECAM, in order to enhance the differences among the weights and make the largest weights more overwhelming, the new model incorporates a nonlinear function in the calculation of weights. Several possible similarity measures suitable for this model are suggested. Simulation results of the performance of the new model with different measures show that, loaded with 500 64-component patterns, the model can sustain noise with power about one fifth to three fifths of the average signal power.     

Morphological associative memories

The theory of artificial neural networks has been successfully applied to a wide variety of pattern recognition problems. In this theory, the first step in computing the next state of a neuron or in performing the next layer neural network computation involves the linear operation of multiplying neural values by their synaptic strengths and adding the results. A nonlinear activation function usually follows the linear operation in order to provide for nonlinearity of the network and set the next state of the neuron. In this paper we introduce a novel class of artificial neural networks, called morphological neural networks, in which the operations of multiplication and addition are replaced by addition and maximum (or minimum), respectively. By taking the maximum (or minimum) of sums instead of the sum of products, morphological network computation is nonlinear before possible application of a nonlinear activation function. As a consequence, the properties of morphological neural networks are drastically different than those of traditional neural network models. The main emphasis of the research presented here is on morphological associative memories. We examine the computing and storage capabilities of morphological associative memories and discuss differences between morphological models and traditional semilinear models such as the Hopfield net.     

Gray-scale morphological associative memories

Neural models of associative memories are usually concerned with the storage and the retrieval of binary or bipolar patterns. Thus far, the emphasis in research on morphological associative memory systems has been on binary models, although a number of notable features of autoassociative morphological memories (AMMs) such as optimal absolute storage capacity and one-step convergence have been shown to hold in the general, gray-scale setting. In this paper, we make extensive use of minimax algebra to analyze gray-scale autoassociative morphological memories. Specifically, we provide a complete characterization of the fixed points and basins of attractions which allows us to describe the storage and recall mechanisms of gray-scale AMMs. Computer simulations using gray-scale images illustrate our rigorous mathematical results on the storage capacity and the noise tolerance of gray-scale morphological associative memories (MAMs). Finally, we introduce a modified gray-scale AMM model that yields a fixed point which is closest to the input pattern with respect to the Chebyshev distance and show how gray-scale AMMs can be used as classifiers.     

Bipolar spectral associative memories

Nonlinear spectral associative memories are proposed as quantized frequency domain formulations of nonlinear, recurrent associative memories in which volatile network attractors are instantiated by attractor waves. In contrast to conventional associative memories, attractors encoded in the frequency domain by convolution may be viewed as volatile online inputs, rather than nonvolatile, off-line parameters. Spectral memories hold several advantages over conventional associative memories, including decoder/attractor separability and linear scalability, which make them especially well suited for digital communications. Bit patterns may be transmitted over a noisy channel in a spectral attractor and recovered at the receiver by recurrent, spectral decoding. Massive nonlocal connectivity is realized virtually, maintaining high symbol-to-bit ratios while scaling linearly with pattern dimension. For n-bit patterns, autoassociative memories achieve the highest noise immunity, whereas heteroassociative memories offer the added flexibility of achieving various code rates, or degrees of extrinsic redundancy. Due to linear scalability, high noise immunity and use of conventional building blocks, spectral associative memories hold much promise for achieving robust communication systems. Simulations are provided showing bit error rates for various degrees of decoding time, computational oversampling, and signal-to-noise ratio.     

Recurrent correlation associative memories

A model for a class of high-capacity associative memories is presented. Since they are based on two-layer recurrent neural networks and their operations depend on the correlation measure, these associative memories are called recurrent correlation associative memories (RCAMs). The RCAMs are shown to be asymptotically stable in both synchronous and asynchronous (sequential) update modes as long as their weighting functions are continuous and monotone nondecreasing. In particular, a high-capacity RCAM named the exponential correlation associative memory (ECAM) is proposed. The asymptotic storage capacity of the ECAM scales exponentially with the length of memory patterns, and it meets the ultimate upper bound for the capacity of associative memories. The asymptotic storage capacity of the ECAM with limited dynamic range in its exponentiation nodes is found to be proportional to that dynamic range. Design and fabrication of a 3-mm CMOS ECAM chip is reported. The prototype chip can store 32 24-bit memory patterns, and its speed is higher than one associative recall operation every 3 mus. An application of the ECAM chip to vector quantization is also described.     

利用动态核的形态联想记忆网络的研究

在文献[1]的基础上,提出了一个基于动态核的形态联想记忆网络方法,特点是同一幅图像,如果其所含的噪声情况不同,则其核也将不同,从而较好地解决了图像含有随机噪声时的联想记忆问题。实验证明,此方法具有良好的性能,双向联想记忆的准确率优于文献[1]中介绍的方法。     

Oscillatory neural associative memories with synapses based on memristor bridges

An approach to the implementation of electronic associative memories with tunable weights based on the resistor bridges containing memristors—a bidirectional associative memory (BAM) and an associative memory based on the Hopfield network—is proposed. These memories we implement as a networks of coupled phase oscillators. The conditions for the use of the operational amplifier in a comparator mode for implementing the step activation function are determined. It is shown how to use the CMOS transistors switches to control the memristance value. The experiments using LTSPICE models show that for the reference binary images with size 3 × 3 the proposed networks converges to the reference images (and, accordingly, to their inversion) with a random uniform distribution of binary pixel values of the input images. In all experiments we have no error states in spite of the number of reference patterns exceeds the classical estimations for traditional BAM and Hopfield networks.     

Volume quadric hologram-based associative memories

The class of associative memories based on volume nonlinearly recorded holograms is represented. It is shown that the quadratic nonlinearity of an amplitude response of a volume hologram recorded with a combined reference wave imparts to the hologram specific phase conjugate properties, which can be used for implementation of error-correcting auto- and heteroassociative memories. Within the quadric hologram approximation, we elaborate and demonstrate the architectures both for self-conjugate autoassociative reconstruction with a true tone rendering and for crosstalk-free heteroassociative reconstruction of complex optical singals accumulated at one carrier without interference.