A neural network for visual pattern recognition

A model of a neural network is presented that offers insight into the brain's complex mechanisms as well as design principles for information processors. The model has properties and abilities that most modern computers and pattern recognizers do not possess; pattern recognition, selective attention, segmentation, and associative recall. When a composite stimulus consisting of two or more patterns is presented, the model pays selective attention to each of the patterns one after the other, segments a pattern from the rest, and recognizes it separately in contrast to earlier models. This model has perfect associative recall, even for deformed patterns, without regard to their positions. It can be trained to recognize any set of patterns     

Associative Gray Level Pattern Processing using Binary Decomposition and α β Memories

In this note we show how a binary memory can be used to recall gray-level patterns. We take as example the α β associative memories recently proposed in Yáñez, Associative Memories based on order Relations and Binary Operators(In Spanish), PhD Thesis, Center for computing Research, February of 2002, only useful in the binary case. Basically, the idea consists on that given a set of gray-level patterns to be first memorized: (1) Decompose them into their corresponding binary patterns, and (2) Build the corresponding binary associative memory (one memory for each binary layer) with each training pattern set (by layers). A given pattern or a distorted version of it, it is recalled in three steps: (1) Decomposition of the pattern by layers into its binary patterns, (2) Recalling of each one of its binary components, layer by layer also, and (3) Reconstruction of the pattern from the binary patterns already recalled in step 2. The proposed methodology operates at two phases: training and recalling. Conditions for perfect recall of a pattern either from the fundamental set or from a distorted version of one them are also given. Experiments where the efficiency of the proposal is tested are also given.     

Scene analysis by integrating primitive segmentation andassociative memory

Scene analysis is a major aspect of perception and continues to challenge machine perception. This paper addresses the scene-analysis problem by integrating a primitive segmentation stage with a model of associative memory. The model is a multistage system that consists of an initial primitive segmentation stage, a multimodule associative memory, and a short-term memory (STM) layer. Primitive segmentation is performed by a locally excitatory globally inhibitory oscillator network (LEGION), which segments the input scene into multiple parts that correspond to groups of synchronous oscillations. Each segment triggers memory recall and multiple recalled patterns then interact with one another in the STM layer. The STM layer projects to the LEGION network, giving rise to memory-based grouping and segmentation. The system achieves scene analysis entirely in phase space, which provides a unifying mechanism for both bottom-up analysis and top-down analysis. The model is evaluated with a systematic set of three-dimensional (3-D) line drawing objects, which are arranged in an arbitrary fashion to compose input scenes that allow object occlusion. Memory-based organization is responsible for a significant improvement in performance. A number of issues are discussed, including input-anchored alignment, top-down organization, and the role of STM in producing context sensitivity of memory recall.     

A Bidirectional Hetero-Associative Memory for True-Color Patterns

Classical bidirectional associative memories (BAM) have poor memory storage capacity, are sensitive to noise, are subject to spurious steady states during recall, and can only recall bipolar patterns. In this paper, we introduce a new bidirectional hetero-associative memory model for true-color patterns that uses the associative model with dynamical synapses recently introduced in Vazquez and Sossa (Neural Process Lett, Submitted, 2008). Synapses of the associative memory could be adjusted even after the training phase as a response to an input stimulus. Propositions that guarantee perfect and robust recall of the fundamental set of associations are provided. In addition, we describe the behavior of the proposed associative model under noisy versions of the patterns. At last, we present some experiments aimed to show the accuracy of the proposed model with a benchmark of true-color patterns.     

Second-order asymmetric BAM design with a maximal basin of attraction

Bidirectional associative memory (BAM) generalizes the associative memory (AM) to be capable of performing two-way recalling of pattern pairs. Asymmetric bidirectional associative memory (ABAM) is a variant of BAM relaxed with connection weight symmetry restriction and enjoys a much better performance than a conventional BAM structure. Higher-order associative memories (HOAMs) are reputed for their higher memory capacity than the first-order counterparts. The paper concerns the design of a second-order asymmetric bidirectional associative memory (SOABAM) with a maximal basin of attraction, whose extension to a HOABAM is possible and straightforward. First, a necessary and sufficient condition is derived for the connection weight matrix of SOABAM that can guarantee the recall of all prototype pattern pairs. A local training rule which is adaptive in the learning step size is formulated. Then derived is a theorem, designing a SOABAM further enlarging the quantities required to meet the complete recall theorem will enhance the capability of evolving a noisy pattern to converge to its association pattern vector without error. Based on this theorem, our algorithm is also modified to ensure each training pattern is stored with a basin of attraction as large as possible.     

Storage Capacity of the Exponential Correlation Associative Memory

In this paper we analyze the pattern storage capacity of the exponential correlation associative memory (ECAM). This architecture was first studied by Chiueh and Goodman [3] who concluded that, under certain conditions on the input patterns, the memory has a storage capacity that was exponential in the length of the bit-patterns. A recent analysis by Pelillo and Hancock [9], using the Kanerva picture of recall, concluded that the storage capacity was limited by 2 ^N–1/N ² patterns. Both of these analyses can be criticised on the basis that they overlook the role of initial bit-errors in the recall process and deal only with the capacity for perfect pattern recall. In other words, they fail to model the effect of presenting corrupted patterns to the memory. This can be expected to lead to a more pessimistic limit. Here we model the performance of the ECAM when presented with corrupted input patterns. Our model leads to an expression for the storage capacity of the ECAM both in terms of the length of the bit-patterns and the probability of bit-corruption in the original input patterns. These storage capacities agree closely with simulation. In addition, our results show that slightly superior performance can be obtained by selecting an optimal value of the exponential constant.     

Double-pattern associative memory neural network with pattern loop

A double-pattern associative memory neural network with “pattern loop” is proposed. It can store 2N bit bipolar binary patterns up to the order of 2^2N , retrieve part or all of the stored patterns which all have the minimum Hamming distance with input pattern, completely eliminate spurious patterns, and has higher storing efficiency and reliability than conventional associative memory. The length of a pattern stored in this associative memory can be easily extended from 2N to κN.     

Quantum Pattern Recognition

I review and expand the model of quantum associative memory that I have recently proposed. In this model binary patterns of n bits are stored in the quantum superposition of the appropriate subset of the computational basis of n qbits. Information can be retrieved by performing an input-dependent rotation of the memory quantum state within this subset and measuring the resulting state. The amplitudes of this rotated memory state are peaked on those stored patterns which are closest in Hamming distance to the input, resulting in a high probability of measuring a memory pattern very similar to it. The accuracy of pattern recall can be tuned by adjusting a parameter playing the role of an effective temperature. This model solves the well-known capacity shortage problem of classical associative memories, providing a large improvement in capacity. PACS: 03.67.-a     

The Kosko Subsethood Fuzzy Associative Memory (KS-FAM): Mathematical Background and Applications in Computer Vision

Many well-known fuzzy associative memory (FAM) models can be viewed as (fuzzy) morphological neural networks (MNNs) because they perform an operation of (fuzzy) mathematical morphology at every node, possibly followed by the application of an activation function. The vast majority of these FAMs represent distributive models given by single-layer matrix memories. Although the Kosko subsethood FAM (KS-FAM) can also be classified as a fuzzy morphological associative memory (FMAM), the KS-FAM constitutes a two-layer non-distributive model. In this paper, we prove several theorems concerning the conditions of perfect recall, the absolute storage capacity, and the output patterns produced by the KS-FAM. In addition, we propose a normalization strategy for the training and recall phases of the KS-FAM. We employ this strategy to compare the error correction capabilities of the KS-FAM and other fuzzy and gray-scale associative memories in terms of some experimental results concerning gray-scale image reconstruction. Finally, we apply the KS-FAM to the task of vision-based self-localization in robotics.     

CLAM: A new model of associative memory

We present a new associative memory model that stores arbitrary bipolar patterns without the problems we can find in other models like BAM or LAM. After identifying those problems we show the new memory topology and we explain its learning and recall stages. Mathematical demonstrations are provided to prove that the new memory model guarantees the perfect retrieval of every stored pattern and also to prove that whatever the input of the memory is, it operates as a nearest neighbor classifier. ©2000 John Wiley & Sons, Inc.     

A bidirectional heteroassociative memory for binary and grey-level patterns

Typical bidirectional associative memories (BAM) use an offline, one-shot learning rule, have poor memory storage capacity, are sensitive to noise, and are subject to spurious steady states during recall. Recent work on BAM has improved network performance in relation to noisy recall and the number of spurious attractors, but at the cost of an increase in BAM complexity. In all cases, the networks can only recall bipolar stimuli and, thus, are of limited use for grey-level pattern recall. In this paper, we introduce a new bidirectional heteroassociative memory model that uses a simple self-convergent iterative learning rule and a new nonlinear output function. As a result, the model can learn online without being subject to overlearning. Our simulation results show that this new model causes fewer spurious attractors when compared to others popular BAM networks, for a comparable performance in terms of tolerance to noise and storage capacity. In addition, the novel output function enables it to learn and recall grey-level patterns in a bidirectional way.     

混沌神经网络在分离叠加模式和多对多联想记忆中的应用

混沌是不含外加随机因素的完全确定性的系统表现出来的界于规则和随机之间的内秉随机行为。脑神经系统是由神经细胞组成的网络。类似于人脑思维的人工神经网络与冯·诺依曼计算机相比,在信息处理方面有很大的优越性。混沌和神经网络相互融合的研究是从90年代开始的,其主要的目标是通过分析大脑的混沌现象,建立含有混沌动力学的神经网络模型(即混沌神经网络模型),将混沌的遍历性、对初始值敏感等特点与神经网络的非线性、自适应、并行处理优势相结合,     

Recurrent correlation associative memories

A model for a class of high-capacity associative memories is presented. Since they are based on two-layer recurrent neural networks and their operations depend on the correlation measure, these associative memories are called recurrent correlation associative memories (RCAMs). The RCAMs are shown to be asymptotically stable in both synchronous and asynchronous (sequential) update modes as long as their weighting functions are continuous and monotone nondecreasing. In particular, a high-capacity RCAM named the exponential correlation associative memory (ECAM) is proposed. The asymptotic storage capacity of the ECAM scales exponentially with the length of memory patterns, and it meets the ultimate upper bound for the capacity of associative memories. The asymptotic storage capacity of the ECAM with limited dynamic range in its exponentiation nodes is found to be proportional to that dynamic range. Design and fabrication of a 3-mm CMOS ECAM chip is reported. The prototype chip can store 32 24-bit memory patterns, and its speed is higher than one associative recall operation every 3 mus. An application of the ECAM chip to vector quantization is also described.     

A new paradigm for real-time parallel storage and recognition of patterns based on a hierarchical organization of associative memories utilizing Walsh function encoding

A new hierarchical Walsh memory which can store and quickly recognize any number of patterns is proposed. A Walsh function based associative memory was found to be capable of storing and recognizing patterns in parallel via purely a software algorithmic technique (namely, without resorting to parallel hardware) while the memory itself only takes a single pattern space of computer memory, due to the Walsh encoding of each pattern. This type of distributed associative memory lends itself to high speed pattern recognition and has been reported earlier in a single memory version. In this paper, the single memory concept has first been extended to a parallel memory module and then to a tree-shaped hierarchy of these parallel modules that are capable of storing and recognizing any number of patterns for practical large scale data applications exemplified by image and speech recognition. The memory hierarchy was built by successively applying k-means clustering to the training data set. In the proposed architecture, the clustered data subsets are stored respectively into a parallel memory module where the module allocation is optimized using the genetic algorithm to realize a minimal implementation of the memory structure. The system can recognize all the training patterns with 100% accuracy and further, can also generalize on similar data. In order to demonstrate its efficacy with large scale real world data, we stored and recognized over 500 faces while at same time, achieving much reduced recognition time and storage space than template matching.     

多值指数式多向联想记忆模型

多向联想记忆MDAM(multidirectional associative memory)模型是Kosko双向联想记忆模型BAM(bidirectional associative memory)的一个直接推广,它可应用于数据融合及维数分裂,使模型能处理大维数输入问题.目前所提出的若干种多向模型均局限于二值输入/输出模式对,但如在图象处理等的实际应用中,所处理的模式均是多值的.本文的目的就是提出一个多值指数式多向联想记忆模型MVeMDAM(multivalued exponential multidi     

A study of pattern recovery in recurrent correlation associative memories

In this paper, we analyze the recurrent correlation associative memory (RCAM) model of Chiueh and Goodman (1990, 1991). This is an associative memory in which stored binary memory patterns are recalled via an iterative update rule. The update of the individual pattern-bits is controlled by an excitation function, which takes as its argument the inner product between the stored memory patterns and the input patterns. Our contribution is to analyze the dynamics of pattern recall when the input patterns are corrupted by noise of a relatively unrestricted class. We show how to identify the excitation function which maximizes the separation (the Fisher discriminant) between the uncorrupted realization of the noisy input pattern and the remaining patterns residing in the memory. The excitation function which gives maximum separation is exponential when the input bit-errors follow a binomial distribution. We develop an expression for the expectation value of bit-error probability on the input pattern after one iteration. We show how to identify the excitation function which minimizes the bit-error probability. The relationship between the excitation functions which result from the two different approaches is examined for a binomial distribution of bit-errors. We develop a semiempirical approach to the modeling of the dynamics of the RCAM.     

BAM Learning of Nonlinearly Separable Tasks by Using an Asymmetrical Output Function and Reinforcement Learning

Most bidirectional associative memory (BAM) networks use a symmetrical output function for dual fixed-point behavior. In this paper, we show that by introducing an asymmetry parameter into a recently introduced chaotic BAM output function, prior knowledge can be used to momentarily disable desired attractors from memory, hence biasing the search space to improve recall performance. This property allows control of chaotic wandering, favoring given subspaces over others. In addition, reinforcement learning can then enable a dual BAM architecture to store and recall nonlinearly separable patterns. Our results allow the same BAM framework to model three different types of learning: supervised, reinforcement, and unsupervised. This ability is very promising from the cognitive modeling viewpoint. The new BAM model is also useful from an engineering perspective; our simulations results reveal a notable overall increase in BAM learning and recall performances when using a hybrid model with the general regression neural network (GRNN).      

最小平方形态联想记忆模型的研究与实现

Ritter等人借助形态学理论提出了形态联想记忆模型（MAM Morphological Associatvie Memory)，其中所构建模型的两个权值矩阵和可分别用以回忆腐蚀和膨胀噪声模式，但不能回忆混合噪声模式，故本文提出了一个最小平方形态联想记忆模式（LSMAM least squares MAM)来克服MAM的不足，以达到既可分别识别腐蚀和膨胀噪声模式，也可以识别混合型噪声模式的目的，因此更适用于实际情形，实验结果表明了该方法的可行性。     

Behavioural study of median associative memory under true-colour image patterns

Median associative memories (MED-AMs) are a special type of associative memory that substitutes the maximum and minimum operators of a morphological associative memory with the median operator. This associative model has been applied to restore grey scale images and provided a better performance than morphological associative memories when the patterns are altered with mixed noise. Despite their power, MED-AMs have not been adopted in problems related with true-colour patterns. In this paper, we describe how MED-AMs can be applied to problems involving true-colour patterns. Furthermore, a complete study of the behaviour of this associative model in the restoration of true-colour images is performed using a benchmark of 16,000 images altered by different noise types.     

基于认知的联想记忆仿真研究

在对联想记忆神经网络的研究中,为提高现有联想记忆网络的存储能力以及相似模式和多值模式的联想成功率,提出了一种新的联想记忆网络。样本模式信息存储在动态权值矩阵中,网络根据不同的输入模式可自适应地调节当前权值矩阵。与传统联想网络相比,输入模式的信息不仅给出了联想记忆的初值,且在联想记忆过程中起到启发式搜索的作用,使网络的存储能力和联想成功率得到较好的改善。尤其可以有效地实现相似模式以及多值模式的联想记忆功能。仿真结果验证了方法的有效性。