A feedforward bidirectional associative memory

In contrast to conventional feedback bidirectional associative memory (BAM) network models, a feedforward BAM network is developed based on a one-shot design algorithm of O(p(2)(n+m)) computational complexity, where p is the number of prototype pairs and n, m are the dimensions of the input/output bipolar vectors. The feedforward BAM is an n-p-m three-layer network of McCulloch-Pitts neurons with storage capacity 2(min{m,n}) and guaranteed perfect bidirectional recall. The overall network design procedure is fully scalable in the sense that any number p=/<2(min{m,n}) of bidirectional associations can be implemented. The prototype patterns may be arbitrarily correlated. With respect to inference performance, it is shown that the Hamming attractive radius of each prototype reaches the maximum possible value. Simulation studies and comparisons illustrate and support these theoretical developments.     

A bidirectional heteroassociative memory for binary and grey-level patterns

Typical bidirectional associative memories (BAM) use an offline, one-shot learning rule, have poor memory storage capacity, are sensitive to noise, and are subject to spurious steady states during recall. Recent work on BAM has improved network performance in relation to noisy recall and the number of spurious attractors, but at the cost of an increase in BAM complexity. In all cases, the networks can only recall bipolar stimuli and, thus, are of limited use for grey-level pattern recall. In this paper, we introduce a new bidirectional heteroassociative memory model that uses a simple self-convergent iterative learning rule and a new nonlinear output function. As a result, the model can learn online without being subject to overlearning. Our simulation results show that this new model causes fewer spurious attractors when compared to others popular BAM networks, for a comparable performance in terms of tolerance to noise and storage capacity. In addition, the novel output function enables it to learn and recall grey-level patterns in a bidirectional way.     

Optimization of stochastic networks using simulated annealing for the storage and recalling of compressed images using SOM

In this paper we are studying the optimization of Stochastic Hopfield neural network and the hybrid SOM–Hopfield neural network for the storage and recalling of fingerprint images. The feature extraction of these images has been performed using FFT, DWT and SOM. The feature vectors are stored in the Hopfield network with Hebbian learning and modified Pseudoinverse learning rules. The study explores the tolerance of Hopfield neural networks for reducing the effect of spurious minima in the recalling process by employing the Simulated annealing process. It is observed from the simulations that the capabilities of the Hopfield network can be sufficiently enhanced by making modifications in the feature extraction of the input data. DWT and SOM together can be used to significantly enhance the recall efficiency. The probability of error in recall in the form of spurious minima is minimized by adopting simulated annealing process in the pattern recalling process.     

A MOS circuit for a nonmonotonic neural network with excellentretrieval capabilities

An MOS circuit is proposed for implementing a nonmonotonic transfer characteristic of a neural network. The present research is motivated by the recent results of theoretical studies showing excellent equilibrium properties of networks with the nonmonotonic neural units. These properties include enhancement of storage capacity and complete elimination of noise in associative memory recall. The simple form of the transfer characteristic enables one to implement it with a simple electrical circuit of standard MOS transistors. SPICE simulation results are shown for the behavior of the neural units in associative memory recall.     

Competition and cooperation in neuronal processing

A new type of model neuron is introduced as a building block of an associative memory. The neuron, which has a number of receptor zones, processes both the amplitude and the frequency of input signals, associating a small number of features encoded by those signals. Using this two-parameter input in our model compared to the one-dimensional inputs of conventional model neurons (e.g., the McCulloch Pitts neuron) offers an increased memory capacity. In our model, there is a competition among inputs in each zone with a subsequent cooperation of the winners to specify the output. The associative memory consists of a network of such neurons. A state-space model is used to define the neurodynamics. We explore properties of the neuron and the network and demonstrate its favorable capacity and recall capabilities. Finally, the network is used in an application designed to find trademarks that sound alike.     

Second-order asymmetric BAM design with a maximal basin of attraction

Bidirectional associative memory (BAM) generalizes the associative memory (AM) to be capable of performing two-way recalling of pattern pairs. Asymmetric bidirectional associative memory (ABAM) is a variant of BAM relaxed with connection weight symmetry restriction and enjoys a much better performance than a conventional BAM structure. Higher-order associative memories (HOAMs) are reputed for their higher memory capacity than the first-order counterparts. The paper concerns the design of a second-order asymmetric bidirectional associative memory (SOABAM) with a maximal basin of attraction, whose extension to a HOABAM is possible and straightforward. First, a necessary and sufficient condition is derived for the connection weight matrix of SOABAM that can guarantee the recall of all prototype pattern pairs. A local training rule which is adaptive in the learning step size is formulated. Then derived is a theorem, designing a SOABAM further enlarging the quantities required to meet the complete recall theorem will enhance the capability of evolving a noisy pattern to converge to its association pattern vector without error. Based on this theorem, our algorithm is also modified to ensure each training pattern is stored with a basin of attraction as large as possible.     

对数-指数形态学联想记忆

利用对数和指数算子构建了一种新的形态学联想记忆方法,简称LEMAM.理论分析表明:自联想LEMAM(简称ALEMAM)具有无限存储能力、一步回忆记忆、一定的抵抗腐蚀噪声或膨胀噪声的能力,在输入完全或在一定的噪声范围内,能够保证完全回忆记忆;异联想LEMAM(简称HLEMAM)在输入完全情况下,不能保证完全回忆记忆,但当满足一定条件时,也能够达到完美联想记忆.对比实验结果表明:在一些情况下,LEMAM能够取得较好的联想记忆效果.总体来说,LEMAM丰富了形态学联想记忆的理论和实践,可以作为一种神经计算模型加以研究和利用.     

Autonomous learning with complex dynamics

Traditionally, associative memory models are based on point attractor dynamics, where a memory state corresponds to a stationary point in state space. However, biological neural systems seem to display a rich and complex dynamics whose function is still largely unknown. We use a neural network model of the olfactory cortex to investigate the functional significance of such dynamics, in particular with regard to learning and associative memory. the model uses simple network units, corresponding to populations of neurons connected according to the structure of the olfactory cortex. All essential dynamical properties of this system are reproduced by the model, especially oscillations at two separate frequency bands and aperiodic behavior similar to chaos. By introducing neuromodulatory control of gain and connection weight strengths, the dynamics can change dramatically, in accordance with the effects of acetylcholine, a neuromodulator known to be involved in attention and learning in animals. With computer simulations we show that these effects can be used for improving associative memory performance by reducing recall time and increasing fidelity. the system is able to learn and recall continuously as the input changes, mimicking a real world situation of an artificial or biological system in a changing environment. © 1995 John Wiley & Sons, Inc.     

Two coding strategies for bidirectional associative memory

Enhancements of the encoding strategy of a discrete bidirectional associative memory (BAM) reported by B. Kosko (1987) are presented. There are two major concepts in this work: multiple training, which can be guaranteed to achieve recall of a single trained pair under suitable initial conditions of data, and dummy augmentation, which can be guaranteed to achieve recall of all trained pairs if attaching dummy data to the training pairs is allowable. In representative computer simulations, multiple training has been shown to lead to an improvement over the original Kosko strategy for recall of multiple pairs as well. A sufficient condition for a correlation matrix to make the energies of the training pairs be local minima is discussed. The use of multiple training and dummy augmentation concepts are illustrated, and theorems underlying the results are presented.     

一种基于区间优化的神经网络学习算法

神经网络的学习算法通常是采用梯度下降法，此方法容易陷入局部极小而得到次最优解。另外，对于有些应用来说，用于训练网络的样本的输入／输出数据无法精确给出，而只能以一定的范围的形式给出，这就给传统的神经网络带来了困难。该文提出了一种基于区间优化的神经网络学习算法，可以很好地解决上面所提到的传统神经网络学习算法的缺点。     

A Weakly Connected Memristive Neural Network for Associative Memory

In this paper, we propose a star-like weakly connected memristive neural network which is organized in such a way that each cell only interacts with the central cells. By using the describing function method and Malkin’s theorem the phase deviation of this dynamical network is obtained. And then, under the Hebbian learning rule the phase deviation is designed as a desired model for associative memory. Moreover, we take the store and recall of digital images as an example to demonstrate the performance of associative memory. The main contribution of this paper is supply a useful mechanism which the new potential circuit element memristor can be used to realize the associative.     

混沌神经网络在分离叠加模式和多对多联想记忆中的应用

混沌是不含外加随机因素的完全确定性的系统表现出来的界于规则和随机之间的内秉随机行为。脑神经系统是由神经细胞组成的网络。类似于人脑思维的人工神经网络与冯·诺依曼计算机相比,在信息处理方面有很大的优越性。混沌和神经网络相互融合的研究是从90年代开始的,其主要的目标是通过分析大脑的混沌现象,建立含有混沌动力学的神经网络模型(即混沌神经网络模型),将混沌的遍历性、对初始值敏感等特点与神经网络的非线性、自适应、并行处理优势相结合,     

基于结构学习和迭代自映射的自联想记忆模型

传统的人工神经元网络连接结构是固定的,是对权值的学习.提出一种基于生理神经元特征的人工神经元模型,并在以此为单元构成的用于实现自联想记忆的神经网络上进行对结构的学习.学习算法以设定神经元的输入/输出感受野、调整突触和轴突末梢的连接、并行的自投影迭代为特征.给出了此网络模型的矩阵描述和实验结果.     

基于爱因斯坦t-模的模糊联想记忆的学习算法

为神经网络提供有效学习算法是神经网络研究的关键问题。文章利用t-模的伴随蕴涵算子,为基于Max和Tes合成的模糊联想记忆网络Max-TesFAM提供了一种新的学习算法,此处Tes是由爱因斯坦提出的一种t-模算子。从理论上严格证明了,只要Max-TesFAM能完整可靠地存储所给的多个模式对,则该新的学习算法一定能找到使得网络能完整可靠存储这些模式对的所有连接权矩阵的最大者。最后,用实验说明了所提出的学习算法的有效性。     

Robust image association by recurrent neural subnetworks

A neural network consisting of a gallery of independent subnetworks is developed for associative memory which stores and recalls gray scale images. Each original image is encoded by a unique stable state of one of neural recurrent subnetworks. Comparing to Amari-Hopfield associative memory, our solution has no spurious states, is less sensitive to noise, and its network complexity is significantly lower. Computer simulations confirm that associative recall in this system for images of natural scenes is very robust. Colored additive and multiplicative noise with standard deviation up to =2 can be removed perfectly from normalized image. The same observations are valid for spiky noise distributed on up to 70% of image area. Even if we remove up to 95% pixels from the original image in deterministic or random way, still the network performs the correct association.     

An RCE-based Associative Memory with Application to Human Face Recognition

Many models of neural network-based associative memory have been proposed and studied. However, most of these models do not have a rejection mechanism and hence are not practical for many real-world associative memory problems. For example, in human face recognition, we are given a database of face images and the identity of each image. Given an input image, the task is to associate when appropriate the image with the corresponding name of the person in the database. However, the input image may be that of a stranger. In this case, the system should reject the input. In this paper, we propose a practical associative memory model that has a rejection mechanism. The structure of the model is based on the restricted Coulomb energy (RCE) network. The capacity of the proposed memory is desibed by two measures: the ability of the system to correctly identify known individuals, and the ability of the system to reject individuals who are not in the database. Experimental results are given which show how the performance of the system varies as the size of the database increases up to 1000 individuals.     

基于区间优化的神经网络全局优化方法

Hopfield神经网络被广泛应用于优化问题的求解中,而传统的Hopfield网络通常基于梯度下降法,此方法容易陷入局部极小而得到次最优解或收敛到问题的不可行解。另外,当用于训练网络样本的输入/输出数据无法精确给出,而只能以一定的范围的形式给出时,传统的神经网络学习方法就无能为力了。论文提出了一种基于区间优化的神经网络学习算法,可以很好地解决上面所提到的传统神经网络学习算法的缺点。     

BAM Learning of Nonlinearly Separable Tasks by Using an Asymmetrical Output Function and Reinforcement Learning

Most bidirectional associative memory (BAM) networks use a symmetrical output function for dual fixed-point behavior. In this paper, we show that by introducing an asymmetry parameter into a recently introduced chaotic BAM output function, prior knowledge can be used to momentarily disable desired attractors from memory, hence biasing the search space to improve recall performance. This property allows control of chaotic wandering, favoring given subspaces over others. In addition, reinforcement learning can then enable a dual BAM architecture to store and recall nonlinearly separable patterns. Our results allow the same BAM framework to model three different types of learning: supervised, reinforcement, and unsupervised. This ability is very promising from the cognitive modeling viewpoint. The new BAM model is also useful from an engineering perspective; our simulations results reveal a notable overall increase in BAM learning and recall performances when using a hybrid model with the general regression neural network (GRNN).      

一种基于MRⅡ算法的三层二值双向联想记忆网络

传统的两层二值双向联想记忆(BAM)网络因其结构的限制存在着存储容量有限、区分小差别模式和存储非正交模式能力不足的缺陷,结构上将其扩展至三层网络是一个有效的解决思路,但是三层二值BAM网络的学习是一个难题,而三层连续型BAM网络又存在处理二值问题不方便的问题。为了解决这些问题,提出一种三层结构的二值双向联想记忆网络,创新之处是采用了二值多层前向网络的MRⅡ算法实现了三层二值BAM网络的学习。实验结果表明,基于MRⅡ算法的三层二值BAM网络极大地提高了网络的存储容量和模式区分能力,同时保留了二值网络特定的优势,具有较高的理论与实用价值。     

Scene analysis by integrating primitive segmentation andassociative memory

Scene analysis is a major aspect of perception and continues to challenge machine perception. This paper addresses the scene-analysis problem by integrating a primitive segmentation stage with a model of associative memory. The model is a multistage system that consists of an initial primitive segmentation stage, a multimodule associative memory, and a short-term memory (STM) layer. Primitive segmentation is performed by a locally excitatory globally inhibitory oscillator network (LEGION), which segments the input scene into multiple parts that correspond to groups of synchronous oscillations. Each segment triggers memory recall and multiple recalled patterns then interact with one another in the STM layer. The STM layer projects to the LEGION network, giving rise to memory-based grouping and segmentation. The system achieves scene analysis entirely in phase space, which provides a unifying mechanism for both bottom-up analysis and top-down analysis. The model is evaluated with a systematic set of three-dimensional (3-D) line drawing objects, which are arranged in an arbitrary fashion to compose input scenes that allow object occlusion. Memory-based organization is responsible for a significant improvement in performance. A number of issues are discussed, including input-anchored alignment, top-down organization, and the role of STM in producing context sensitivity of memory recall.