双向联想记忆神经网络抗噪声的能力

This paper analyzes noise sensitivity of bidirectional association memory (BAM) and shows that the anti-noise capability of BAM relates not only to the minimum absolute value of net inputs(MAV), as some researchers found, but also to the variance of weights associated with synapse connections. In fact, it is determined by the quotient of these two factors. On this base, a novel learning algorithm—small variance leaning for BAM(SVBAM) is proposed, which is to decrease the variance of the weights of synapse matrix. Simulation experiments show that the algorithm can decrease the variance of weights efficiently, therefore, noise immunity of BAM is improved. At the same time, perfect recall of all training pattern pairs still can be guaranteed by the algorithm.     

Evolutionary Pseudo-Relaxation Learning Algorithm for Bidirectional Associative Memory

This paper analyzes the sensitivity to noise in BAM (Bidirectional Associative Memory), and then proves the noise immunity of BAM relates not only to the minimum absolute value of net inputs (MAV) but also to the variance of weights associated with synapse connections. In fact, it is a positive monotonically increasing function of the quotient of MAV divided by the variance of weights. Besides, the performance of pseudo-relaxation method depends on learning parameters(λ and ξ), but the relation of them is not linear. So it is hard to find a best combination of λ and ξ which leads to the best BAM performance. And it is obvious that pseudo-relaxation is a kind of local optimization method, so it cannot guarantee to get the global optimal solution. In this paper, a novel learning algorithm EPRBAM (evolutionary psendo-relaxation learning algorithm for bidirectional association memory) employing genetic algorithm and pseudo-relaxation method is proposed to get feasible solution of BAM weight matrix. This algorithm uses the quotient as the fitness of each individual and employs pseudo-relaxation method to adjust individual solution when it does not satisfy constraining condition any more after genetic operation. Experimental results show this algorithm improves noise immunity of BAM greatly. At the same time, EPRBAM does not depend on learning parameters and can get global optimal solution.     

A bidirectional heteroassociative memory for binary and grey-level patterns

Typical bidirectional associative memories (BAM) use an offline, one-shot learning rule, have poor memory storage capacity, are sensitive to noise, and are subject to spurious steady states during recall. Recent work on BAM has improved network performance in relation to noisy recall and the number of spurious attractors, but at the cost of an increase in BAM complexity. In all cases, the networks can only recall bipolar stimuli and, thus, are of limited use for grey-level pattern recall. In this paper, we introduce a new bidirectional heteroassociative memory model that uses a simple self-convergent iterative learning rule and a new nonlinear output function. As a result, the model can learn online without being subject to overlearning. Our simulation results show that this new model causes fewer spurious attractors when compared to others popular BAM networks, for a comparable performance in terms of tolerance to noise and storage capacity. In addition, the novel output function enables it to learn and recall grey-level patterns in a bidirectional way.     

Encoding strategy for maximum noise tolerance bidirectional associative memory

In this paper, the basic bidirectional associative memory (BAM) is extended by choosing weights in the correlation matrix, for a given set of training pairs, which result in a maximum noise tolerance set for BAM. We prove that for a given set of training pairs, the maximum noise tolerance set is the largest, in the sense that this optimized BAM will recall the correct training pair if any input pattern is within the maximum noise tolerance set and at least one pattern outside the maximum noise tolerance set by one Hamming distance will not converge to the correct training pair. This maximum tolerance set is the union of the maximum basins of attraction. A standard genetic algorithm (GA) is used to calculate the weights to maximize the objective function which generates a maximum tolerance set for BAM. Computer simulations are presented to illustrate the error correction and fault tolerance properties of the optimized BAM.     

Second-order asymmetric BAM design with a maximal basin of attraction

Bidirectional associative memory (BAM) generalizes the associative memory (AM) to be capable of performing two-way recalling of pattern pairs. Asymmetric bidirectional associative memory (ABAM) is a variant of BAM relaxed with connection weight symmetry restriction and enjoys a much better performance than a conventional BAM structure. Higher-order associative memories (HOAMs) are reputed for their higher memory capacity than the first-order counterparts. The paper concerns the design of a second-order asymmetric bidirectional associative memory (SOABAM) with a maximal basin of attraction, whose extension to a HOABAM is possible and straightforward. First, a necessary and sufficient condition is derived for the connection weight matrix of SOABAM that can guarantee the recall of all prototype pattern pairs. A local training rule which is adaptive in the learning step size is formulated. Then derived is a theorem, designing a SOABAM further enlarging the quantities required to meet the complete recall theorem will enhance the capability of evolving a noisy pattern to converge to its association pattern vector without error. Based on this theorem, our algorithm is also modified to ensure each training pattern is stored with a basin of attraction as large as possible.     

An analysis of high-capacity discrete exponential BAM

An exponential bidirectional associative memory (eBAM) using an exponential encoding scheme is discussed. It has a higher capacity for pattern pair storage than conventional BAMs. A new energy function is defined. The associative memory takes advantage of the exponential nonlinearity in the evolution equations such that the signal-to-noise ratio (SNR) is significantly increased. The energy of the eBAM decreases as the recall process proceeds, ensuring the stability of the system. The increase of SNR consequently enhances the capacity of the BAM. The capacity of the exponential BAM is estimated.     

Guaranteed recall of all training pairs for bidirectionalassociative memory

Necessary and sufficient conditions are derived for the weights of a generalized correlation matrix of a bidirectional associative memory (BAM) which guarantee the recall of all training pairs. A linear programming/multiple training (LP/MT) method that determines weights which satisfy the conditions when a solution is feasible is presented. The sequential multiple training (SMT) method is shown to yield integers for the weights, which are multiplicities of the training pairs. Computer simulation results, including capacity comparisons of BAM, LP/MT BAM, and SMT BAM, are presented.     

Weighted learning of bidirectional associative memories by globalminimization

A weighted learning algorithm for bidirectional associative memories (BAMs) by means of global minimization, where each desired pattern is weighted, is described. According to the cost function that measures the goodness of the BAM, the learning algorithm is formulated as a global minimization problem and solved by a gradient descent rule. The learning approach guarantees not only that each desired pattern is stored as a stable state, but also that the basin of attraction is constructed as large as possible around each desired pattern. The existence of the weights, the asymptotic stability of each desired pattern and its basin of attraction, and the convergence of the proposed learning algorithm are investigated in an analytic way. A large number of computer experiments are reported to demonstrate the efficiency of the learning rule.     

The effect of weight fault on associative networks

In the past three decades, the properties of associative networks has been extensively investigated. However, most existing results focus on the fault-free networks only. In implementation, network faults can be exhibited in different forms, such as open weight fault and multiplicative weight noise. This paper studies the effect of weight fault on the performance of the bidirectional associative memory (BAM) model when multiplicative weight noise and open weight fault present. Assuming that connection weights are corrupted by these two common fault models, we study how many number of pattern pairs can be stored in a faulty BAM. Since one of important feature of associative network is error correction, we also study the number of pattern pairs can be stored in a faulty BAM when there are some errors in the initial stimulus pattern.     

Encoding method for bidirectional associative memory usingprojection on convex sets

The traditional encoding method of bidirectional associative memory (BAM) suggested by Kosko (1988) is based on the correlation method with which the capacity is very small. The enhanced Householder encoding algorithm (EHCA) presented here is developed on the basis of the Householder encoding algorithm (HCA) and projection on convex sets (POCS). The capacity of BAM with HCA tends to the dimension of the pattern pairs. Unfortunately, in BAM with HCA there are two different interconnection matrices and hence BAM with HCA may not converge when the initial stimulus is not one of the library patterns. In EHCA the two matrices found by HCA are reduced into one matrix by POCS. Hence, the convergent property of BAM can be maintained. Simulation results show that the capacity of BAM with EHCA is greatly improved.     

一种基于MRⅡ算法的三层二值双向联想记忆网络

传统的两层二值双向联想记忆(BAM)网络因其结构的限制存在着存储容量有限、区分小差别模式和存储非正交模式能力不足的缺陷,结构上将其扩展至三层网络是一个有效的解决思路,但是三层二值BAM网络的学习是一个难题,而三层连续型BAM网络又存在处理二值问题不方便的问题。为了解决这些问题,提出一种三层结构的二值双向联想记忆网络,创新之处是采用了二值多层前向网络的MRⅡ算法实现了三层二值BAM网络的学习。实验结果表明,基于MRⅡ算法的三层二值BAM网络极大地提高了网络的存储容量和模式区分能力,同时保留了二值网络特定的优势,具有较高的理论与实用价值。     

双向联想记忆学习与回忆的研究

本文对双向联想记忆(BAM)的学习与回忆过程进行了详细的分析。在学习过程中,先是运用自适应非对称BAM算法进行学习,进而采用设置印象门限的反复记忆算法进行学习,本文从理论上证明了印象门限与样本吸引域之间的关系,指出反复记忆方法的理论依据。回忆过程中,采用非零阈值函数的运行方程,提出了阈值学习方法,并且从理论上证明了非零阈值函数的运行方程的采用,可进一步扩大吸引域。为了进一步扩大网络的信息存储量,本文引入了并联的BAM结构。本文方法的采纳,使得BAM网络的信息存储量、误差校正能力等得到很大程度的提高。     

BAM Learning of Nonlinearly Separable Tasks by Using an Asymmetrical Output Function and Reinforcement Learning

Most bidirectional associative memory (BAM) networks use a symmetrical output function for dual fixed-point behavior. In this paper, we show that by introducing an asymmetry parameter into a recently introduced chaotic BAM output function, prior knowledge can be used to momentarily disable desired attractors from memory, hence biasing the search space to improve recall performance. This property allows control of chaotic wandering, favoring given subspaces over others. In addition, reinforcement learning can then enable a dual BAM architecture to store and recall nonlinearly separable patterns. Our results allow the same BAM framework to model three different types of learning: supervised, reinforcement, and unsupervised. This ability is very promising from the cognitive modeling viewpoint. The new BAM model is also useful from an engineering perspective; our simulations results reveal a notable overall increase in BAM learning and recall performances when using a hybrid model with the general regression neural network (GRNN).      

A Bidirectional Hetero-Associative Memory for True-Color Patterns

Classical bidirectional associative memories (BAM) have poor memory storage capacity, are sensitive to noise, are subject to spurious steady states during recall, and can only recall bipolar patterns. In this paper, we introduce a new bidirectional hetero-associative memory model for true-color patterns that uses the associative model with dynamical synapses recently introduced in Vazquez and Sossa (Neural Process Lett, Submitted, 2008). Synapses of the associative memory could be adjusted even after the training phase as a response to an input stimulus. Propositions that guarantee perfect and robust recall of the fundamental set of associations are provided. In addition, we describe the behavior of the proposed associative model under noisy versions of the patterns. At last, we present some experiments aimed to show the accuracy of the proposed model with a benchmark of true-color patterns.     

A feedforward bidirectional associative memory

In contrast to conventional feedback bidirectional associative memory (BAM) network models, a feedforward BAM network is developed based on a one-shot design algorithm of O(p(2)(n+m)) computational complexity, where p is the number of prototype pairs and n, m are the dimensions of the input/output bipolar vectors. The feedforward BAM is an n-p-m three-layer network of McCulloch-Pitts neurons with storage capacity 2(min{m,n}) and guaranteed perfect bidirectional recall. The overall network design procedure is fully scalable in the sense that any number p=/<2(min{m,n}) of bidirectional associations can be implemented. The prototype patterns may be arbitrarily correlated. With respect to inference performance, it is shown that the Hamming attractive radius of each prototype reaches the maximum possible value. Simulation studies and comparisons illustrate and support these theoretical developments.     

Orthogonal schemes for bidirectional associative memories

Two issues are addressed in this paper. Firstly, it investigates some important properties of bidirectional associative memories (BAM) and proposes an improved capacity estimate. Those properties are the encoding form of the input pattern pairs as sell as their decoding, the orthogonality of the pattern pairs, the similarity of associated patterns, and the density of the pattern pairs. Secondly, it proposes an implementation approach to improve the storage capacity. The approach embraces three proposed methods, i.e., the bipolar-orthogonal augmentation, the set partition, and the combined method. Along with those methods is the construction of the set of bipolar orthogonal patterns.     

改进的指数双向联想记忆模型及性能估计

提出了一个新的改进型指数双向联想记忆模型(improved eBAM,简称IeBAM).通过定义有界且随状态改变而下降的能量函数,证明了IeBAM在状态的同、异步更新方式下的稳定性,一方面排除了Wang的修正指数BAM(modified eBAM,简称MeBAM)和Jeng的eBAM(exponential BAM)的稳定性证明中所作的不合理假设;另一方面,放宽了对BAM(bidirectional associative memory)的连续性假设的要求,并避免了补码问题.理论分析和计算机模拟结果表明,     

模糊联想记忆网络的增强学习算法

针对 Kosko提出的最大最小模糊联想记忆网络存在的问题 ,通过对这种网络连接权学习规则的改进 ,给出了另一种权重学习规则 ,即把 Kosko的前馈模糊联想记忆模型发展成为模糊双向联想记忆模型 ,并由此给出了模糊快速增强学习算法 ,该算法能存储任意给定的多值训练模式对集 .其中对于存储二值模式对集 ,由于其连接权值取值 0或 1,因而该算法易于硬件电路和光学实现 .实验结果表明 ,模糊快速增强学习算法是行之有效的 .     

Performance analysis of the bidirectional associative memory and animproved model from the matched-filtering viewpoint

This paper discusses the bidirectional associative memory (BAM) model from the matched-filtering viewpoint and offers it a new interpretation. Our attention is focused on the problem of stability and attractivity of equilibrium states. Several sufficient and/or necessary conditions are presented. To improve the BAM performance, an exponential function is used to enhance the correlations between the binary vectors of the retrieval key and that of the stored pattern similar to the key. The modified model is shown to be asymptotically stable. Theoretical analysis and simulation results demonstrate that the modified model performs much better than the original BAM in terms of memory capacity and error correction capability.     

一个新的高阶双向联想记忆模型及其性能估计

提出了一个新的高阶双向联想记忆模型.它推广了由Tai及Jeng所提出的高阶双向联想记忆模型HOBAM(higher-order bidirectional associative memory)及修正的具有内连接的双向联想记忆模型MIBAM(modified intraconnected bidirectional associative memory),通过定义能量函数,证明了新模型在同步与异步更新方式下的稳定性,从而能够保证所有被训练模式对成为该模型的渐近稳定点.借助统计分析原理,估计了所提模型的存储容量.计算机模拟证实此模型不仅具有较高的存储容量,而且还具有较好的纠错能力.