A synthesis procedure for brain-state-in-a-box neural networks

In this paper, some new qualitative properties of discrete-time neural networks based on the "brain-state-in-a-box" model are presented. These properties concern both the characterization of equilibrium points and the global dynamical behavior. Next, the analysis results are used as guidelines in developing an efficient synthesis procedure for networks that function as associative memories. A constrained design algorithm is presented that gives completely stable dynamical neural networks sharing some interesting features. It is guaranteed the absence of nonbinary stable equilibria, that is stable states with nonsaturated components. It is guaranteed that in close proximity (Hamming distance one) of the stored patterns there is no other binary equilibrium point. Moreover, the presented method allows one to optimize a design parameter that controls the size of the attraction basins of the stored vectors and the accuracy needed in a digital realization of the network.     

Analysis and synthesis of a class of discrete-time neural networksdescribed on hypercubes

A qualitative analysis is presented for a class of synchronous discrete-time neural networks defined on hypercubes in the state space. Analysis results are utilized to establish a design procedure for associative memories to be implemented on the present class of neural networks. To demonstrate the storage ability and flexibility of the synthesis procedure, several specific examples are considered. The design procedure has essentially the same desirable features as the results of J. Li et al. (1988, 1989) for continuous-time neural networks. For a given system dimension, networks designed by the present method may have the ability to store more patterns (as asymptotically stable equilibria) than corresponding discrete-time networks designed by other techniques. The design method guarantees the storage of all the desired patterns as asymptotically stable equilibrium points. The present method provides guidelines for reducing the number of spurious states and for estimating the extent of the patterns' domains of attraction. The present results provide a means of implementing neural networks by serial processors and special digital hardware.     

基于约束区域的连续时间联想记忆神经网络

传统的联想记忆神经网络模型是根据联想记忆点设计权值。文中提出一种根据联想记忆点设计基于约束区域的神经网络模型,它保证了渐近稳定的平衡点集与样要点集相同,不渐近稳定的平衡点恰为实际的拒识状态,并且吸引域分布合理。它具有学习和遗忘能力,还具有记忆容量大和电路可实现优点,是理想的联想记忆器。     

Recurrent neural nets as dynamical Boolean systems with applicationto associative memory

Discrete-time/discrete-state recurrent neural networks are analyzed from a dynamical Boolean systems point of view in order to devise new analytic and design methods for the class of both single and multilayer recurrent artificial neural networks. With the proposed dynamical Boolean systems analysis, we are able to formulate necessary and sufficient conditions for network stability which are more general than the well-known but restrictive conditions for the class of single layer networks: (1) symmetric weight matrix with (2) positive diagonal and (3) asynchronous update. In terms of design, we use a dynamical Boolean systems analysis to construct a high performance associative memory. With this Boolean memory, we can guarantee that all fundamental memories are stored, and also guarantee the size of the basin of attraction for each fundamental memory.     

Design of GBSB neural associative memories using semidefiniteprogramming

This paper concerns reliable search for the optimally performing GBSB (generalized brain-state-in-a-box) neural associative memory given a set of prototype patterns to be stored as stable equilibrium points. First, we observe some new qualitative properties of the GBSB model. Next, we formulate the synthesis of GBSB neural associative memories as a constrained optimization problem. Finally, we convert the optimization problem into a semidefinite program (SDP), which can be solved efficiently by recently developed interior point methods. The validity of this approach is illustrated by a design example.     

基于遗传算法的Hopfield联想记忆吸引域的优化

由于一般离散Hopfield神经网络存在很多伪稳定点.使稳定点的吸引域变小.网络很难获得真正的最优解.因此,提出将遗传算法应用到Hopfield联想记忆神经网络中.利用遗传算法对复杂、多峰、非线性极不可微函数实现全局搜索性质.对Hopfield联想记忆吸引域进行优化,使待联想模式跳出伪模式的吸引域.使Hopfield网络在较高噪信比的情况下保持较高的联想成功率.仿真结果证明了该方法的有效性.     

基于约束区域的BSB联想记忆模型

提出一种根据联想记忆点设计基于约束区域的ＢＳＢ（Ｂｒａｉｎ－Ｓｔａｔｅ－ｉｎｍ－ａ－Ｂｏｘ）神经网络模型,它保证了渐近稳定的平衡点集与样本点集相同,不渐近稳定的平衡点恰为实际的拒识状态,并且吸引域分布合理,从而将ＥＳＢ完善为理想的联想记忆器。     

Dynamical analysis of the brain-state-in-a-box (BSB) neural models

A stability analysis is performed for the brain-state-in-a-box (BSB) neural models with weight matrices that need not be symmetric. The implementation of associative memories using the analyzed class of neural models is also addressed. In particular, the authors modify the BSB model so that they can better control the extent of the domains of attraction of stored patterns. Generalizations of the results obtained for the BSB models to a class of cellular neural networks are also discussed.     

A novel continuous-time neural network for realizing associativememory

A novel neural network is proposed in this paper for realizing associative memory. The main advantage of the neural network is that each prototype pattern is stored if and only if as an asymptotically stable equilibrium point. Furthermore, the basin of attraction of each desired memory pattern is distributed reasonably (in the Hamming distance sense), and an equilibrium point that is not asymptotically stable is really the state that cannot be recognized. The proposed network also has a high storage as well as the capability of learning and forgetting, and all its components can be implemented. The network considered is a very simple linear system with a projection on a closed convex set spanned by the prototype patterns. The advanced performance of the proposed network is demonstrated by means of simulation of a numerical example.     

离散Hopfield双向联想记忆神经网络的稳定性分析

首先将离散Hopfield双向联想记忆神经网络转化成一个特殊的离散Hopfield网络 模型.在此基础上,对离散Hopfield双向联想记忆神经网络的全局渐近稳定性和全局指数稳 定性进行了新的分析.证明了神经网络连接权矩阵在给定的约束条件下有唯一的而且是渐近 稳定的平衡点.利用Lyapunov方程正对角解的存在性得到了几个判定平衡点为全局渐近稳 定和全局指数稳定的充分条件.这些条件可以用于设计全局渐近稳定和全局指数稳定的神经 网络.所做的分析扩展了以前的稳定性结果.     

含时延的联想记忆神经网络的指数吸引域和指数收敛速度

采用不等式技巧和非负矩阵性质, 给出了含时延的联想记忆神经网络平衡点的指数吸引域和指数收敛速度估计以及指数稳定的一些判断条件.     

Equilibrium characterization of dynamical neural networks and asystematic synthesis procedure for associative memories

Several novel results concerning the characterization of the equilibrium conditions of a continuous-time dynamical neural network model and a systematic procedure for synthesizing associative memory networks with nonsymmetrical interconnection matrices are presented. The equilibrium characterization focuses on the exponential stability and instability properties of the network equilibria and on equilibrium confinement, viz., ensuring the uniqueness of an equilibrium in a specific region of the state space. While the equilibrium confinement result involves a simple test, the stability results given obtain explicit estimates of the degree of exponential stability and the regions of attraction of the stable equilibrium points. Using these results as valuable guidelines, a systematic synthesis procedure for constructing a dynamical neural network that stores a given set of vectors as the stable equilibrium points is developed.     

On the design of BSB neural associative memories using semidefinite programming

This article is concerned with the reliable search for optimally performing BSB (brain state in a box) neural associative memories given a set of prototype patterns to be stored as stable equilibrium points. By converting and/or modifying the nonlinear constraints of a known formulation for the synthesis of BSB-based associative memories into linear matrix inequalities, we recast the synthesis into semidefinite programming problems and solve them by recently developed interior point methods. The validity of this approach is illustrated by a design example.     

基于线性规划的联想记忆神经网络模型

提出一种基于优化线性函数的神经网络联想记忆器,打破了将待识别模式作为网络起始点的常规,它能保证渐近稳定的平衡点集与样本点集相同,吸引域分布合理,不渐近稳定的平衡点恰为实际的拒识模式,并且电路实现容易,对拒识模式有清楚的解释。理论分析和计算机模拟都表明本文的模型是理想的联想记忆器,还可降低对硬件的精度要求。     

Analysis and design of associative memories based on recurrent neural network with discontinuous activation functions

This paper considers a recurrent neural network (RNN) with a special class of discontinuous activation function which is piecewise constants in the state space. One sufficient condition is established to ensure that the novel recurrent neural networks can have (4k−1)ⁿ locally exponential stable equilibrium points. Such RNN is suitable for synthesizing high-capacity associative memories. The design procedure is presented with the method of singular value decomposition. Finally, the validity and performance of the results are illustrated by use of two numerical examples.     

一类时延反馈神经网络的稳定性及吸引域的估计

反馈型神经网络由于具有极为丰富的动力学行为和整体计算能力（如优化、联想、振荡和混饨）而倍受关注，近几年的研究表明，当网络的时延足够小时，具有延时的对称Hopfield型神经网络和无时延情况一样也是全局稳定的．本文通过构造适当Lyapunov泛函的方法，对一类具有时延的反馈型神经网络平衡点的渐近稳定性进行了分析，得到了平衡点渐近稳定的充分条件：要检验一个有时间延迟的反馈型神经网络的稳定性，只要测试一个特定矩阵的定性性质或一个特定不等式即可．最后我们也提供了一种估计网络渐近稳定平衡点吸引域的方法．     

Associative memory design for 256 gray-level images using a multilayer neural network

A design procedure is presented for neural associative memories storing gray-scale images. It is an evolution of a previous work based on the decomposition of the image with 2/sup L/ gray levels into L binary patterns, stored in L uncoupled neural networks. In this letter, an L-layer neural network is proposed with both intralayer and interlayer connections. The connections between different layers introduce interactions among all the neurons, increasing the recall performance with respect to the uncoupled case. In particular, the proposed network can store images with the commonly used number of 256 gray levels instead of 16, as in the previous approach.     

Hamiltonian perspective on compartmental reaction–diffusion networks

Inspired by the recent developments in modeling and analysis of reaction networks, we provide a geometric formulation of the reversible reaction networks under the influence of diffusion. Using the graph knowledge of the underlying reaction network, the obtained reaction–diffusion system is a distributed-parameter port-Hamiltonian system on a compact spatial domain. Motivated by the need for computer-based design, we offer a spatially consistent discretization of the PDE system and, in a systematic manner, recover a compartmental ODE model on a simplicial triangulation of the spatial domain. Exploring the properties of a balanced weighted Laplacian matrix of the reaction network and the Laplacian of the simplicial complex, we characterize the space of equilibrium points and provide a simple stability analysis on the state space modulo the space of equilibrium points. The paper rules out the possibility of the persistence of spatial patterns for the compartmental balanced reaction–diffusion networks.     

Design and Analysis of High-Capacity Associative Memories Based on a Class of Discrete-Time Recurrent Neural Networks

This paper presents a design method for synthesizing associative memories based on discrete-time recurrent neural networks. The proposed procedure enables both hetero- and autoassociative memories to be synthesized with high storage capacity and assured global asymptotic stability. The stored patterns are retrieved by feeding probes via external inputs rather than initial conditions. As typical representatives, discrete-time cellular neural networks (CNNs) designed with space-invariant cloning templates are examined in detail. In particular, it is shown that procedure herein can determine the input matrix of any CNN based on a space-invariant cloning template which involves only a few design parameters. Two specific examples and many experimental results are included to demonstrate the characteristics and performance of the designed associative memories.      

Network capacity analysis for latent attractor computation

Attractor networks have been one of the most successful paradigms in neural computation, and have been used as models of computation in the nervous system. Recently, we proposed a paradigm called 'latent attractors' where attractors embedded in a recurrent network via Hebbian learning are used to channel network response to external input rather than becoming manifest themselves. This allows the network to generate context-sensitive internal codes in complex situations. Latent attractors are particularly helpful in explaining computations within the hippocampus--a brain region of fundamental significance for memory and spatial learning. Latent attractor networks are a special case of associative memory networks. The model studied here consists of a two-layer recurrent network with attractors stored in the recurrent connections using a clipped Hebbian learning rule. The firing in both layers is competitive--K winners take all firing. The number of neurons allowed to fire, K, is smaller than the size of the active set of the stored attractors. The performance of latent attractor networks depends on the number of such attractors that a network can sustain. In this paper, we use signal-to-noise methods developed for standard associative memory networks to do a theoretical and computational analysis of the capacity and dynamics of latent attractor networks. This is an important first step in making latent attractors a viable tool in the repertoire of neural computation. The method developed here leads to numerical estimates of capacity limits and dynamics of latent attractor networks. The technique represents a general approach to analyse standard associative memory networks with competitive firing. The theoretical analysis is based on estimates of the dendritic sum distributions using Gaussian approximation. Because of the competitive firing property, the capacity results are estimated only numerically by iteratively computing the probability of erroneous firings. The analysis contains two cases: the simple case analysis which accounts for the correlations between weights due to shared patterns and the detailed case analysis which includes also the temporal correlations between the network's present and previous state. The latter case predicts better the dynamics of the network state for non-zero initial spurious firing. The theoretical analysis also shows the influence of the main parameters of the model on the storage capacity.