Complex-valued multistate neural associative memory

A model of a multivalued associative memory is presented. This memory has the form of a fully connected attractor neural network composed of multistate complex-valued neurons. Such a network is able to perform the task of storing and recalling gray-scale images. It is also shown that the complex-valued fully connected neural network may be considered as a generalization of a Hopfield network containing real-valued neurons. A computational energy function is introduced and evaluated in order to prove network stability for asynchronous dynamics. Storage capacity as related to the number of accessible neuron states is also estimated.     

Characteristics of the Hopfield associative memory utilizingisomorphism relations

Isomorphism relations are utilized to analyze the Hopfield associative memory. When the number of fundamental memories m=/<3, it is proved that two Hopfield associative memories are isomorphic if they have the same mutual distances between the fundamental memories. The number of stable states and the synchronous convergence time of a Hopfield associative memory are shown to be less than or equal to 2 to the power 2(m-1) and 4 to the power 2(m-1), respectively, where m>/=1.     

A Boolean Hebb rule for binary associative memory design

A binary associative memory design procedure that gives a Hopfield network with a symmetric binary weight matrix is introduced in this paper. The proposed method is based on introducing the memory vectors as maximal independent sets to an undirected graph, which is constructed by Boolean operations analogous to the conventional Hebb rule. The parameters of the resulting network is then determined via the adjacency matrix of this graph in order to rind a maximal independent set whose characteristic vector is close to the given distorted vector. We show that the method provides attractiveness for each memory vector and avoids spurious memories whenever the set of given memory vectors satisfy certain compatibility conditions, which implicitly imply sparsity. The applicability of the design method is finally investigated by a quantitative analysis of the compatibility conditions.     

A new neural associative memory model

We present a new associative memory model based on the Hamming memory, but where the winner-take-all network part is replaced by a layer of nodes with somewhat complex node functions. This new memory can produce output vectors with individual “don't know” bits. the simulations demonstrate that this memory model works appropriately. © 1992 John Wiley & Sons, Inc.     

An energy function-based design method for discrete hopfield associative memory with attractive fixed points

An energy function-based autoassociative memory design method to store a given set of unipolar binary memory vectors as attractive fixed points of an asynchronous discrete Hopfield network (DHN) is presented. The discrete quadratic energy function whose local minima correspond to the attractive fixed points of the network is constructed via solving a system of linear inequalities derived from the strict local minimality conditions. The weights and the thresholds are then calculated using this energy function. If the inequality system is infeasible, we conclude that no such asynchronous DHN exists, and extend the method to design a discrete piecewise quadratic energy function, which can be minimized by a generalized version of the conventional DHN, also proposed herein. In spite of its computational complexity, computer simulations indicate that the original method performs better than the conventional design methods in the sense that the memory can store, and provide the attractiveness for almost all memory sets whose cardinality is less than or equal to the dimension of its elements. The overall method, together with its extension, guarantees the storage of an arbitrary collection of memory vectors, which are mutually at least two Hamming distances away from each other, in the resulting network.     

CMOS current-mode neural associative memory design with on-chiplearning

Based on the Grossberg mathematical model called the outstar, a modular neural net with on-chip learning and memory is designed and analyzed. The outstar is the minimal anatomy that can interpret the classical conditioning or associative memory. It can also be served as a general-purpose pattern learning device. To realize the outstar, CMOS (complimentary metal-oxide semiconductor) current-mode analog dividers are developed to implement the special memory called the ratio-type memory. Furthermore, a CMOS current-mode analog multiplier is used to implement the correlation. The implemented CMOS outstar can on-chip store the relative ratio values of the trained weights for a long time. It can also be modularized to construct general neural nets. HSPICE (a circuit simulator of Meta Software, Inc.) simulation results of the CMOS outstar circuits as associative memory and pattern learner have successfully verified their functions. The measured results of the fabricated CMOS outstar circuits have also successfully confirmed the ratio memory and on-chip learning capability of the circuits. Furthermore, it has been shown that the storage time of the ratio memory can be as long as five minutes without refreshment. Also the outstar can enhance the contrast of the stored pattern within a long period. This makes the outstar circuits quite feasible in many applications.     

A feedforward bidirectional associative memory

In contrast to conventional feedback bidirectional associative memory (BAM) network models, a feedforward BAM network is developed based on a one-shot design algorithm of O(p(2)(n+m)) computational complexity, where p is the number of prototype pairs and n, m are the dimensions of the input/output bipolar vectors. The feedforward BAM is an n-p-m three-layer network of McCulloch-Pitts neurons with storage capacity 2(min{m,n}) and guaranteed perfect bidirectional recall. The overall network design procedure is fully scalable in the sense that any number p=/<2(min{m,n}) of bidirectional associations can be implemented. The prototype patterns may be arbitrarily correlated. With respect to inference performance, it is shown that the Hamming attractive radius of each prototype reaches the maximum possible value. Simulation studies and comparisons illustrate and support these theoretical developments.     

Encoding method for bidirectional associative memory usingprojection on convex sets

The traditional encoding method of bidirectional associative memory (BAM) suggested by Kosko (1988) is based on the correlation method with which the capacity is very small. The enhanced Householder encoding algorithm (EHCA) presented here is developed on the basis of the Householder encoding algorithm (HCA) and projection on convex sets (POCS). The capacity of BAM with HCA tends to the dimension of the pattern pairs. Unfortunately, in BAM with HCA there are two different interconnection matrices and hence BAM with HCA may not converge when the initial stimulus is not one of the library patterns. In EHCA the two matrices found by HCA are reduced into one matrix by POCS. Hence, the convergent property of BAM can be maintained. Simulation results show that the capacity of BAM with EHCA is greatly improved.     

A new complex-valued formulation and eigenvalue analysis of the Helmholtz equation by boundary element method

By considering the close relationship between the multiple reciprocity boundary element formulation and that of the fundamental solution of the Helmholtz differential operator, we present a new complex-valued integral equation formulation for the eigenvalue analysis of the scalar-valued Helmholtz equation. Eigenvalues are determined as local minima of the determinant of the coefficient matrix of the discretized equation iteratively by the Newton scheme. The necessary recurrence formula is derived and computed with high efficiency, due to polynomial representation of the matrix components. Some example computations demonstrate the utility of the proposed formulation and eigenvalue determination scheme, and construction of adaptive boundary elements for the eigenvalue determination is attempted.     

神经网络关联存储器的设计及应用

神经网络的存储能力一直是一个重大的缺陷,其存储主要体现在权重系数上,因此参数量一多,训练起来就十分困难。给神经网络设计一个外部关联存储器,能有效对神经网络的输入进行关联查询,并将查询的结果作为辅助输入传入到神经网络中去。此外,设计了自然语言语句的向量嵌入模型,并将模型和关联存储器集合起来形成一个自动关联语句语义向量的关联存储系统,其性能指标达到了设计要求。     

A modified Hopfield auto-associative memory with improved capacity

This paper describes a new procedure to implement a recurrent neural network (RNN), based on a new approach to the well-known Hopfield autoassociative memory. In our approach a RNN is seen as a complete graph G and the learning mechanism is also based on Hebb's law, but with a very significant difference: the weights, which control the dynamics of the net, are obtained by coloring the graph G. Once the training is complete, the synaptic matrix of the net will be the weight matrix of the graph. Any one of these matrices will fulfil some spatial properties, for this reason they will be referred to as tetrahedral matrices. The geometrical properties of these tetrahedral matrices may be used for classifying the n-dimensional state-vector space in n classes. In the recall stage, a parameter vector is introduced, which is related with the capacity of the network. It may be shown that the bigger the value of the ith component of the parameter vector is, the lower the capacity of the [i] class of the state-vector space becomes. Once the capacity has been controlled, a new set of parameters that uses the statistical deviation of the prototypes to compare them with those that appear as fixed points is introduced, eliminating thus a great number of parasitic fixed points.     

A new method for the design of energy transfer filters

This paper is concerned with the development of a new method for the design of energy transfer filters (ETFs). ETFs are a new class of non-linear filters recently proposed by the authors, which employ non-linear effects to transfer signal energy from one frequency band to a different frequency location. The new method uses the powerful orthogonal least squares (OLS) algorithm to solve the least squares problem associated with the design and compared with previous methods achieves much better filtering performance.     

A Hopfield network learning method for bipartite subgraph problem

We present a gradient ascent learning method of the Hopfield neural network for bipartite subgraph problem. The method is intended to provide a near-optimum parallel algorithm for solving the bipartite subgraph problem. To do this we use the Hopfield neural network to get a near-maximum bipartite subgraph, and increase the energy by modifying weights in a gradient ascent direction of the energy to help the network escape from the state of the near-maximum bipartite subgraph to the state of the maximum bipartite subgraph or better one. A large number of instances are simulated to verify the proposed method with the simulation results showing that the solution quality is superior to that of best existing parallel algorithm. We also test the learning method on total coloring problem. The simulation results show that our method finds optimal solution in every test graph.     

A new memory allocation method for shared memory multiprocessors with large virtual address space

This paper proposes a new memory allocation method for shared memory multiprocessors with large virtual address spaces. An evaluation of its performance is also presented. For effective use of shared memory multiprocessors, it is important that no processor's execution is blocked. If several processors simultaneously access a shared variable, their processes are blocked and access to the variable is serialized. Thus, frequent access to shared variables reduces the parallelism. In particular, the parallelism is significantly reduced when a special shared variable – the ‘allocation pointer’ – is frequently accessed in the dynamic object allocation by an application program. In this paper, we propose a new method for allocating physical memory pages where the allocation pointer is monotonically increased in the virtual address space in contrast to the conventional method. This allows the critical sections for access to the allocation pointer to be executed effectively and atomically by using the fetch-and-add primitive. Our method improves the application program's parallelism by access to the allocation pointer with considerably short blocking time to the process. © 1997 John Wiley & Sons, Ltd.     

Gabor wavelet associative memory for face recognition

This letter describes a high-performance face recognition system by combining two recently proposed neural network models, namely Gabor wavelet network (GWN) and kernel associative memory (KAM), into a unified structure called Gabor wavelet associative memory (GWAM). GWAM has superior representation capability inherited from GWN and consequently demonstrates a much better recognition performance than KAM. Extensive experiments have been conducted to evaluate a GWAM-based recognition scheme using three popular face databases, i.e., FERET database, Olivetti-Oracle Research Lab (ORL) database and AR face database. The experimental results consistently show our scheme's superiority and demonstrate its very high-performance comparing favorably to some recent face recognition methods, achieving 99.3% and 100% accuracy, respectively, on the former two databases, exhibiting very robust performance on the last database against varying illumination conditions.     

On multiple training for bidirectional associative memory

The minimal number of times for using a pair for training to guarantee recall of that pair among a set of training pairs is derived for a bidirectional associative memory.     

Two coding strategies for bidirectional associative memory

Enhancements of the encoding strategy of a discrete bidirectional associative memory (BAM) reported by B. Kosko (1987) are presented. There are two major concepts in this work: multiple training, which can be guaranteed to achieve recall of a single trained pair under suitable initial conditions of data, and dummy augmentation, which can be guaranteed to achieve recall of all trained pairs if attaching dummy data to the training pairs is allowable. In representative computer simulations, multiple training has been shown to lead to an improvement over the original Kosko strategy for recall of multiple pairs as well. A sufficient condition for a correlation matrix to make the energies of the training pairs be local minima is discussed. The use of multiple training and dummy augmentation concepts are illustrated, and theorems underlying the results are presented.     

Memristor-based chaotic neural networks for associative memory

In chaotic neural networks, the rich dynamic behaviors are generated from the contributions of spatio-temporal summation, continuous output function, and refractoriness. However, a large number of spatio-temporal summations in turn make the physical implementation of a chaotic neural network impractical. This paper proposes and investigates a memristor-based chaotic neural network model, which adequately utilizes the memristor with unique memory ability to realize the spatio-temporal summations in a simple way. Furthermore, the associative memory capabilities of the proposed memristor-based chaotic neural network have been demonstrated by conventional methods, including separation of superimposed pattern, many-to-many associations, and successive learning. Thanks to the nanometer scale size and automatic memory ability of the memristors, the proposed scheme is expected to greatly simplify the structure of chaotic neural network and promote the hardware implementation of chaotic neural networks.     

A new learning algorithm with logarithmic performance index for complex-valued neural networks

In a fully complex-valued feed-forward network, the convergence of the Complex-valued Back Propagation (CBP) learning algorithm depends on the choice of the activation function, learning sample distribution, minimization criterion, initial weights and the learning rate. The minimization criteria used in the existing versions of CBP learning algorithm in the literature do not approximate the phase of complex-valued output well in function approximation problems. The phase of a complex-valued output is critical in telecommunication and reconstruction and source localization problems in medical imaging applications. In this paper, the issues related to the convergence of complex-valued neural networks are clearly enumerated using a systematic sensitivity study on existing complex-valued neural networks. In addition, we also compare the performance of different types of split complex-valued neural networks. From the observations in the sensitivity analysis, we propose a new CBP learning algorithm with logarithmic performance index for a complex-valued neural network with exponential activation function. The proposed CBP learning algorithm directly minimizes both the magnitude and phase errors and also provides better convergence characteristics. Performance of the proposed scheme is evaluated using two synthetic complex-valued function approximation problems, the complex XOR problem, and a non-minimum phase equalization problem. Also, a comparative analysis on the convergence of the existing fully complex and split complex networks is presented.     

A 16-Kbit Θ-search associative memory

The design of a high-capacity Θ-search associative memory (Θ∈{<,>,⩽,⩾,=≠}) is presented. PSPICE simulation and layouts show that the proposed Θ-search associative memory chip consisting of 256 words, each 64-b long, can fit on a 13.5-mm×9.5-mm chip. It can perform maskable Θ-search operations over its contents in 110 ns