Multilayer associative neural networks (MANN''s): storage capacityversus perfect recall

The objective of this paper is to to resolve important issues in artificial neural nets-exact recall and capacity in multilayer associative memories. These problems have imposed restrictions on coding strategies. We propose the following triple-layered hybrid neural network: the first synapse is a one-shot associative memory using the modified Kohonen's adaptive learning algorithm with arbitrary input patterns; the second one is Kosko's bidirectional associative memory consisting of orthogonal input/output basis vectors such as Walsh series satisfying the strict continuity condition; and finally, the third one is a simple one-shot associative memory with arbitrary output images. A mathematical framework based on the relationship between energy local minima (capacity of the neural net) and noise-free recall is established. The robust capacity conditions of this multilayer associative neural network that lead to forming the local minima of the energy function at the exact training pairs are derived. The chosen strategy not only maximizes the total number of stored images but also completely relaxes any code-dependent conditions of the learning pairs.     

Quaternionic multistate Hopfield neural network with extended projection rule

The aim of this paper is to investigate storing and recalling performances of embedded patterns on associative memory. The associative memory is composed of quaternionic multistate Hopfield neural network. The state of a neuron in the network is described by three kinds of discretized phase with fixed amplitude. These phases are set to discrete values with arbitrary divide size. Hebbian rule and projection rule are used for storing patterns to the network. Recalling performance is evaluated through storing random patterns with changing the divide size of the phases in a neuron. Color images are also embedded and their noise tolerance is explored.     

The modulation of learning state in a biological associative memory: an in vitro,in vivo,and in computo study of object recognition in mammalian olfactory cortex

During learning of overlapping input patterns in an associative memory, recall of previously stored patterns can interfere with the learning of new patterns. Most associative memory models avoid this difficulty by ignoring the effect of previously modified connections during learning, by clamping network activity to the patterns to be learned. Through the interaction of experimental and modeling techniques, we now have evidence to suggest that a somewhat analogous approach may have been taken by biology within the olfactory cerebral cortex. Specifically we have recently discovered that the naturally occurring neuromodulator acetylcholine produces a variety of effects on cortical cells and circuits which, when taken together, can prevent memory interference in a biologically realistic memory model. Further, it has been demonstrated that these biological mechanisms can actually improve the memory storage performance of previously published abstract neural network associative memory models.     

An RCE-based Associative Memory with Application to Human Face Recognition

Many models of neural network-based associative memory have been proposed and studied. However, most of these models do not have a rejection mechanism and hence are not practical for many real-world associative memory problems. For example, in human face recognition, we are given a database of face images and the identity of each image. Given an input image, the task is to associate when appropriate the image with the corresponding name of the person in the database. However, the input image may be that of a stranger. In this case, the system should reject the input. In this paper, we propose a practical associative memory model that has a rejection mechanism. The structure of the model is based on the restricted Coulomb energy (RCE) network. The capacity of the proposed memory is desibed by two measures: the ability of the system to correctly identify known individuals, and the ability of the system to reject individuals who are not in the database. Experimental results are given which show how the performance of the system varies as the size of the database increases up to 1000 individuals.     

Fuzzy systems and neural networks in software engineering project management

To make reasonable estimates of resources, costs, and schedules, software project managers need to be provided with models that furnish the essential framework for software project planning and control by supplying important management numbers concerning the state and parameters of the project that are critical for resource allocation. Understanding that software development is not a mechanistic process brings about the realization that parameters that characterize the development of software possess an inherent fuzziness, thus providing the rationale for the development of realistic models based on fuzzy set or neural theories.Fuzzy and neural approaches offer a key advantage over traditional modeling approaches in that they aremodel-free estimators. This article opens up the possibility of applying fuzzy estimation theory and neural networks for the purpose of software engineering project management and control, using Putnam's manpower buildup index (MBI) estimation model as an example. It is shown that the MBI selection process can be based upon 64 different fuzzy associative memory (FAM) rules. The same rules are used to generate 64 training patterns for a feedforward neural network. The fuzzy associative memory and neural network approaches are compared qualitatively through estimation surfaces. The FAM estimation surfaces are stepped, whereas those from the neural system are smooth. Also, the FAM system sets up much faster than the neural system. FAM rules obtained from logical antecedent-consequent pairs are maintained distinct, giving the user the ability to determine which FAM rule contributed how much membership activation to a concluded output.     

An adaptive associative memory system based on autonomous reaction between image memories

To construct a “thinking-like” processing system, a new architecture of an adaptive associative memory system is proposed. This memory system treats “images” as basic units of information, and adapts to the environment of the external world by means of autonomous reactions between the images. The images do not have to be clear, distinct symbols or patterns; they can be ambiguous, indistinct symbols or patterns as well. This memory system is a kind of neural network made up of nodes and links called a localist spreading activation network. Each node holds one image in a localist manner. Images in high-activity nodes interact autonomously and generate new images and links. By this reaction between images, various forms of images are generated automatically under constraints of links with adjacent nodes. In this system, three simple image reaction operations are defined. Each operation generates a new image by combining pseudofigures or features and links of two images. This work was presented, in part, at the Fourth International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–22, 1999     

Double-pattern associative memory neural network with pattern loop

A double-pattern associative memory neural network with “pattern loop” is proposed. It can store 2N bit bipolar binary patterns up to the order of 2^2N , retrieve part or all of the stored patterns which all have the minimum Hamming distance with input pattern, completely eliminate spurious patterns, and has higher storing efficiency and reliability than conventional associative memory. The length of a pattern stored in this associative memory can be easily extended from 2N to κN.     

Distance-Based Sparse Associative Memory Neural Network Algorithm for Pattern Recognition

A sparse two-Dimension distance weighted approach for improving the performance of exponential correlation associative memory (ECAM) and modified exponential correlation associative memory (MECAM) is presented in this paper. The approach is inspired by biological visual perception mechanism and extensively existing sparse small-world network phenomenon. By means of the approach, the two new associative memory neural networks, i.e., distance-based sparse ECAM (DBS-ECAM) and distance-based sparse MECAM (DBS-MECAM), are induced by introducing both the decaying two-Dimension distance factor and small-world architecture into ECAM and MECAM’s evolution rule for image processing application. Such a new configuration can reduce the connection complexity of conventional fully connected associative memories so that makes AM’ VLSI implementation easier. More importantly, the experiments performed on the binary visual images show DBS-ECAM and DBS-MECAM can learn and recognize patterns more effectively than ECAM and MECAM, respectively.     

Hopfield在计算机编程中的应用与研究

Hopfield网络,又称联想记忆网络。文中根据Hopfleld神经网络构造一个应用于计算机代码编程中的联想存储器。联想记忆是该存储器的重要功能,它具有信息记忆和信息联想的特点,能够从不完整的或模糊的信息联想出存储在记忆中的某个完整清晰的信息模式。根据这一原理,用H0pfield联想存储器知识和eclipse插件机制来搭建嵌入在eclipse开发工具中一个知识可拓展的动态帮助插件,实现根据残缺不全的java代码联想到完整的java代码的功能,并进一步阐述Hopfield神经网络在计算机代码编程中的应用前景和发展方向。     

3D Filtering by Block Matching and Convolutional Neural Network for Image Denoising

Block matching based 3D filtering methods have achieved great success in image denoising tasks. However, the manually set filtering operation could not well describe a good model to transform noisy images to clean images. In this paper, we introduce convolutional neural network (CNN) for the 3D filtering step to learn a well fitted model for denoising. With a trainable model, prior knowledge is utilized for better mapping from noisy images to clean images. This block matching and CNN joint model (BMCNN) could denoise images with different sizes and different noise intensity well, especially images with high noise levels. The experimental results demonstrate that among all competing methods, this method achieves the highest peak signal to noise ratio (PSNR) when denoising images with high noise levels (σ > 40), and the best visual quality when denoising images with all the tested noise levels.     

基于一种新型联想记忆神经网络的传感器故障诊断

联想记忆网络是一种反馈型神经网络。由于反馈型网络会收敛于某个稳定状态，因此，可用于联想记忆。神经网络具有高度的并行处理能力和极强的非线性映射能力，可以实现故障与征兆之间复杂的非线性映射关系，因此在机械故障诊断领域中显示了很大的应用潜力。本文以模拟人脑由部分记忆而联想整体的特点为基础，通过引入联想记忆衰减因子，改进神经网络结构和学习算法．应用于系统的故障诊断。     

SAR despeckling neural network with logarithmic convolutional product model

ABSTRACT

Synthetic aperture radar (SAR) images are inevitably contaminated by speckle noise due to its coherent imaging mechanism. Speckle noise obscures the intrinsic radar cross section (RCS) information in SAR images. This article proposes a novel deep neural network architecture specifically designed for despeckling purpose. It uses a convolutional neural network to extract image features and reconstruct a discrete RCS probability density function (PDF). It is trained by a hybrid loss function which measures the distance between the actual SAR image intensity PDF and the estimated one which is derived from convolution between the reconstructed RCS PDF and prior speckle PDF. The network can be trained by either purely simulated image patches or real SAR images. Experiment results on both simulated SAR images and real NASA/JPL AIRSAR images are used to test the performance, and the results show the efficacy of the proposed despeckling neural network compared with three state-of-the-art filters.     

基于离散Hopfield神经网络的噪声数字识别

在Hebb学习规则的基础上,运用离散Hopficld神经网络的联想记忆能力,对含有噪声而产生畸变的0~9数字进行了识别。通过改进神经网络的记忆样本,即先对记忆样本做正交化处理,再对改进后的记忆样本进行学习,得到相应的权值矩阵,然后利用改进后的离散Hopfield神经网络根据待识别噪声数字的信息联想已记忆的数字。实验结果表明,改进后的神经网络对噪声数字有较好的识别效果,提高了记忆能力和识别的正确率。     

A MOS circuit for a nonmonotonic neural network with excellentretrieval capabilities

An MOS circuit is proposed for implementing a nonmonotonic transfer characteristic of a neural network. The present research is motivated by the recent results of theoretical studies showing excellent equilibrium properties of networks with the nonmonotonic neural units. These properties include enhancement of storage capacity and complete elimination of noise in associative memory recall. The simple form of the transfer characteristic enables one to implement it with a simple electrical circuit of standard MOS transistors. SPICE simulation results are shown for the behavior of the neural units in associative memory recall.     

Computer simulations of association-based image search mechanisms basing on theory of active perception

We discuss an approach to development of an associative memory model from the viewpoint of the theory of active perception. The theory of active perception allows one to develop the treebased memory model without the defects of the kd-tree and the vp-tree. Applications of the proposed model for solving problems of an image search by content from database are described. Also, we present the results of computer simulations directed at searching of similar and distorted images.     

Associative Memory Based Image and Object Recognition by Quantum Holography

A quantum associative memory, much more natural than those of quantum computers, is presented. Neural-net-like processing with real-valued variables is transformed into processing with quantum waves. Successful computer simulations of image storage and retrieval are reported. Our Hopfield-like algorithm allows quantum implementation with holographic procedure using present-day quantum-optics techniques. This brings many advantages over classical Hopfield neural nets and quantum computers with logic gates.     

利用动态核的形态联想记忆网络的研究

在文献[1]的基础上,提出了一个基于动态核的形态联想记忆网络方法,特点是同一幅图像,如果其所含的噪声情况不同,则其核也将不同,从而较好地解决了图像含有随机噪声时的联想记忆问题。实验证明,此方法具有良好的性能,双向联想记忆的准确率优于文献[1]中介绍的方法。     

Color Characterization and Balancing by a Nonlinear Line Attractor Network for Image Enhancement

A novel method to map high dynamic range scenes to low dynamic range images utilizing the concept of color characterization, enhancement, and balancing is described in this letter. Each pixel of the image is first characterized by extracting the relationship of the red, green, and blue components along with its corresponding neighbors using a nonlinear line attractor network to form an associative memory. Then, the illumination enhancement process is performed using a hyperbolic tangent function to provide dynamic range compression to each pixel in the image. The slope of the hyperbolic tangent function is controlled using a parameter that is determined by the local and global statistics of the image to facilitate the change of the intensity level. A color balancing process restores the original color characteristics of the image based on learned associative memory matrices which eliminate image distortion due to improper recombination of red, green and blue components after enhancement. Experiments conducted on images captured at extremely uneven lighting environments show that the proposed method outperforms other image enhancement algorithms.     

NDRAM: nonlinear dynamic recurrent associative memory for learning bipolar and nonbipolar correlated patterns

This paper presents a new unsupervised attractor neural network, which, contrary to optimal linear associative memory models, is able to develop nonbipolar attractors as well as bipolar attractors. Moreover, the model is able to develop less spurious attractors and has a better recall performance under random noise than any other Hopfield type neural network. Those performances are obtained by a simple Hebbian/anti-Hebbian online learning rule that directly incorporates feedback from a specific nonlinear transmission rule. Several computer simulations show the model's distinguishing properties.     

Machine learning and directional switching median-based filter for highly corrupted images

In this paper, two-stage machine learning-based noise detection scheme has been proposed for identification of salt-and- pepper impulse noise which gives excellent detection results for highly corrupted images. In the first stage, a window of size $3\times 3$ is taken from image and some other features of this window are used as input to neural network. This scheme has distinction of having very low missed detection (MD) and false positives rates. In the second stage, decision tree-based algorithm (J48) is applied on some well-known statistical parameters to generate rules for noise detection. These noise detection methods give promising results for identification of noise from highly corrupted images. A modified version of switching median filter (directional weighted switching median filter) is proposed for noise removal. Performance of noise detector is measured using MD and false alarm FA. Filtering results are compared with state-of-the-art noise removal techniques in terms of peak signal-to-noise ratio and structural similarity index measure. Extensive experiments are performed to show that the proposed technique gives better results than state-of-the-art noise detection and filtering methods.