Stochastic Resonance in Recurrent Neural Network with Hopfield-Type Memory

Stochastic resonance (SR) is known as a phenomenon in which the presence of noise helps a nonlinear system in amplifying a weak (under barrier) signal. In this paper, we investigate how SR behavior can be observed in practical autoassociative neural networks with the Hopfield-type memory under the stochastic dynamics. We focus on SR responses in two systems which consist of three and 156 neurons. These cases are considered as effective double-well and multi-well models. It is demonstrated that the neural network can enhance weak subthreshold signals composed of the stored pattern trains and have higher coherence abilities between stimulus and response.     

Two digital circuits for a fully parallel stochastic neural network

This paper presents two digital circuits that allow the implementation of a fully parallel stochastic Hopfield neural network (SHNN). In a parallel SHNN with n neurons, the n*n stochastic signals s (ij) pulse with probability which are proportional to the synapse inputs, are simultaneously available. The proposed circuits calculate the summation of the stochastic input pulses to neuron i(F(i)=Sigma(j) s(ij)). The resulting network achieves considerable speed up with respect to the previous network.     

The novel aggregation function-based neuron models in complex domain

The computational power of a neuron lies in the spatial grouping of synapses belonging to any dendrite tree. Attempts to give a mathematical representation to the grouping process of synapses continue to be a fascinating field of work for researchers in the neural network community. In the literature, we generally find neuron models that comprise of summation, radial basis or product aggregation function, as basic unit of feed-forward multilayer neural network. All these models and their corresponding networks have their own merits and demerits. The MLP constructs global approximation to input–output mapping, while a RBF network, using exponentially decaying localized non-linearity, constructs local approximation to input–output mapping. In this paper, we propose two compensatory type novel aggregation functions for artificial neurons. They produce net potential as linear or non-linear composition of basic summation and radial basis operations over a set of input signals. The neuron models based on these aggregation functions ensure faster convergence, better training and prediction accuracy. The learning and generalization capabilities of these neurons have been tested over various classification and functional mapping problems. These neurons have also shown excellent generalization ability over the two-dimensional transformations.     

Stochastic synchrony of chaos in a pulse-coupled neural network with both chemical and electrical synapses among inhibitory neurons

The synchronous firing of neurons in a pulse-coupled neural network composed of excitatory and inhibitory neurons is analyzed. The neurons are connected by both chemical synapses and electrical synapses among the inhibitory neurons. When electrical synapses are introduced, periodically synchronized firing as well as chaotically synchronized firing is widely observed. Moreover, we find stochastic synchrony where the ensemble-averaged dynamics shows synchronization in the network but each neuron has a low firing rate and the firing of the neurons seems to be stochastic. Stochastic synchrony of chaos corresponding to a chaotic attractor is also found.     

基于PID神经网络的后非线性盲源分离算法

PID神经网络是一种新型的前向神经元网络，它的隐层单元包含比例（P）、积分（1）、微分（D）元，各层神经元个数、连接方式、连接权初值均按PID控制规律的基本原则确定。本文研究了一种新的后非线性盲源分离算法，用最大熵值方法推导了PID神经网络算法的后非线性分离学习公式，该算法可用于线性或后非线性的混叠信号。对输入2个混叠信号时，用单个PI神经网络分离；对输入3个混叠信号时，用单个PID神经网络分离；对输入更多的混叠信号时，可采用多个独立的PID神经网络来分离。仿真结果验证了单个PID神经网络算法，能分离线性或后非线性混叠信号。     

Coherent Response in a Chaotic Neural Network

We set up a signal-driven scheme of the chaotic neural network with the coupling constants corresponding to certain information, and investigate the stochastic resonance-like effects under its deterministic dynamics, comparing with the conventional case of Hopfield network with stochastic noise. It is shown that the chaotic neural network can enhance weak subthreshold signals and have higher coherence abilities between stimulus and response than those attained by the conventional stochastic model.     

基于径向基函数神经网络的混沌干扰信号检测

为提高强混沌背景下谐波信号的检测能力,提高系统的信噪比,提出了一种在混沌背景噪声中提取正弦信号的RBF神经网络方法。依据混沌吸引子固有的几何特性和混沌系统轨迹点在流形中的演化规律,建立混沌系统的RBF神经网络单步预测模型,改进了网络的学习算法,利用RBF神经网络对输入扰动的敏感,预测出误差信号。分析了在低信噪比下的检测性能。通过对Lorenz流和实际舰船辐射噪声信号中的信号检测进行计算机仿真实验,验证了算法的有效性和可行性,并且实验表明信噪比最低达-40dB时,仍能有效检测出信号。     

An Iterative Learning Control of Nonlinear Systems Using Neural Network Design

In this paper, a feedforward neural network with sigmoid hidden units is used to design a neural network based iterative learning controller for nonlinear systems with state dependent input gains. No prior offline training phase is necessary, and only a single neural network is employed. All the weights of the neurons are tuned during the iteration process in order to achieve the desired learning performance. The adaptive laws for the weights of neurons and the analysis of learning performance are determined via Lyapunov‐like analysis. A projection learning algorithm is used to prevent drifting of weights. It is shown that the tracking error vector will asymptotically converges to zero as the iteration goes to infinity, and the all adjustable parameters as well as internal signals remain bounded.     

Courbage神经元映射模型的随机共振研究

采用神经元二维映射模型,通过数字仿真研究了高斯白噪声对神经元非线性动力学特性的影响.研究发现,噪声可以诱导具有次阈值输入信号的神经元产生动作电位和随机共振.随机共振现象的产生与否和噪声强度的大小以及输入信号的频率具有密切的关系.另外,还研究了系统的控制参数对随机共振现象的影响.     

Compound binomial processes in neural integration

Explores some of the properties of stochastic digital signal processing in which the input signals are represented as sequences of Bernoulli events. The event statistics of the resulting stochastic process may be governed by compound binomial processes, depending upon how the individual input variables to a neural network are stochastically multiplexed. Similar doubly stochastic statistics can also result from datasets which are Bernoulli mixtures, depending upon the temporal persistence of the mixture components at the input terminals to the network. The principal contribution of these results is in determining the required integration period of the stochastic signals for a given precision in pulsed digital neural networks.     

Prescribed performance synchronization for uncertain chaotic systems with input saturation based on neural networks

In this paper, a prescribed performance adaptive neural network synchronization is investigated for a class of unknown chaotic systems in the presence of input saturation and external unknown disturbances. A prescribed performance function is employed to transform the constraint problem of chaotic synchronization control error into the problem of guaranteeing the boundedness of the transformed error. By introducing the Gaussian error function, the input saturation is handled. A neural network-based synchronization control scheme is then developed. Under the developed synchronization control scheme, the synchronization of uncertain chaotic systems is achieved with different initial conditions. Numerical simulation results further demonstrate the effectiveness of the proposed synchronization control scheme for unknown chaotic systems subject to external unknown disturbances and input saturation.

     

Intrinsic adaptation in autonomous recurrent neural networks

A massively recurrent neural network responds on one side to input stimuli and is autonomously active, on the other side, in the absence of sensory inputs. Stimuli and information processing depend crucially on the quality of the autonomous-state dynamics of the ongoing neural activity. This default neural activity may be dynamically structured in time and space, showing regular, synchronized, bursting, or chaotic activity patterns. We study the influence of nonsynaptic plasticity on the default dynamical state of recurrent neural networks. The nonsynaptic adaption considered acts on intrinsic neural parameters, such as the threshold and the gain, and is driven by the optimization of the information entropy. We observe, in the presence of the intrinsic adaptation processes, three distinct and globally attracting dynamical regimes: a regular synchronized, an overall chaotic, and an intermittent bursting regime. The intermittent bursting regime is characterized by intervals of regular flows, which are quite insensitive to external stimuli, interceded by chaotic bursts that respond sensitively to input signals. We discuss these findings in the context of self-organized information processing and critical brain dynamics.     

Simulations of construction learning for neuron-computer resources

It is well known that information processing in the brain depends on neuron systems. Simple neuron systems are neural networks, and their learning methods have been studied. However, we believe that research on large-scale neural network systems is still incomplete. Here, we propose a learning method for millions of neurons as resources for a neuron computer. The method is a type of recurrent path-selection, so the neural network objective must have nesting structures. This method is executed at high speed. When information processing is executed by analogue signals, the accumulation of errors is a grave problem. We equipped a neural network with a digitizer and AD/DA (Analogue Digital) converters constructed of neurons. They retain all information signals and guarantee precision in complex operations. By using these techniques, we generated an image shifter constructed of 8.6 million neurons. We believe that there is the potential to design a neuron computer using this scheme. This work was presented in part at the Fifth International Symposium on Artificial Life and Robotics, Oita, Japan, January 26–28, 2000     

Adaptive near optimal neural control for a class of discrete-time chaotic system

In this paper, an adaptive critic neural network controller is designed for a class of discrete-time chaotic system. The critic neural network is used to approximate the long-term function. In contrast with the existing results for discrete-time chaotic systems, in this paper, a near optimal control input can be generated when the long-term function is minimized. It is proven that the tracking error, the adaptation laws and the control input are uniformly bounded. A simulation example is employed to illustrate the effectiveness of the proposed algorithm.     

An Annealed Chaotic Competitive Learning Network with Nonlinear Self-feedback and Its Application in Edge Detection

An unsupervised parallel approach called Annealed Chaotic Competitive Learning Network (ACCLN) for the optimization problem is proposed in this paper. The goal is to modify an unsupervised scheme based on the competitive neural network using the chaotic technique governed by an annealing strategy so that on-line learning and parallel implementation to find near-global solution for image edge detection is feasible. In the ACCLN, the edge detection is conceptually considered as a clustering problem. Here, it is a kind of competitive learning network model imposed by a 2-dimensional input layer and an output layer working toward minimizing an objective function defined as the contextual information. The interconnection strength, composed by an internal state and a transient state with a non-linear self-feedback manner, is connected between neurons in input and output layers. To harness the chaotic dynamic and convergence process, an annealing strategy is also embedded into the ACCLN. In addition to retain the characteristics of the conventional neural units, the ACCLN displays a rich range of behavior reminiscent of that observed in neurons. Unlike the conventional neural network, the ACCLN has rich range and flexible dynamics, so that it can be expected to have higher ability of searching for globally optimal or near-optimum results.     

Generalized and partial synchronization of coupled neural networks

Synchronization of neural signals has been proposed as a temporal coding scheme representing cooperated computation in distributed cortical networks. Previous theoretical studies in that direction mainly focused on the synchronization of coupled oscillatory subsystems and neglected more complex dynamical modes, that already exist on the single-unit level. In this paper we study the parametrized time-discrete dynamics of two coupled recurrent networks of graded neurons. Conditions for the existence of partially synchronized dynamics of these systems are derived, referring to a situation where only subsets of neurons in each sub-network are synchronous. The coupled networks can have different architectures and even a different number of neurons. Periodic as well as quasiperiodic and chaotic attractors constrained to a manifold M of synchronized components are observed. Examples are discussed for coupled 3-neuron networks having different architectures, and for coupled 2-neuron and 3-neuron networks. Partial synchronization of different degrees is demonstrated by numerical results for selected sets of parameters. In conclusion, the results show that synchronization phenomena far beyond completely synchronized oscillations can occur even in simple coupled networks. The type of the synchronization depends in an intricate way on stimuli, history and connectivity as well as other parameters of the network. Specific inputs can further switch between different operational modes in a complex way, suggesting a similarly rich spatio-temporal behaviour in real neural systems.     

Neural networks with quantum gated nodes

With a view to investigating similarities in aspects of biological neural networks with quantum ones, so that quantum machines can be developed in future with some of the advantages of biological systems of information processing where a certain amount of indeterminism and the multiple connectivities between nodes offer advantages not seemingly obtainable from usual electronic devices working with classical gates, we present here some results for a quantum neural network with quantum gates. After reviewing the general principles of a biological network and a quantum one, we study a specific model network with qubits, i.e. quantum bits, replacing classical neurons having deterministic states, and also with quantum operators in place of the classical action potentials observed in biological contexts. With our choice of gates interconnecting the neural lattice, the state of the system behaves in ways reflecting both the strength of coupling between neurons as well as the initial conditions, as in biological systems. We find that, depending on whether there is a threshold for emission from excited to ground state, the system shows either chaotic oscillations or coherent ones with periodicity that depends on the strength of coupling. The initial input also affects the subsequent dynamic behavior of the system, which indicates that it can serve as a dynamic memory system analogous to biological ones. Our results seem to suggest that such quantum networks may contain some advantageous features of biological systems more efficiently than classical electronic devices.     

ADAPTATION BASED ON MEMORY DYNAMICS IN A CHAOTIC NEURAL NETWORK

The complex dynamics that emerge from systems governed by deterministic chaos offer significant advantages to the neuromorphic engineer. Included in these is the potential for a very large memory store and the ease with which chaotic systems can be controlled. By definition, a chaotic system is a periodic. However, during the course of its trajectory through state space, the chaotic system will come infinitely close to points that it has previously visited. These almost repeating trajectories are referred to as Unstable Periodic Orbits (UPOs). Normally, under the influence of chaos, the trajectory would move away exponentially fast from its previous path, thereby describing a new path on the surface of the attractor. It is possible to apply a simple delayed feedback control mechanism to a chaotic system that will constrain it within one of its UPOs. This article presents a neural implementation of this delayed feedback mechanism. The network presented here is able to stabilize different UPOs in response to different input signals, with each UPO corresponding to a dynamic recognition state for that input. We also present two learning rules for this network, which enables it to adapt to novel inputs in a self-organized manner.     

混沌神经网络及其在最优化问题中的应用

首先评述了三种混沌神经网络模型,然后提出了一种新的混沌模拟退火算法。其次将四种方法分别应用于10个城市的施行推销商问题。文中给出了每一模型神经元输出和能量函数随时间演变过程曲线。根据仿真结果,讨论了四种方法的特性与有效。其结论为：提出的模拟退火神经网络比其它网络模型更能获得全局最小解。     

A Network of Neural Oscillators for Fractal Pattern Recognition

Biological neural networks are high dimensional nonlinear systems, which presents complex dynamical phenomena, such as chaos. Thus, the study of coupled dynamical systems is important for understanding functional mechanism of real neural networks and it is also important for modeling more realistic artificial neural networks. In this direction, the study of a ring of phase oscillators has been proved to be useful for pattern recognition. Such an approach has at least three nontrivial advantages over the traditional dynamical neural networks: first, each input pattern can be encoded in a vector instead of a matrix; second, the connection weights can be determined analytically; third, due to its dynamical nature, it has the ability to capture temporal patterns. In the previous studies of this topic, all patterns were encoded as stable periodic solutions of the oscillator network. In this paper, we continue to explore the oscillator ring for pattern recognition. Specifically, we propose algorithms, which use the chaotic dynamics of the closed loops of Stuart–Landau oscillators as artificial neurons, to recognize randomly generated fractal patterns. We manipulate the number of neurons and initial conditions of the oscillator ring to encode fractal patterns. It is worth noting that fractal pattern recognition is a challenging problem due to their discontinuity nature and their complex forms. Computer simulations confirm good performance of the proposed algorithms.