Neuron-synapse IC chip-set for large-scale chaotic neural networks.

We propose a neuron-synapse integrated circuit (IC) chip-set for large-scale chaotic neural networks. We use switched-capacitor (SC) circuit techniques to implement a three-internal-state transiently-chaotic neural network model. The SC chaotic neuron chip faithfully reproduces complex chaotic dynamics in real numbers through continuous state variables of the analog circuitry. We can digitally control most of the model parameters by means of programmable capacitive arrays embedded in the SC chaotic neuron chip. Since the output of the neuron is transfered into a digital pulse according to the all-or-nothing property of an axon, we design a synapse chip with digital circuits. We propose a memory-based synapse circuit architecture to achieve a rapid calculation of a vast number of weighted summations. Both of the SC neuron and the digital synapse circuits have been fabricated as IC forms. We have tested these IC chips extensively, and confirmed the functions and performance of the chip-set. The proposed neuron-synapse IC chip-set makes it possible to construct a scalable and reconfigurable large-scale chaotic neural network with 10000 neurons and 10000/sup 2/ synaptic connections.     

Improved Izhikevich neurons for spiking neural networks

Spiking neural networks constitute a modern neural network paradigm that overlaps machine learning and computational neurosciences. Spiking neural networks use neuron models that possess a great degree of biological realism. The most realistic model of the neuron is the one created by Alan Lloyd Hodgkin and Andrew Huxley. However, the Hodgkin–Huxley model, while accurate, is computationally very inefficient. Eugene Izhikevich created a simplified neuron model based on the Hodgkin–Huxley equations. This model has better computational efficiency than the original proposed by Hodgkin and Huxley, and yet it can successfully reproduce all known firing patterns. However, there are not many articles dealing with implementations of this model for a functional neural network. This study presents a spiking neural network architecture that utilizes improved Izhikevich neurons with the purpose of evaluating its speed and efficiency. Since the field of spiking neural networks has reinvigorated the interest in biological plausibility, biological realism was an additional goal. The network is tested on the correct classification of logic gates (including XOR) and on the iris dataset. Results and possible improvements are also discussed.     

Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses

A mixed-signal very large scale integration (VLSI) chip for large scale emulation of spiking neural networks is presented. The chip contains 2400 silicon neurons with fully programmable and reconfigurable synaptic connectivity. Each neuron implements a discrete-time model of a single-compartment cell. The model allows for analog membrane dynamics and an arbitrary number of synaptic connections, each with tunable conductance and reversal potential. The array of silicon neurons functions as an address-event (AE) transceiver, with incoming and outgoing spikes communicated over an asynchronous event-driven digital bus. Address encoding and conflict resolution of spiking events are implemented via a randomized arbitration scheme that ensures balanced servicing of event requests across the array. Routing of events is implemented externally using dynamically programmable random-access memory that stores a postsynaptic address, the conductance, and the reversal potential of each synaptic connection. Here, we describe the silicon neuron circuits, present experimental data characterizing the 3 mm times 3 mm chip fabricated in 0.5-mum complementary metal-oxide-semiconductor (CMOS) technology, and demonstrate its utility by configuring the hardware to emulate a model of attractor dynamics and waves of neural activity during sleep in rat hippocampus     

A subthreshold MOS neuron circuit based on the Volterra system.

We present an analog neuron circuit consisting of a small number of metal-oxide semiconductor (MOS) devices operating in their subthreshold region. The dynamics of the circuit were designed to be equivalent to the well-known Volterra system to facilitate developing the circuit for a particular application. We show that a simple nonlinear transformation of system variables in the Volterra system enables designing a neuron-like oscillator, which can produce sequences in time of identically shaped pulses (spikes) by using current-mode subthreshold MOS circuits. We present experimental results of the fabricated neuron circuits as well as an application in an inhibitory neural network, where the neurons compete with each other in the frequency and time domains.     

Synaptic dynamics in analog VLSI

Synapses are crucial elements for computation and information transfer in both real and artificial neural systems. Recent experimental findings and theoretical models of pulse-based neural networks suggest that synaptic dynamics can play a crucial role for learning neural codes and encoding spatiotemporal spike patterns. Within the context of hardware implementations of pulse-based neural networks, several analog VLSI circuits modeling synaptic functionality have been proposed. We present an overview of previously proposed circuits and describe a novel analog VLSI synaptic circuit suitable for integration in large VLSI spike-based neural systems. The circuit proposed is based on a computational model that fits the real postsynaptic currents with exponentials. We present experimental data showing how the circuit exhibits realistic dynamics and show how it can be connected to additional modules for implementing a wide range of synaptic properties.     

Short-term memory circuit using hardware ring neural networks

A number of studies have recently been made on various neuron models and neural networks. This research is studied for applications to engineering problems and an understanding of the information processing functions of living organisms. We are studying an asynchronous neural network using a pulse-type hardware neuron model (P-HNM). Recently, we have been trying to construct a short-term memory circuit using hardware ring neural networks (RNN) with P-HNM. In this article, we discuss the construction of a short-term memory circuit using the hardware RNN, and conduct experiments that explain the characteristics of the network through circuit simulation using PSpice. As a result, we verify that the RNN which is proposed in this article can be used as the short-term memory circuit.This work was presented, in part, at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004     

Time series prediction with single multiplicative neuron model

Single neuron models are typical functional replica of the biological neuron that are derived using their individual and group responses in networks. In recent past, a lot of work in this area has produced advanced neuron models for both analog and binary data patterns. Popular among these are the higher-order neurons, fuzzy neurons and other polynomial neurons. In this paper, we propose a new neuron model based on a polynomial architecture. Instead of considering all the higher-order terms, a simple aggregation function is used. The aggregation function is considered as a product of linear functions in different dimensions of the space. The functional mapping capability of the proposed neuron model is demonstrated through some well known time series prediction problems and is compared with the standard multilayer neural network.     

脉冲神经网络研究进展综述

近年来,起源于计算神经科学的脉冲神经网络因其具有丰富的时空动力学特征、多样的编码机制、契合硬件的事件驱动特性等优势,在神经形态工程和类脑计算领域已得到广泛的关注.脉冲神经网络与当前计算机科学导向的以深度卷积网络为代表的人工神经网络的交叉融合被认为是发展人工通用智能的有力途径.对此,回顾了脉冲神经网络的发展历程,将其划分为神经元模型、训练算法、编程框架、数据集以及硬件芯片等5个重点方向,全方位介绍脉冲神经网络的最新进展和内涵,讨论并分析了脉冲神经网络领域各个重点方向的发展机遇和挑战.希望本综述能够吸引不同学科的研究者,通过跨学科的思想交流与合作研究,推动脉冲神经网络领域的发展.     

Solution methods for a new class of simple model neurons

Izhikevich (2003) proposed a new canonical neuron model of spike generation. The model was surprisingly simple yet able to accurately replicate the firing patterns of different types of cortical cell. Here, we derive a solution method that allows efficient simulation of the model.     

Modeling and complexity in neural networks

In this paper, we study nonlinear spatio-temporal dynamics in synchronous and asynchronous chaotic neural networks from the viewpoint of the modeling and complexity of the dynamic brain. First, the possible roles and functions of spatio-temporal neurochaos are considered with a model of synchronous chaotic neural networks composed of a neuron model with a chaotic map. Second, deterministic point-process dynamics with spikes of action potentials is demonstrated with a biologically more plausible model of asynchronous chaotic neural networks. Last, the possibilities of inventing a new brain-type of computing system are discussed on the basis of these models of chaotic neural networks. This work was presented, in part, at the Third International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–21, 1998.     

Equivalence between some dynamical systems for optimization

It is shown by the derivation of solution methods for an elementary optimization problem that the stochastic relaxation in image analysis, the Potts neural networks for combinatorial optimization and interior point methods for nonlinear programming have common formulation of their dynamics. This unification of these algorithms leads us to possibility for real time solution of these problems with common analog electronic circuits.     

Analog implementation of pulse-coupled neural networks

This paper presents a compact architecture for analog CMOS hardware implementation of voltage-mode pulse-coupled neural networks (PCNN). The hardware implementation methods shows inherent fault tolerance specialties and high speed, which is usually more than an order of magnitude over the software counterpart. A computational style described in this article mimics a biological neural network using pulse-stream signaling and analog summation and multiplication, pulse-stream encoding technique uses pulse streams to carry information and control analog circuitry, while storing further analog information on the time axis. The main feature of the proposed neuron circuit is that the structure is compact, yet exhibiting all the basic properties of natural biological neurons. Functional and structural forms of neural and synaptic functions are presented along with simulation results. Finally, the proposed design is applied to image processing to demonstrate successful restoration of images and their features.     

An analog silicon retina with multichip configuration

The neuromorphic silicon retina is a novel analog very large scale integrated circuit that emulates the structure and the function of the retinal neuronal circuit. We fabricated a neuromorphic silicon retina, in which sample/hold circuits were embedded to generate fluctuation-suppressed outputs in the previous study . The applications of this silicon retina, however, are limited because of a low spatial resolution and computational variability. In this paper, we have fabricated a multichip silicon retina in which the functional network circuits are divided into two chips: the photoreceptor network chip (P chip) and the horizontal cell network chip (H chip). The output images of the P chip are transferred to the H chip with analog voltages through the line-parallel transfer bus. The sample/hold circuits embedded in the P and H chips compensate for the pattern noise generated on the circuits, including the analog communication pathway. Using the multichip silicon retina together with an off-chip differential amplifier, spatial filtering of the image with an odd- and an even-symmetric orientation selective receptive fields was carried out in real time. The analog data transfer method in the present multichip silicon retina is useful to design analog neuromorphic multichip systems that mimic the hierarchical structure of neuronal networks in the visual system.     

Application of Feed-forward Artificial Neural Networks to the Identification of Defective Analog Integrated Circuits

This paper presents a new approach for detecting defects in analog integrated circuits using a feed-forward neural network trained by the resilient error back-propagation method. A feed-forward neural network has been used for detecting faults in a simple analog CMOS circuit by representing the differences observed in power supply current of fault-free and faulty circuits. The identification of defects was performed in time and frequency domains, followed by a comparison of results achieved in both domains. We show that resilient back-propagation neural networks can be a very efficient and versatile approach for identifying defective analog circuits. Moreover, this approach is not limited to the supply current analysis, because it also offers monitoring of other circuit parameters. The type of defects detected by the resilient backpropagation neural networks, as well as other possible applications of this approach, are discussed.     

Adaptive neural coding dependent on the time-varying statistics of the somatic input current

It is generally assumed that nerve cells optimize their performance to reflect the statistics of their input. Electronic circuit analogs of neurons require similar methods of self-optimization for stable and autonomous operation. We here describe and demonstrate a biologically plausible adaptive algorithm that enables a neuron to adapt the current threshold and the slope (or gain) of its current-frequency relationship to match the man (or dc offset) and variance (or dynamic range or contrast) of the time-varying somatic input current. The adaptation algorithm estimates the somatic current signal from the spike train by way of the intracellular somatic calcium concentration, thereby continuously adjusting the neurons' firing dynamics. This principle is shown to work in an analog VLSI-designed silicon neuron.     

Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions

We establish two conditions that ensure the nondivergence of additive recurrent networks with unsaturating piecewise linear transfer functions, also called linear threshold or semilinear transfer functions. As Hahnloser, Sarpeshkar, Mahowald, Douglas, and Seung (2000) showed, networks of this type can be efficiently built in silicon and exhibit the coexistence of digital selection and analog amplification in a single circuit. To obtain this behavior, the network must be multistable and nondivergent, and our conditions allow determining the regimes where this can be achieved with maximal recurrent amplification. The first condition can be applied to nonsymmetric networks and has a simple interpretation of requiring that the strength of local inhibition match the sum over excitatory weights converging onto a neuron. The second condition is restricted to symmetric networks, but can also take into account the stabilizing effect of nonlocal inhibitory interactions. We demonstrate the application of the conditions on a simple example and the orientation-selectivity model of Ben-Yishai, Lev Bar-Or, and Sompolinsky (1995). We show that the conditions can be used to identify in their model regions of maximal orientation-selective amplification and symmetry breaking.     

Analog Genetic Encoding for the Evolution of Circuits and Networks

This paper describes a new kind of genetic representation called analog genetic encoding (AGE). The representation is aimed at the evolutionary synthesis and reverse engineering of circuits and networks such as analog electronic circuits, neural networks, and genetic regulatory networks. AGE permits the simultaneous evolution of the topology and sizing of the networks. The establishment of the links between the devices that form the network is based on an implicit definition of the interaction between different parts of the genome. This reduces the amount of information that must be carried by the genome, relatively to a direct encoding of the links. The application of AGE is illustrated with examples of analog electronic circuit and neural network synthesis. The performance of the representation and the quality of the results obtained with AGE are compared with those produced by genetic programming.     

Performance analysis for a K-winners-take-all analog neural network: basic theory

In a previous work, the authors proposed an analog Hopfield-type neural network that identified the K largest components of a list of real numbers. In this work, we identify computable restrictions on the parameters, in order that the network can repeatedly process lists, one after the other, at a given rate. A complete mathematical analysis gives analytical bounds for the time required in terms of circuit parameters, the length of the lists, and the relative separation of list elements. This allows practical setting of circuit parameters for required clocking times. The emphasis is on high gain functioning of each neuron. Numerical investigations show the accuracy of the theoretical predictions, and study the influence of various parameters on performance.     

脉冲神经元环路振荡发放与环路选择

在Izhikevich提出的脉冲神经元模型中,引入随机变化的输入电流,使神经元的脉冲发放具有随机性,不同数量的神经元采用连接权值组成网络的脉冲发放。实验结果表明,选择适当的连接权值可以得到环路的持续振荡发放。通过脉冲发放,可以在网络中选择神经环路,完成环路记忆联想过程,并给出研究脉冲神经智能的新思路。