Neuron-synapse IC chip-set for large-scale chaotic neural networks.

We propose a neuron-synapse integrated circuit (IC) chip-set for large-scale chaotic neural networks. We use switched-capacitor (SC) circuit techniques to implement a three-internal-state transiently-chaotic neural network model. The SC chaotic neuron chip faithfully reproduces complex chaotic dynamics in real numbers through continuous state variables of the analog circuitry. We can digitally control most of the model parameters by means of programmable capacitive arrays embedded in the SC chaotic neuron chip. Since the output of the neuron is transfered into a digital pulse according to the all-or-nothing property of an axon, we design a synapse chip with digital circuits. We propose a memory-based synapse circuit architecture to achieve a rapid calculation of a vast number of weighted summations. Both of the SC neuron and the digital synapse circuits have been fabricated as IC forms. We have tested these IC chips extensively, and confirmed the functions and performance of the chip-set. The proposed neuron-synapse IC chip-set makes it possible to construct a scalable and reconfigurable large-scale chaotic neural network with 10000 neurons and 10000/sup 2/ synaptic connections.     

A two-variable silicon neuron circuit based on the Izhikevich model

The silicon neuron is an analog electronic circuit that reproduces the dynamics of a neuron. It is a useful element for artificial neural networks that work in real time. Silicon neuron circuits have to be simple, and at the same time they must be able to realize rich neuronal dynamics in order to reproduce the various activities of neural networks with compact, low-power consumption, and an easy-to-configure circuit. We have been developing a silicon neuron circuit based on the Izhikevich model, which has rich dynamics in spite of its simplicity. In our previous work, we proposed a simple silicon neuron circuit with low power consumption by reconstructing the mathematical structure in the Izhikevich model using an analog electronic circuit. In this article, we propose an improved circuit in which all of the MOSFETs are operated in the sub-threshold region.     

Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses

A mixed-signal very large scale integration (VLSI) chip for large scale emulation of spiking neural networks is presented. The chip contains 2400 silicon neurons with fully programmable and reconfigurable synaptic connectivity. Each neuron implements a discrete-time model of a single-compartment cell. The model allows for analog membrane dynamics and an arbitrary number of synaptic connections, each with tunable conductance and reversal potential. The array of silicon neurons functions as an address-event (AE) transceiver, with incoming and outgoing spikes communicated over an asynchronous event-driven digital bus. Address encoding and conflict resolution of spiking events are implemented via a randomized arbitration scheme that ensures balanced servicing of event requests across the array. Routing of events is implemented externally using dynamically programmable random-access memory that stores a postsynaptic address, the conductance, and the reversal potential of each synaptic connection. Here, we describe the silicon neuron circuits, present experimental data characterizing the 3 mm times 3 mm chip fabricated in 0.5-mum complementary metal-oxide-semiconductor (CMOS) technology, and demonstrate its utility by configuring the hardware to emulate a model of attractor dynamics and waves of neural activity during sleep in rat hippocampus     

Computing motion using analog VLSI vision chips: An experimental comparison among different approaches

We have designed, built and tested a number of analog CMOS VLSI circuits for computing 1-D motion from the time-varying intensity values provided by an array of on-chip phototransistors. We present experimental data for two such circuits and discuss their relative performance. One circuit approximates the correlation model while a second chip uses resistive grids to compute zero-crossings to be tracked over time by a separate digital processor. Both circuits integrate image acquisition with image processing functions and compute velocity in real time. For comparison, we also describe the performance of a simple motion algorithm using off-the-shelf digital components. We conclude that analog circuits implementing various correlation-like motion algorithms are more robust than our previous analog circuits implementing gradient-like motion algorithms.     

Wavelet based fault detection in analog VLSI circuits using neural networks

This paper deals with a new method of testing analog VLSI circuits, using wavelet transform for analog circuit response analysis and artificial neural networks (ANN) for fault detection. Pseudo-random patterns generated by Linear Feedback Shift Register (LFSR) are used as input test patterns. The wavelet coefficients obtained for the fault-free and faulty cases of the circuits under test (CUT) are used to train the neural network. Two different architectures, back propagation and probabilistic neural networks are trained with the test data. To minimize the neural network architecture, normalization and principal component analysis are done on the input data before it is applied to the neural network. The proposed method is validated with two IEEE benchmark circuits, namely, the operational amplifier and state variable filter.     

On the computational power of winner-take-all

This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (also referred to as McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-take-all in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our theoretical analysis also provides answers to two basic questions raised by neurophysiologists in view of the well-known asymmetry between excitatory and inhibitory connections in cortical circuits: how much computational power of neural networks is lost if only positive weights are employed in weighted sums and how much adaptive capability is lost if only the positive weights are subject to plasticity.     

Real-time computing without stable states: a new framework for neural computation based on perturbations

A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.     

A systematic method for configuring VLSI networks of spiking neurons

An increasing number of research groups are developing custom hybrid analog/digital very large scale integration (VLSI) chips and systems that implement hundreds to thousands of spiking neurons with biophysically realistic dynamics, with the intention of emulating brainlike real-world behavior in hardware and robotic systems rather than simply simulating their performance on general-purpose digital computers. Although the electronic engineering aspects of these emulation systems is proceeding well, progress toward the actual emulation of brainlike tasks is restricted by the lack of suitable high-level configuration methods of the kind that have already been developed over many decades for simulations on general-purpose computers. The key difficulty is that the dynamics of the CMOS electronic analogs are determined by transistor biases that do not map simply to the parameter types and values used in typical abstract mathematical models of neurons and their networks. Here we provide a general method for resolving this difficulty. We describe a parameter mapping technique that permits an automatic configuration of VLSI neural networks so that their electronic emulation conforms to a higher-level neuronal simulation. We show that the neurons configured by our method exhibit spike timing statistics and temporal dynamics that are the same as those observed in the software simulated neurons and, in particular, that the key parameters of recurrent VLSI neural networks (e.g., implementing soft winner-take-all) can be precisely tuned. The proposed method permits a seamless integration between software simulations with hardware emulations and intertranslatability between the parameters of abstract neuronal models and their emulation counterparts. Most important, our method offers a route toward a high-level task configuration language for neuromorphic VLSI systems.     

A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity

We present a mixed-mode analog/digital VLSI device comprising an array of leaky integrate-and-fire (I&F) neurons, adaptive synapses with spike-timing dependent plasticity, and an asynchronous event based communication infrastructure that allows the user to (re)configure networks of spiking neurons with arbitrary topologies. The asynchronous communication protocol used by the silicon neurons to transmit spikes (events) off-chip and the silicon synapses to receive spikes from the outside is based on the "address-event representation" (AER). We describe the analog circuits designed to implement the silicon neurons and synapses and present experimental data showing the neuron's response properties and the synapses characteristics, in response to AER input spike trains. Our results indicate that these circuits can be used in massively parallel VLSI networks of I&F neurons to simulate real-time complex spike-based learning algorithms.     

A subthreshold MOS neuron circuit based on the Volterra system.

We present an analog neuron circuit consisting of a small number of metal-oxide semiconductor (MOS) devices operating in their subthreshold region. The dynamics of the circuit were designed to be equivalent to the well-known Volterra system to facilitate developing the circuit for a particular application. We show that a simple nonlinear transformation of system variables in the Volterra system enables designing a neuron-like oscillator, which can produce sequences in time of identically shaped pulses (spikes) by using current-mode subthreshold MOS circuits. We present experimental results of the fabricated neuron circuits as well as an application in an inhibitory neural network, where the neurons compete with each other in the frequency and time domains.     

Modeling selective attention using a neuromorphic analog VLSI device

Attentional mechanisms are required to overcome the problem of flooding a limited processing capacity system with information. They are present in biological sensory systems and can be a useful engineering tool for artificial visual systems. In this article we present a hardware model of a selective attention mechanism implemented on a very large-scale integration (VLSI) chip, using analog neuromorphic circuits. The chip exploits a spike-based representation to receive, process, and transmit signals. It can be used as a transceiver module for building multichip neuromorphic vision systems. We describe the circuits that carry out the main processing stages of the selective attention mechanism and provide experimental data for each circuit. We demonstrate the expected behavior of the model at the system level by stimulating the chip with both artificially generated control signals and signals obtained from a saliency map, computed from an image containing several salient features.     

Analog implementation of a Kohonen map with on-chip learning.

Kohonen maps are self-organizing neural networks that classify and quantify n-dimensional data into a one- or two-dimensional array of neurons. Most applications of Kohonen maps use simulations on conventional computers, eventually coupled to hardware accelerators or dedicated neural computers. The small number of different operations involved in the combined learning and classification process, however, makes the Kohonen model particularly suited to a dedicated VLSI implementation, taking full advantage of the parallelism and speed that can be obtained on the chip. A fully analog implementation of a one-dimensional Kohonen map, with on-chip learning and refreshment of on-chip analog synaptic weights, is proposed. The small number of transistors in each cell allows a high degree of parallelism in the operations, which greatly improves the computation speed compared to other implementations. The storage of analog synaptic weights, based on the principle of current copiers, is emphasized. It is shown that this technique can be used successfully for the realization of VLSI Kohonen maps.     

Application of Feed-forward Artificial Neural Networks to the Identification of Defective Analog Integrated Circuits

This paper presents a new approach for detecting defects in analog integrated circuits using a feed-forward neural network trained by the resilient error back-propagation method. A feed-forward neural network has been used for detecting faults in a simple analog CMOS circuit by representing the differences observed in power supply current of fault-free and faulty circuits. The identification of defects was performed in time and frequency domains, followed by a comparison of results achieved in both domains. We show that resilient back-propagation neural networks can be a very efficient and versatile approach for identifying defective analog circuits. Moreover, this approach is not limited to the supply current analysis, because it also offers monitoring of other circuit parameters. The type of defects detected by the resilient backpropagation neural networks, as well as other possible applications of this approach, are discussed.     

Time-domain analysis methodology for large-scale RLC circuits and its applications

With soaring work frequency and decreasing feature sizes, VLSI circuits with RLC parasitic components are more like analog circuits and should be carefully analyzed in physical design. However, the number of extracted RLC components is typically too large to be analyzed efficiently by using present analog circuit simulators like SPICE. In order to speedup the simulations without error penalty, this paper proposes a novel methodology to compress the time-descritized circuits resulted from numerical integration approximation at every time step. The main contribution of the methodology is the efficient structure-level compression of DC circuits containing many current sources, which is an important complement to present circuit analysis theory. The methodology consists of the following parts: 1) An approach is proposed to delete all intermediate nodes of RL branches. 2) An efficient approach is proposed to compress and back-solve parallel and serial branches so that it is error-free and of linear complexity to analyze circuits of tree topology. 3) The Y toπtransformation method is used to error-free reduce and back-solve the intermediate nodes of ladder circuits with the linear complexity. Thus, the whole simulation method is very accurate and of linear complexity to analyze circuits of chain topology. Based on the methodology, we propose several novel algorithms for efficiently solving RLC-model transient power/ground (P/G) networks. Among them, EQU-ADI algorithm of linear-complexity is proposed to solve RLC P/G networks with mesh-tree or mesh-chain topologies. Experimental results show that the proposed method is at least two orders of magnitude faster than SPICE while it can scale linearly in both time- and memory-complexity to solve very large P/G networks.     

An analog VLSI recurrent neural network learning a continuous-timetrajectory

Real-time algorithms for gradient descent supervised learning in recurrent dynamical neural networks fail to support scalable VLSI implementation, due to their complexity which grows sharply with the network dimension. We present an alternative implementation in analog VLSI, which employs a stochastic perturbation algorithm to observe the gradient of the error index directly on the network in random directions of the parameter space, thereby avoiding the tedious task of deriving the gradient from an explicit model of the network dynamics. The network contains six fully recurrent neurons with continuous-time dynamics, providing 42 free parameters which comprise connection strengths and thresholds. The chip implementing the network includes local provisions supporting both the learning and storage of the parameters, integrated in a scalable architecture which can be readily expanded for applications of learning recurrent dynamical networks requiring larger dimensionality. We describe and characterize the functional elements comprising the implemented recurrent network and integrated learning system, and include experimental results obtained from training the network to represent a quadrature-phase oscillator.     

PWM型VLSI神经网络在故障诊断中的应用

An improved pulse width modulation (PWM) neural network VLSI circuit for fault diagnosis is presented, which differs from the software-based fault diagnosis approach and exploits the merits of neural network VLSI circuit. A simple synapse multiplier is introduced, which has high precision, large linear range and less switching noise effects. A voltage-mode sigmoid circuit with adjustable gain is introduced for realization of different neuron activation functions. A voltage-pulse conversion circuit required for PWM is also introduced, which has high conversion precision and linearity. These 3 circuits are used to design a PWM VLSI neural network circuit to solve noise fault diagnosis for a main bearing. It can classify the fault samples directly. After signal processing, feature extraction and neural network computation for the analog noise signals including fault information, each output capacitor voltage value of VLSI circuit can be obtained, which represents Euclid distance between the corresponding fault signal template and the diagnosing signal, The real-time online recognition of noise fault signal can also be realized.     

A new wide range Euclidean distance circuit for neural network hardware implementations

In this paper, we describe an analog very large-scale integration (VLSI) implementation of a wide range Euclidean distance computation circuit - the key element of many synapse circuits. This circuit is essentially a wide-range absolute value circuit that is designed to be as small as possible (80 /spl times/ 76 /spl mu/m) in order to achieve maximum synapse density while maintaining a wide range of operation (0.5 to 4.5 V) and low power consumption (less than 200 /spl mu/W). The circuit has been fabricated in 1.5-/spl mu/m technology through MOSIS. We present simulated and experimental results of the circuit, and compare these results. Ultimately, this circuit is intended for use as part of a high-density hardware implementation of a self-organizing map (SOM). We describe how this circuit can be used as part of the SOM and how the SOM is going to be used as part of a larger bio-inspired vision system based on the octopus visual system.     

Analog-digital simulations of full conductance-based networks of spiking neurons with spike timing dependent plasticity

We introduce and test a system for simulating networks of conductance-based neuron models using analog circuits. At the single-cell level, we use custom-designed analog circuits (ASICs) that simulate two types of spiking neurons based on Hodgkin-Huxley like dynamics: "regular spiking" excitatory neurons with spike-frequency adaptation, and "fast spiking" inhibitory neurons. Synaptic interactions are mediated by conductance-based synaptic currents described by kinetic models. Connectivity and plasticity rules are implemented digitally through a real time interface between a computer and a PCI board containing the ASICs. We show a prototype system of a few neurons interconnected with synapses undergoing spike-timing dependent plasticity (STDP), and compare this system with numerical simulations. We use this system to evaluate the effect of parameter dispersion on the behavior of small circuits of neurons. It is shown that, although the exact spike timings are not precisely emulated by the ASIC neurons, the behavior of small networks with STDP matches that of numerical simulations. Thus, this mixed analog-digital architecture provides a valuable tool for real-time simulations of networks of neurons with STDP. They should be useful for any real-time application, such as hybrid systems interfacing network models with biological neurons.     

模拟神经网络VLSI脉冲流技术在故障诊断中的应用分析

介绍了模拟神经网络VLSI脉冲流技术实现神经网络模式识别硬件电路的方法，并且直接将故障分类。提出利用包含有故障信息的原始模拟噪声信号，经过前置信号处理和神经网络运算，得出VLSI电路输出端电容的电压值－代表待识别信号与模板故障信号的“欧氏距离”，以实现噪声故障信号的实时硬件在线识别。     

基于仿真数据的神经网络故障诊断方法

针对某些模拟电路的历史故障信息,专家知识及其诊断经验难以获取的状况,提出了一种基于仿真数据的神经网络故障诊断方法.通过使用PSpice电路仿真软件模拟实际电路,生成训练样本训练神经网络,从而建立了电路的输出响应与电路中元件实际值之间的映射.以电路的输出响应和技术指标为判断依据,诊断电路的当前状态,定位故障元件及其偏差.最后以带通滤波器电路为例,对整个过程进行了仿真试验,验证了方法的可行性.