A programmable analog neural network processor

An analog neural network breadboard consisting of 256 neurons and 2048 programmable synaptic weights of 5 bits each is constructed and tested. The heart of the processor is an array of custom-programmable synapse (resistor) chips on a reconfigurable neuron board. The analog bandwidth of the system is 90 kHz. The breadboard is used to demonstrate the application of neural network learning to the problem of real-time adaptive mirror control. The processor control is 21 actuators of an adaptive mirror with a step-response setting time of 5 ms. The demonstration verified that it is possible to modify the control law of the high-speed analog loop using neural network training without stopping the control loop.     

A high-speed analog neural processor

Targeted at high-energy physics research applications, our special-purpose analog neural processor can classify up to 70 dimensional vectors within 50 nanoseconds. The decision-making process of the implemented feedforward neural network enables this type of computation to tolerate weight discretization, synapse nonlinearity, noise, and other non-ideal effects. Although our prototype does not take advantage of advanced CMOS technology, and was fabricated using a 2.5-μm CMOS process, it performs 6 billion multiplications per second, with only 2 W dissipation, and has as high as 1.5 Gbyte/s equivalent bandwidth     

An analog VLSI implementation of Hopfield's neural network

The basic requirements for electronic implementations of the fully connected Hopfield network are examined, highlighting the reasons why the authors regard analog implementations as more appropriate. Analog VLSI networks are then discussed, with particular reference to the selection of memory points and the design of the synapse, and experimental results are given. A test chip containing 14 neurons and 196 synapses is described     

An analog VLSI recurrent neural network learning a continuous-timetrajectory

Real-time algorithms for gradient descent supervised learning in recurrent dynamical neural networks fail to support scalable VLSI implementation, due to their complexity which grows sharply with the network dimension. We present an alternative implementation in analog VLSI, which employs a stochastic perturbation algorithm to observe the gradient of the error index directly on the network in random directions of the parameter space, thereby avoiding the tedious task of deriving the gradient from an explicit model of the network dynamics. The network contains six fully recurrent neurons with continuous-time dynamics, providing 42 free parameters which comprise connection strengths and thresholds. The chip implementing the network includes local provisions supporting both the learning and storage of the parameters, integrated in a scalable architecture which can be readily expanded for applications of learning recurrent dynamical networks requiring larger dimensionality. We describe and characterize the functional elements comprising the implemented recurrent network and integrated learning system, and include experimental results obtained from training the network to represent a quadrature-phase oscillator.     

Toward a general-purpose analog VLSI neural network with on-chiplearning

This paper describes elements necessary for a general-purpose low-cost very large scale integration (VLSI) neural network. By choosing a learning algorithm that is tolerant of analog nonidealities, the promise of high-density analog VLSI is realized. A 64-synapse, 8-neuron proof-of-concept chip is described. The synapse, which occupies only 4900 mum(2) in a 2-mum technology, includes a hybrid of nonvolatile and dynamic weight storage that provides fast and accurate learning as well as reliable long-term storage with no refreshing. The architecture is user-configurable in any one-hidden-layer topology. The user-interface is fully microprocessor compatible. Learning is accomplished with minimal external support; the user need only present inputs, targets, and a clock. Learning is fast and reliable. The chip solves four-bit parity in an average of 680 ms and is successful in about 96% of the trials.     

An analog processor architecture for a neural network classifier

Many neural-like algorithms currently under study support classification tasks. Several of these algorithms base their functionality on LVQ-like procedures to find locations of centroids in the data space, and on kernel (or radial-basis) functions centered on these centroids to approximate functions or probability densities. A generic analog chip could implement in a parallel way all basic functions found in these algorithms, permitting construction of a fast, portable classification system     

A VLSI neural processor for image data compression usingself-organization networks

An adaptive electronic neural network processor has been developed for high-speed image compression based on a frequency-sensitive self-organization algorithm. The performance of this self-organization network and that of a conventional algorithm for vector quantization are compared. The proposed method is quite efficient and can achieve near-optimal results. The neural network processor includes a pipelined codebook generator and a paralleled vector quantizer, which obtains a time complexity O(1) for each quantization vector. A mixed-signal design technique with analog circuitry to perform neural computation and digital circuitry to process multiple-bit address information are used. A prototype chip for a 25-D adaptive vector quantizer of 64 code words was designed, fabricated, and tested. It occupies a silicon area of 4.6 mmx6.8 mm in a 2.0 mum scalable CMOS technology and provides a computing capability as high as 3.2 billion connections/s. The experimental results for the chip and the winner-take-all circuit test structure are presented.     

Pulse-stream VLSI neural networks mixing analog and digitaltechniques

The pulse-stream technique, which represents neural states as sequences of pulses, is reviewed. Several general issues are raised, and generic methods appraised, for pulsed encoding, arithmetic, and intercommunication schemes. Two contrasting synapse designs are presented and compared. The first is based on a fully analog computational form in which the only digital component is the signaling mechanism itself-asynchronous, pulse-rate encoded digital voltage pulses. In this circuit, multiplication occurs in the voltage/current domain. The second design uses more conventional digital memory for weight storage, with synapse circuits based on pulse stretching. Integrated circuits implementing up to 15000 analog, fully programmable synaptic connections are described. A demonstrator project is described in which a small robot localization network is implemented using asynchronous, analog, pulse-stream devices.     

Optoelectronic winner-take-all VLSI shunting neural network

In this paper we present an analog winner-take-all MOS VLSI (metal-oxide semiconductor/very large scale integration) optoelectronic network. By varying either the input current or circuit parameters, the circuit can evidence several different behaviors such as contrast enhancement, strict winner-take-all, or winner-take-all with hysteresis. Simulation and experimental results from the prototype circuit are also discussed.     

Another K-winners-take-all analog neural network

An analog Hopfield type neural network is given, that identifies the K largest components of a list d of N real numbers. The neurons are identical, with a tanh characteristic, and the weight matrix is symmetric and fully filled. The list to be processed is a summand of the input currents of the neurons, and the network is started from zero. We provide easily computable restrictions on the parameters. The main emphasis here is on the magnitude of the neuronal gain. A complete mathematical analysis is given. The trajectories are shown to eventually have positive components precisely in the positions given by the K largest elements in the input list.     

A programmable VLSI retina for rough vision

A VLSI retina is a device that intimately associates an optoelectronic layer with processing facilities on a monolithic circuit. Combining acquisition and processing provides a better balance between between data flows and bandwidths. It is also expected to reveal fruitful shortcuts between microelectronic phenomena and vision-oriented information processing. Yet, except for simplistic environments and applications, analog hardware will not suffice to process and compact the raw image flow from photosensitive arrays. To solve this output problem, an on-chip array of bare boolean processors can be used to provide versatility from programmability. Since the monolithic constraint implies a memory shortage, the abilities of such a retina will be limited to a rough type of vision, but specific algorithmic techniques can cope with it. We have used shift registers with some tricky circuitry to build a minimal retina boolean processor with less than 30 transistors. The successful integration and testing of and experimentation with such a 65×76 retina are presented.     

VLSI implementation of a neural network model

The authors describe a complementary metal-oxide-semiconductor (CMOS) very-large-scale integrated (VLSI) circuit implementing a connectionist neural-network model. It consists of an array of 54 simple processors fully interconnected with a programmable connection matrix. This experimental design tests the behavior of a large network of processors integrated on a chip. The circuit can be operated in several different configurations by programming the interconnections between the processors. Tests made with the circuit working as an associative memory and as a pattern classifier were so encouraging that the chip has been interfaced to a minicomputer and is being used as a coprocessor in pattern-recognition experiments. This mode of operation is making it possible to test the chip's behavior in a real application and study how pattern-recognition algorithms can be mapped in such a network     

Learning in analog neural network hardware

Hardware implementations of neuroprocessor architectures are currently enjoying commercial availability for the first time ever. This development has been caused in part by the requirement for real-time solutions to time critical neural network applications. Massively parallel asynchronous neuromorphic representations are inherently capable of very high computational speeds when properly cast in the “right stuff”, i.e. electronic or optoelectronic hardware. However, hardware based learning in such systems is still at a primitive stage. In practise, simulations are typically performed in software, and the resulting synaptic weight capturing the input-output transformation subsequently quantized and down-loaded onto the neural hardware. However, because of the numerous discrepancies between the software and hardware, such systems are inherently poor in performance. In this paper we report on chip-in-the-loop learning systems assembled from custom analog “building blocks” hardware.     

Digitally programmable analog building blocks for theimplementation of artificial neural networks

This paper describes the design, experimental characterization and behavior modeling of a homogeneous set of building blocks necessary to construct in analog hardware feed-forward artificial neural networks. A novel synapse architecture is proposed using a quasi-passive D/A (digital-to-analog) converter followed by a four-quadrant analog-digital multiplier, its main advantages are 1) increased signal input range; 2) improved area/weight resolution ratio; 3) on-chip refreshing of the weight value; and 4) serial loading the weight bits. The neurons are built using MOS (metal-oxide semiconductor) transistors operating in the saturation region and exploiting the inherent quadratic characteristics. Experimental results obtained from a demonstration prototype chip realized in a 1.2 mum double-poly, double-metal CMOS (complimentary MOS) technology show good agreement with the design specifications. A simple application of the proposed building blocks is illustrated based on the mixed-signal simulation of the corresponding behavior models constructed from the experimental characterization data.     

Wavelet based fault detection in analog VLSI circuits using neural networks

This paper deals with a new method of testing analog VLSI circuits, using wavelet transform for analog circuit response analysis and artificial neural networks (ANN) for fault detection. Pseudo-random patterns generated by Linear Feedback Shift Register (LFSR) are used as input test patterns. The wavelet coefficients obtained for the fault-free and faulty cases of the circuits under test (CUT) are used to train the neural network. Two different architectures, back propagation and probabilistic neural networks are trained with the test data. To minimize the neural network architecture, normalization and principal component analysis are done on the input data before it is applied to the neural network. The proposed method is validated with two IEEE benchmark circuits, namely, the operational amplifier and state variable filter.     

脉频调制神经网络VLSI的设计及应用

本文提出了一种用于故障诊断识别的改进脉冲频率调制(PFM)VLSI神经网络电路,改进了传统的基于软件的机械故障诊断模式,发挥了神经网络超大规模集成电路(VLSI)的优势.利用单层感知器网络、场效应管电路实现了一种新的数字模拟混合突触乘法/加法器电路,而且该神经网络电路的突触权值不需要学习调整,降低了电路的复杂性.以此电路为基础,设计了进行主轴承噪声故障诊断的神经网络故障识别系统.将含有故障信息的原始噪声信号,经过前置信号处理分析、故障特征值提取和神经网络运算,得出VLSI电路输出端电容的电压——代表待识别信号与模板故障信号的“欧氏距离”,进而判断出故障的类别.经过仿真测试,基于硬件的诊断系统的识别性能接近于基于软件的系统.     

A Neural Network Appraoch to Fault Diagnosis in Analog Circuits

Thia paper presents a neural network based fault diagnosis approach for analog circuits,taking the tolerances of circuit elements into account.Specifically,a normalization rule of input information,a pseudo-fault domain border(PFDB)pattern selection method and a new output error function are proposed for training the backpropagation(BP) network to be a fault diagnoser.Experimental results demonstrate that the diagnoser performs as well as or better than any classical approaches in terms of accuracy,and provides at least an order-of-magnitude improvement in post-fault diagnostic speed.     

A 100-GOPS programmable processor for vehicle vision systems

New generations of automobiles will include driver assistance systems requiring powerful, low-cost processors to handle video/camera applications and to enable fast, convenient application development. Shrinking feature sizes on processors already in development will bring substantial increases in system speed and functionality.     

A monolithic Hough transform processor based on restructurable VLSI

The implementation of a Hough transform processor using a wafer-scale-integration technology, restructurable VLSI circuit is described. The Hough transform is typically used as a grouping operation in an image processing sequence. The transform discussed here groups pixels in order to extract linear features. This calculation is realized with a wafer-scale processor that allows a complete line extraction system to be integrated on a single PC board. Also discussed is the use of the CAD tools that allowed this processor to be realized without incurring silicon layout and processing overhead     

Integrating heuristic knowledge and optimization models forcommunication network design

A method is developed for integrating heuristic design knowledge with optimization models to create a tool for the topological design of computer communication networks. Design choices are based on suggestions from optimization models, as well as heuristic knowledge, which interact through a blackboard. A truth maintenance system (TMS) records justification for current design choices, as well as promising alternatives. A dependency-directed backtracking mechanism works with the TMS to choose other alternatives as warranted. This hybrid tool can consider a wider range of design requirements than is possible using one type of knowledge alone, is flexible in handling variations in these requirements, and has a modular structure which facilitates incremental refinement. Computational results on separate networks show it is effective in identifying good low-cost solutions