A programmable analog cellular neural network CMOS chip for highspeed image processing

A high speed analog image processor chip is presented. It is based on the cellular neural network architecture. The implementation of an analog programmable CNN-chip in a standard CMOS technology is discussed. The control parameters or templates in all cells are under direct user control and are tunable over a continuous value range from 1/4 to 4. This tuning property is implemented with a compact current scaling circuit based on MOS transistors operating in the linear region. A 4×4 CNN prototype system has been designed in a 2.4 μm CMOS technology and successfully tested. The cell density is 380 cells/cm² and the cell time constant is 10 μs. The current drain for a typical template is 40 μA/cell. The real-time image processing capabilities of the system are demonstrated. From this prototype it is estimated that a 128×128 fully programmable analog image processing system can be integrated on a single chip using a standard digital submicron CMOS technology. This work demonstrates that powerful high speed programmable analog processing systems can be built using standard CMOS technologies     

A four-quadrant CMOS analog multiplier for analog neural networks

A four-quadrant CMOS analog multiplier is presented. The multiplier uses the square-law characteristic of an MOS transistor in saturation. Its major advantage over other four-quadrant multipliers is its combination of small area and low power consumption. In addition, unlike almost all other designs of four-quadrant multipliers, this design has single ended inputs so that the inputs do not need to be pre-processed before being fed to the multiplier, thus saving additional area. These properties make the multiplier very suitable for use in the implementation of artificial neural networks. The design was fabricated through MOSIS using the standard 2 μm CMOS process. Experimental results obtained from it are presented     

Learning capacitive weights in analog CMOS neural networks

Implementations of artificial neural networks as analog VLSI circuits differ in their method of synaptic weight storage (digital weights, analog EEPROMs, or capacitive weights) and in whether learning is performed locally at the synapses or off-chip. In this paper, we explain the principles of analog networks with in situ or local synaptic learning of capacitive weights, with test results of CMOS implementations from our laboratory. Synapses for both simple Hebbian and mean field networks are investigated. Synaptic weights may be refreshed by periodic rehearsal on the training data, which compensates for temperature drift or other nonstationarity. Compact high-performance layouts have been obtained in which learning adjusts for component variability.     

A new approach to sizing analog CMOS building blocks using pre-compiled neural network models

There has been a constant endeavor towards improving the available circuit design automation tools to match technological advancements in the electronic industry. However, inadequate research efforts in the analog domain are holding back the exploitation of advanced technologies. A dearth of design expertise in the analog domain is the principal driving force for the growth of Design Automation (DA) tools. Transistor sizing is one of the most crucial steps in the analog IC design. In this paper, we put forward a new computer aided design framework for the sizing of transistors in MOS Integrated Circuit (IC) amplifiers by incorporating powerful modeling capabilities of Artificial Neural Networks (ANN). ANNs have proven to be efficient and accurate modeling tools in several applications. The proposed tool is capable of directly computing transistor related design parameters, of the MOS IC amplifier and associated peripheral circuitry. The proposed tool thus avoids several time-consuming simulations and/or tuning runs at the very bottom level of analog IC amplifier implementation, using a given CMOS process. It also reduces manual intervention in the design process, thus enhancing the automation of the design process. This paper presents design examples of several analog IC functional modules that are developed and verified successfully.     

An analog continuous-time neural network

An analog continuous-time neural network is described. Building blocks which include the capability for on-chip learning and an example network are described and test results are presented. We are using analog nonvolatile CMOS floating-gate memories for storage of the neural weights. The floating-gate memories are programmed by illuminating the entire chip with ultraviolet light. The subthreshold operation of the CMOS transistor in analog VLSI has a very low power dissipation which can be utilized to build larger computational systems, e.g., neural networks. The experimental results show that the floating-gate memories are promising, and that the building blocks are operating as separate units; however, especially the time constants involved in the computations of the continuous-time analog neural network should be studied further.     

A programmable analog CMOS synapse for neural networks

This paper presents a programmable analog synapse for use in both feedforward and feedback neural networks. The synapse consists of two complementary floating-gate MOSFETs which are programmable in both directions by Fowler-Nordheim tunneling. The P-transistor and the N-transistor are programmable independently with pulses of different amplitude and duration, and hence finer weight adjustment is made possible. An experimental 4×4 synapse array has been designed, which in addition has 32 analog CMOS switches and x–y decoders to select a synapse cell for programming. It has been fabricated using a standard 2-m, double-polysilicon CMOS technology. Simulation results confirm that output current of synapse is proportional to the product of the input voltage and weight and also shows both inhibitory and excitatory current. Current summing effect has been observed at the input of a neuron. This array is designed using modular and regular structured elements, and hence is easily expandable to larger networks.     

A programmable analog neural network chip

A generic chip is implemented in CMOS to facilitate studying networks by building them in analog VLSI. By utilizing the well-known properties of charge storage and charge injection in a novel way, the authors have achieved a high enough level of complexity (>10³ weights and 10 bits of analog depth) to be interesting, in spite of the limitation of a modest 6.00×3.5-mm² die size required by a multiproject fabrication run. If the cell were optimized to represent fixed-weight networks by eliminating weight decay and bidirectional weight changes, the density could easily be increased by a factor of 2 with no loss in resolution. Once a weight change vector has been written to the RAM cells, charge transfers can be clocked at a rate of 2 MHz, corresponding to peak learning rates of 2×10⁹ weight changes/second and exceeding the throughput of `neural network accelerators' by two orders of magnitude     

Physically realistic fault models for analog CMOS neural networks

A general methodology for the development of physically realistic fault models for VLSI neural networks is presented. The derived fault models are explained and characterized in detail. The application of this methodology to an analog CMOS implementation of fixed-weight (i.e., pretrained), binary-valued neural networks is reported. It is demonstrated that these techniques can be used to accurately evaluate defect sensitivities in VLSI neural network circuitry. It is also shown that this information can be used to guide the design of circuitry which fully utilizes a neural network's potential for defect tolerance     

1.5 μm CMOS gate arrays with analog/digital macros designedusing common base arrays

Mixed analog and digital circuits are realized on a 1.5 μm silicon-gate CMOS chip with +5 V power supply only. The circuit uses CMOS digital gate arrays of 0.32 K to 19.6 K cells and is created without any additional turnaround time or any restriction on the design. Typical internal digital gate (two-input NAND) speed, with a fanout of 3 and a wire length of 3 mm, is 1.4 ns. A voltage comparator with ±8 mV maximum input offset voltage and 60 ns response time, digital-to-analog and analog-to-digital converters with 4-, 6-, and 8-bit resolution, respectively, and an analog switch of 25 Ω on-resistance can be realized on the same chip with digital circuits. Using this technology, about one-tenth of the turnaround time can be achieved compared with full-custom LSIs for the same system. The product development flow and computer-aided-design tools for designing mixed analog and digital gate arrays are the same as for digital gate arrays     

Testing of programmable analog neural network chips

Artificial neural network chips can achieve high-speed performance in solving complex computational problems for signal and information processing applications. These chips contain regular circuit units such as synapse matrices that interconnect linear arrays of input and output neurons. The neurons and synapses may be implemented in an analog or digital design style. Although the neural processing has some degree of fault tolerance, a significant percentage of processing defects can result in catastrophic failure of the neural network processors. Systematic testing of these arrays of circuitry is of great importance in order to assure the quality and reliability of VLSI neural network processor chips. The proposed testing method consists of parametric test and behavioral test. Two programmable analog neural chips have been designed and fabricated. The systematic approach used to test the chips is described, and measurement results on parametric test are presented.This research was partially supported by DARPA under Contract MDA 972-90-C-0037 and by National Science Foundation under Grant MIP-8904172.     

Two analog counters for neural network implementation

Simple analog circuits which are useful for the implementation of the synchronous Boltzmann machine learning algorithms are presented. A simple charge-transfer-based analog counter is described. The authors give a functional model of its behavior and analyze the differences between this model and the counter implementation. They also present simulation results and the test of a prototype. Along the same lines, they study a switched-current-based counter, which achieves better results (dynamic range, linearity) through higher complexity     

An analog feed-forward neural network with on-chip learning

An analog continuous-time neural network with on-chip learning is presented. The 4-3-2 feed-forward network with a modified back-propagation learning scheme was build using micropower building blocks in a double poly, double metal 2 CMOS process. The weights are stored in non-volatile UV-light programmable analog floating gate memories. A differential signal representation is used to design simple building blocks which may be utilized to build very large neural networks. Measured results from on-chip learning are shown and an example of generalization is demonstrated. The use of micro-power building blocks allows very large networks to be implemented without significant power consumption.     

A Beta basis function neural network in CMOS subthreshold mode

Beta basis function neural networks (BBFNNs) are powerful systems for learning and universal approximation. In this paper, we present a hardware implementation of the Beta neuron using the CMOS subthreshold mode. We describe the low power–low voltage analogue Beta neuron circuit. Three main modules are used to realize the electronic Beta function: a logarithmic currentto-voltage converter, a multiplier and an exponential voltage-to-current converter. Simulation results show the validity of our neural hardware implementation. The parameters of the electronic Beta function are controlled independently by current sources. This analogue implementation could be used easily to realize analogue BBFNNs.     

An analog VLSI neural network with on-chip perturbation learning

An analog very large scale integration (VLSI) neural network intended for cost-sensitive, battery-powered, high-volume applications is described. Weights are stored in the analog domain using a combination of dynamic and nonvolatile memory that allows both fast learning and reliable long-term storage. The synapse occupies 4.9 K μm² in a 2-μm technology. On-chip controlled perturbation-based gradient descent allows fast learning with very little external support. Other distinguishing features include a reconfigurable topology and a temperature-independent feedforward path. An eight-neuron, 64-synapse proof-of-concept chip reliably solves the exclusive-or problem in ten's of milliseconds and 4-b parity in hundred's of milliseconds     

Design and characterization of analog VLSI neural network modules

A systematic method for testing large arrays of analog, digital, or mixed-signal circuit components that constitute VLSI neural networks is described. This detailed testing procedure consists of a parametric test and a behavioral test. Characteristics of the input neuron, synapse, and output neuron circuits are used to distinguish between faulty and useful chips. Stochastic analysis of the parametric test results can be used to predict chip yield information. Several measurement results from two analog neural network processor designs that are fabricated in 2 μm double-polysilicon CMOS technologies are presented to demonstrate the testing procedure     

Direction finding in phased arrays with a neural network beamformer

Adaptive neural network processing of phased-array antenna received signals promises to decrease antenna manufacturing and maintenance costs while increasing mission uptime and performance between repair actions. We introduce one such neural network which performs aspects of digital beamforming with imperfectly manufactured, degraded, or failed antenna components. This paper presents measured results achieved with an adaptive radial basis function (ARBF) artificial neural network architecture which learned the single source direction finding (DF) function of an eight-element X-band array having multiple, unknown failures and degradations. We compare the single source DF performance of this ARBF neural network, whose internal weights are computed using a modified gradient descent algorithm, with another radial basis function network, Linnet, whose weights are calculated using linear algebra. Both networks are compared to a traditional DF approach using monopulse     

Programmable-weight building blocks for analog VLSI neural network processors

Although the neural network paradigms have the intrinsic potential for parallel operations, a traditional computer cannot fully exploit it because of the serial hardware configuration. By using the analog circuit design approach, a large amount of parallel functional units can be realized in a small silicon area. In addition, appropriate accuracy requirements for neural operation can be satisfied. Components for a general-purpose neural chip have been designed and fabricated. Dynamically adjusted weight value storage provides programmable capability. Possible reconfigurable schemes for a general-purpose neural chip are also presented. Test of the prototype neural chip has been successfully conducted and an expected result has been achieved.     

An analog CMOS autopilot

Results are presented of an analog LSI CMOS missile autopilot. The autopilot is a two-chip set which requires a total area of 210000 sq miles and consumes 700 mW of power. The set is fabricated in a double-poly p-well silicon-gate technology. The chips perform a wide array of analog functions, including precision filtering, full-wave demodulation, digital-to-analog conversion, limiting, pulsewidth modulation, and offset cancellation. A number of digital functions are also provided. The chip set was functionally correct on the first iteration after computer-aided verification. The output noise of the chip set is 6.5 mV, integrated over a bandwidth of 5-500 Hz. Results are presented over a temperature range of -55/spl deg/C to 125/spl deg/C.     

A feed-forward time-multiplexed neural network with mixed-signal neuron-synapse arrays

A new time-multiplexed architecture is proposed for mixed-signal neural networks. MRIII is used for training the network which is more robust for implementing mixed-signal designs. The problem of node addressing and routing for implementing the MRIII is solved by performing the operations in current mode and using a counter. Arrays of mixed-signal multiplying-digital-to-analog (MDAC) blocks are used for synaptic multiplication. A compact architecture with a more linear transfer function is proposed for the MDAC to reduce the area, power consumption and noise. The proposed network is implemented using TSMC CMOS 0.18 μ technology. The results of an XOR (2-2-1) network are presented to show the generality of the design.