Artificial neural networks: a review of commercial hardware

Artificial neural networks (ANN) became a common solution for a wide variety of problems in many fields, such as control and pattern recognition to name but a few. Many solutions found in these and other ANN fields have reached a hardware implementation phase, either commercial or with prototypes. The most frequent solution for the implementation of ANN consists of training and implementing the ANN within a computer. Nevertheless this solution might be unsuitable because of its cost or its limited speed. The implementation might be too expensive because of the computer and too slow when implemented in software. In both cases dedicated hardware can be an interesting solution.

The necessity of dedicated hardware might not imply building the hardware since in the last two decades several commercial hardware solutions that can be used in the implementation have reached the market.

Unfortunately not every integrated circuit will fit the needs: some will use lower precision, some will implement only certain types of networks, some don’t have training built in and the information is not easy to find.

This article is confined to reporting the commercial chips that have been developed specifically for ANN, leaving out other solutions.

This option has been made because most of the other solutions are based on cards which are built either with these chips, Digital Signal Processors or Reduced Instruction Set Computers.     

Reconfigurable hardware for neural networks: binary versus stochastic

This paper is focused on hardware implementation of neural networks. We propose a reconfigurable, low-cost and readily available hardware architecture for an artificial neuron. For this purpose, we use field-programmable gate arrays i.e. FPGAs. As the state-of-the-art FPGAs still lack the gate density necessary to the implementation of large neural networks of thousands of neurons, we use a stochastic process to implement efficiently the computation performed by a neuron. This paper describes and compares the characteristics of two architectures designed to implement feed-forward fully connected artificial neural networks: the first FPGA prototype is based on traditional adders and multipliers of binary inputs while the second takes advantage of stochastic representation of the inputs. The paper compares both prototypes using the time × area classic factor.     

The LEGO strategy: Guidelines for a profitable deployment

ContextWhen dealing with improvements, organizations seek to find a break-even point as early as possible in order to maximize ROI. In some cases such a strategy can lead to long-term failures by not realizing full benefits, when focusing only on the short-term. LEGO (Living Engineering Process) allows building customized process meta-models based on multiple inputs, making an organization more efficient and effective by optimizing resources, time and costs. This paper introduces elements for designing a strategy for more efficient deployments of process improvement initiatives, optimizing the choice of models and elements to be considered as input to the LEGO approach.     

A hardware implementation of a biological neural system for targetlocalization

The optical axes in an array of photoreceptors in the eyes of mantis shrimps have a particular skewing pattern that provides the animal with a monocular distance evaluation. A hardware (HW) device for target recognition was built based on the mathematical model of the biological visual system. The pattern of inputs was simulated by an array of glass fibers connected to phototransistors. In the biological system inputs are picked up by an integrating nerve fiber, in the HW model by an RC network with compartmental threshold devices. The network state is read by a host PC through an array of threshold comparators. The output consists of pulse patterns that can be generated either by a simulation program or in the HW itself. The system can be used as a selective distance/motion detector and can be employed in several applications involving target detection or obstacle avoidance     

Performance evaluation system for probabilistic neural network hardware

The probabilistic neural network (PNN) is one of the most promising neural networks, and is now applied to some real-world applications. In order to speed up the PNN calculation considerably, we have developed a PNN hardware system for video image recognition. The performance of the PNN hardware cannot be evaluated precisely until the evaluation system is completed. In this study, we developed a performance evaluation system for the PNN hardware and demonstrated it using the developed evaluation system.This work was presented, in part, at the 8th International Symposium on Artificial Life and Robotics, Oita, Japan, January 24–26, 2003     

Automatic generation of LEGO building instructions from multiple photographic images of real objects

We introduce a system to reconstruct large scale LEGO models from multiple two dimensional images of objects taken from different views. We employ a unit voxel with an edge length ratio of 5:5:6 for the shape from silhouette method that reconstructs an octree voxel-based three dimensional model with color information from images. We then convert the resulting voxel model with color information into a LEGO sculpture. In order to minimize the number of LEGO bricks, we use a stochastic global optimization method, simulated annealing, to hollow the model as much as possible but keep its strength for portability. Several real complex LEGO models are provided to demonstrate the effectiveness of the proposed method.     

Infrared target-flare discrimination using a ZISC hardware neural network

The last generation of infrared imaging aircraft seekers and trackers uses pattern recognition algorithms to find and keep a lock on an aircraft in the presence of decoy flares. These algorithms identify targets, based on the features of the various objects in the missile’s field of view. Because modern both aircrafts and missiles fly faster than sound, speed of operation of the target identifier is critical. In this article, we propose a target recognition system that respects this time constraint. It is based on an artificial neural network implemented in hardware, as a set of parallel processors on a commercially available silicon chip called a ZISC, for zero instruction set computer. This chip would be integrated in the infrared missile seeker and tracker. We describe the characteristics of the images that the image processing module of this seeker and tracker extracts from the infrared video frames and show how to construct from these translation and rotation invariant features that can be used as input to the neural network. We determine the individual discriminating power of these features by constructing their histograms, which allows us to eliminate some as not being useful for our purpose. Finally, by testing our system on real data, we show that it has a 90% success rate in aircraft-flare identification, and a processing time that during this time, the aircrafts and missiles will have traveled only a few millimeters. Most of the images on which the neural network makes its mistakes are seen to be hard to recognize even by a human expert.     

Layered tile architecture for efficient hardware spiking neural networks

Spiking Neural Network (SNN) is the most recent computational model that can emulate the behaviour of biological neuron system. However, its main drawback is that it is computationally intensive, which limits the system scalability. This paper highlights and discusses the importance and significance of emulating SNNs in hardware devices. A layer-level tile architecture (LTA) is proposed for hardware-based SNNs. The LTA employs a two-level sharing mechanism of computing components at the synapse and neuron levels, and achieves a trade-off between computational complexity and hardware resource costs. The LTA is implemented on a Xilinx FPGA device. Experimental results demonstrate that this approach is capable of scaling to large hardware-based SNNs.     

Inner engraving for the creation of a balanced LEGO sculpture

LEGO is a globally popular toy composed of colorful interlocking plastic bricks that can be assembled in many ways; however, this special feature makes designing a LEGO sculpture particularly challenging. Building a stable sculpture is not easy for a beginner; even an experienced user requires a good deal of time to build one. This paper provides a novel approach to creating a balanced LEGO sculpture for a 3D model in any pose, using centroid adjustment and inner engraving. First, the input 3D model is transformed into a voxel data structure. Next, the model’s centroid is adjusted to an appropriate position using inner engraving to ensure that the model stands stably. A model can stand stably without any struts when the center of mass is moved to the ideal position. Third, voxels are merged into layer-by-layer brick layout assembly instructions. Finally, users will be able to build a LEGO sculpture by following these instructions. The proposed method is demonstrated with a number of LEGO sculptures and the results of the physical experiments are presented.     

Development of a pulse control-type MEMS microrobot with a hardware neural network

This article presents the micro-electro-mechanical systems (MEMS) microrobot which demonstrates locomotion controlled by hardware neural networks (HNN). The size of the microrobot fabricated by the MEMS technology is 4 × 4 × 3.5 mm. The frame of the robot is made of silicon wafer, and it is equipped with a rotary-type actuator, a link mechanism, and six legs. The rotary-type actuator generates rotational movement by applying an electrical current to artificial muscle wires. The locomotion of the microrobot is obtained by the rotation of the rotary-type actuator. As in a living organism, the HNN realized robot control without using any software programs, A/D converters, or additional driving circuits. A central pattern generator (CPG) model was implemented as an HNN system to emulate the locomotion pattern. The MEMS microrobot emulated the locomotion method and the neural networks of an insect with the rotary-type actuator, the link mechanism, and the HNN. The microrobot performed forward and backward locomotion, and also changed direction by inputting an external trigger pulse. The locomotion speed was 0.325 mm/s and the step width was 1.3 mm.     

一种神经网络硬件实现的可重构设计

以BP网络为例,提出了一种可重构神经网络硬件实现方法。通过可重构体系结构、可重构部件的设计,可以灵活地实现不同规模、传递函数及学习方法的神经网络,从而搭建起神经网络快速硬件实现的平台。经过对一个模式识别问题的实现和测试,证明了这种设计方法的可行性。     

LEGO MindStorms: not just for K-12 anymore

Using LEGO Mindstorms robots to support the ACM Computing Curriculum 2001 in lab exercises and projects from the beginner courses in programming to advanced courses in operating systems, compilers, networks, and artificial intelligence.     

Fixed latency on-chip interconnect for hardware spiking neural network architectures

Information in a Spiking Neural Network (SNN) is encoded as the relative timing between spikes. Distortion in spike timings can impact the accuracy of SNN operation by modifying the precise firing time of neurons within the SNN. Maintaining the integrity of spike timings is crucial for reliable operation of SNN applications. A packet switched Network on Chip (NoC) infrastructure offers scalable connectivity for spike communication in hardware SNN architectures. However, shared resources in NoC architectures can result in unwanted variation in spike packet transfer latency. This packet latency jitter distorts the timing information conveyed on the synaptic connections in the SNN, resulting in unreliable application behaviour.     

Low-resource hardware implementation of the hyperbolic tangent for artificial neural networks

Artificial neural networks are a widespread tool with application in a variety of areas ranging from the social sciences to engineering. Many of these applications have reached a hardware implementation phase and have been documented in scientific papers. Unfortunately, most of the implementations have a simplified hyperbolic tangent replacement which has been the most common problem, as well as the most resource-consuming block in terms of hardware. This paper proposes a low-resource hardware implementation of the hyperbolic tangent, by using the simplest solution in order to obtain the lowest error possible thus far with a set of 25 polynomials of third order, obtained with Chebyshev interpolations. The results obtained show that the solution proposed holds a low error while simultaneously promising the use of low resources, as only third-order polynomials are used.     

Characterization of analog local cluster neural network hardware for control.

The local cluster neural network (LCNN) was designed for analog realization especially suited to applications in control systems. It uses clusters of sigmoidal neurons to generate basis functions that are localized in multidimensional input space. Sigmoidal neurons are well suited to analog electronic realization. In this paper, we report the results of extensive measurements that characterize the computational capabilities of the first analog very large scale integration (VLSI) realization of the LCNN. Despite manufacturing fluctuations and the inherent low precision of analog electronics, the test results suggest that it may be suitable for use in feedback control systems.     

The task of building a trusted computing environment on the Intel hardware platform

The problem of building a trusted computing environment on the basis of modern computing devices has been considered. The main features of software execution at different levels of privilege have been indicated. The architecture of the Intel ME subsystem has been considered. The potential threats of Intel ME technology have been highlighted, and possible ways to eliminate them have been given.     

Darwin: A neuromorphic hardware co-processor based on spiking neural networks

Spiking Neural Network (SNN) is a type of biologically-inspired neural networks that perform information processing based on discrete-time spikes, different from traditional Artificial Neural Network (ANN). Hardware implementation of SNNs is necessary for achieving high-performance and low-power. We present the Darwin Neural Processing Unit (NPU), a highly-configurable neuromorphic hardware co-processor based on SNN implemented with digital logic, supporting a configurable number of neurons, synapses and synaptic delays. The Darwin NPU was fabricated by standard 180 nm CMOS technology with area size of 5 × 5 mm² and 70 MHz clock frequency at the worst case. It consumes 0.84 mW/MHz with 1.8 V power supply for typical applications. Two prototype applications are used to demonstrate the performance and efficiency of the Darwin NPU.     

An unsupervised neural network for low-level control of a wheeledmobile robot: noise resistance, stability, and hardware implementation

We have recently introduced a neural network mobile robot controller (NETMORC). This controller, based on previously developed neural network models of biological sensory-motor control, autonomously learns the forward and inverse odometry of a differential drive robot through an unsupervised learning-by-doing cycle. After an initial learning phase, the controller can move the robot to an arbitrary stationary or moving target while compensating for noise and other forms of disturbance, such as wheel slippage or changes in the robot's plant. In addition, the forward odometric map allows the robot to reach targets in the absence of sensory feedback. The controller is also able to adapt in response to long-term changes in the robot's plant, such as a change in the radius of the wheels. In this article we review the NETMORC architecture and describe its simplified algorithmic implementation, we present new, quantitative results on NETMORC's performance and adaptability under noise-free and noisy conditions, we compare NETMORC's performance on a trajectory-following task with the performance of an alternative controller, and we describe preliminary results on the hardware implementation of NETMORC with the mobile robot ROBUTER.     

A new wide range Euclidean distance circuit for neural network hardware implementations

In this paper, we describe an analog very large-scale integration (VLSI) implementation of a wide range Euclidean distance computation circuit - the key element of many synapse circuits. This circuit is essentially a wide-range absolute value circuit that is designed to be as small as possible (80 /spl times/ 76 /spl mu/m) in order to achieve maximum synapse density while maintaining a wide range of operation (0.5 to 4.5 V) and low power consumption (less than 200 /spl mu/W). The circuit has been fabricated in 1.5-/spl mu/m technology through MOSIS. We present simulated and experimental results of the circuit, and compare these results. Ultimately, this circuit is intended for use as part of a high-density hardware implementation of a self-organizing map (SOM). We describe how this circuit can be used as part of the SOM and how the SOM is going to be used as part of a larger bio-inspired vision system based on the octopus visual system.     

Using neural network-based tools for building learning organisations

During the last decade, most European countries have institutionalised a series of measures in order to protect the environment and to provide better public services. Such measures have also been adopted in The Netherlands by Rijkswaterstaat (RWS), part of the ministry of communications and responsible for roads and waterways. One of the current RWS objectives is to provide a quality motorway of high impact to the Dutch road network. In recent years, RWS has committed itself wholeheartedly to Total Quality Management (TQM). It is in the process of making fundamental changes in motorway construction management. For instance, it is considering giving the responsibilities for the quality of road maintenance to sub-contractors. Managers involved in the TQM project feel the need to develop the attitudes and quality-consciousness of all parties involved in the management of the construction and maintenance of the motorway. In an initial stage, it is essential to study the current attitudes to quality in general and to policies related to `providing a high-quality road' of all parties involved. The research project described below is a step in this direction. The research project took place within the broader framework of RWS's interest in `organisational learning'. The research project capitalised on the ability of Artificial Neural Networks (ANNs) to study human behaviour and integrated this with current RWS needs. More particularly, ANNs were used to visualise stakeholder perceptions and to monitor changes in these perceptions. As a result of the research project, a software tool was developed which is currently used in order to support the organisational change process.