Low-resource hardware implementation of the hyperbolic tangent for artificial neural networks

Artificial neural networks are a widespread tool with application in a variety of areas ranging from the social sciences to engineering. Many of these applications have reached a hardware implementation phase and have been documented in scientific papers. Unfortunately, most of the implementations have a simplified hyperbolic tangent replacement which has been the most common problem, as well as the most resource-consuming block in terms of hardware. This paper proposes a low-resource hardware implementation of the hyperbolic tangent, by using the simplest solution in order to obtain the lowest error possible thus far with a set of 25 polynomials of third order, obtained with Chebyshev interpolations. The results obtained show that the solution proposed holds a low error while simultaneously promising the use of low resources, as only third-order polynomials are used.     

一种神经网络硬件实现的可重构设计

以BP网络为例,提出了一种可重构神经网络硬件实现方法。通过可重构体系结构、可重构部件的设计,可以灵活地实现不同规模、传递函数及学习方法的神经网络,从而搭建起神经网络快速硬件实现的平台。经过对一个模式识别问题的实现和测试,证明了这种设计方法的可行性。     

Layered tile architecture for efficient hardware spiking neural networks

Spiking Neural Network (SNN) is the most recent computational model that can emulate the behaviour of biological neuron system. However, its main drawback is that it is computationally intensive, which limits the system scalability. This paper highlights and discusses the importance and significance of emulating SNNs in hardware devices. A layer-level tile architecture (LTA) is proposed for hardware-based SNNs. The LTA employs a two-level sharing mechanism of computing components at the synapse and neuron levels, and achieves a trade-off between computational complexity and hardware resource costs. The LTA is implemented on a Xilinx FPGA device. Experimental results demonstrate that this approach is capable of scaling to large hardware-based SNNs.     

A survey of software and hardware use in artificial neural networks

Artificial neural networks (ANNs) have been widely used over the last three decades. During this period, many hardware and software solutions have been developed and today a new user entering the field can make a fast trial to this artificial intelligence solution with commercial software and hardware, instead of developing a solution from scratch thus saving a lot of time. This work aims at helping new and experienced users even further by sharing the ANNs experience in software and hardware collected. This was achieved through a survey questionnaire about present and past used solutions of software and hardware, as well as future prospects for the development of application areas. To further enlighten the reader, a logistic regression (LR) statistical analysis is performed on the obtained results to extract additional details about the answers obtained from the ANN community. The LR statistical analysis verifies whether the researchers with more than 25 years of experience in ANNs use self-written code when compared to those with less years of experience in the area. The LR statistical analysis also verifies whether researchers with less than 25 years of experience in ANNs use some platform to develop their hardware when compared to those who have more years of experience.     

Issues on representational capabilities of artificial neural networks and their implementation

This work is an organized review on the representational capabilities of artificial neural networks and the questions that arise in their implementation. It covers the Kolmogorov's superposition theorem and different statements regarding how it could be related to the representational power of neural networks. Generalization capability of neural networks is then considered and methods of improving this capability are discussed. Some theorems and statements concerning the bound on the number of hidden layers, form of the activation function, and time complexity of training of neural networks are other subjects of this article. © 1995 John Wiley & Sons, Inc.     

Post-quantum signature schemes for efficient hardware implementation

Post-quantum cryptosystems have attracted a great interest, from researchers, latest. This work introduces two new forms of the hidden discrete logarithm problem and three new post-quantum signature schemes. The finite non-commutative associative algebras of two types are used as the algebraic support of the proposed cryptoschemes: i) containing a global two-sided unit and ii) containing a large set of global left-sided units. The illustrated FPGA implementation results, show the efficiency of the proposed cryptographic schemes, in hardware approaches. Detailed comparisons, with other security hardware implementations, are also presented.     

Machine fusion to enhance the topology preservation of vector quantization artificial neural networks

Artificial neural networks techniques have been successfully applied in vector quantization (VQ) encoding. The objective of VQ is to statistically preserve the topological relationships existing in a data set and to project the data to a lattice of lower dimensions, for visualization, compression, storage, or transmission purposes. However, one of the major drawbacks in the application of artificial neural networks is the difficulty to properly specify the structure of the lattice that best preserves the topology of the data. To overcome this problem, in this paper we introduce merging algorithms for machine-fusion, boosting-fusion-based and hybrid-fusion ensembles of SOM, NG and GSOM networks. In these ensembles not the output signals of the base learners are combined, but their architectures are properly merged. We empirically show the quality and robustness of the topological representation of our proposed algorithm using both synthetic and real benchmarks datasets.     

Compartmental modelling with artificial neural networks

In this paper, we introduce an abbreviated compartmental modelling scheme which may be of interest to those in neuron- based adaptive systems because of the additional scope it provides for studying biologically-inspired learning mechanisms. The scheme, although not as flexible and precise as the general compartmental approach, allows one to design Hodgkin-Huxley style cells, and passive dendritic trees with an arbitrary number of synaptic connections. The trade-offs made for computational performance, may make the modelling scheme more appropriate for practical applications. The modelling scheme is based upon artificial neural networks, which we have used to represent cylindrical compartments (both passive and active) of different lengths, two types of voltage-dependent channels, and basic chemical synapses with variable time constants.     

Speech recognition with artificial neural networks

In this paper, artificial neural networks were used to accomplish isolated speech recognition. The topic was investigated in two steps, consisting of the pre-processing part with Digital Signal Processing (DSP) techniques and the post-processing part with Artificial Neural Networks (ANN). These two parts were briefly explained and speech recognizers using different ANN architectures were implemented on Matlab. Three different neural network models; multi layer back propagation, Elman and probabilistic neural networks were designed. Performance comparisons with similar studies found in the related literature indicated that our proposed ANN structures yield satisfactory results.     

Tracking topology structure adaptively with deep neural networks

Object tracking still remains challenging in computer vision because of the severe object variation, e.g., deformation, occlusion, and rotation. To handle the object variation and achieve robust object tracking performance, we propose a novel relationship-based tracking algorithm using neural networks in this paper. Compared with existing approaches in the literature, our method assumes the targeted object to be consisted of several parts and considers the evolution of the topology structure among these parts. After training a candidate neural network for predicting the probable areas each part may locate at in the successive frame, we then design a novel collaboration neural network to determine the precise area each part will locate at with account for the topology structure among these individual parts, which is learned from their historical physical locations during online tracking process. Experimental results show that the proposed method outperforms state-of-the-art trackers on a benchmark dataset, yielding the significant improvements in accuracy on high-distorted sequences.

     

Rule-based artificial neural networks

Neural networks that are integrated with rule-based systems having a knowledge base offer more capabilities than networks not integrated with such systems.     

Knowledge-based artificial neural networks

Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN (Knowledge-Based Artificial Neural Networks) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.     

Ear recognition with feed-forward artificial neural networks

Biometrics has become one of the most important techniques in recognizing a person’s identity. A person’s face, iris and fingerprint are mostly used in biometrics today. It has been established that there are no two ears exactly alike, even in the cases of identical twins. In this paper, we define a 7-element ear feature set and design and train a feed-forward artificial neural network to recognize a human ear. We train and test the network with 51 ear pictures from 51 different persons. Simulation experiments with various networks with various number of layers and number of neurons per layer and with and without noise are conducted. Results indicate that a 95 % ear recognition accuracy is achieved with a simple 3-layer feed-forward neural network with only a total of 18 neurons even in the presence of some noise. This design outstands previous work in simplicity and implementation cost.     

Atrial fibrillation classification with artificial neural networks

The acquired 72 normal sinus rhythm ECGs and 80 ECGs with atrial fibrillation (AF) are decomposed with ‘db10’ Daebauchies wavelets at level 6 and power spectral density was calculated for each decomposed signal with Welch method. Average power spectral density was calculated for six subbands and normalized to be used as input to the neural network. Levenberg-Marquart backpropagation feed forward neural network was built from logarithmic sigmoid transfer functions in three-layer form. The trained network was tested on 24 normal and 28 AF state ECGs. The classification performance was accomplished as 100% accurate.     

GPU implementation of neural networks

Graphics processing unit (GPU) is used for a faster artificial neural network. It is used to implement the matrix multiplication of a neural network to enhance the time performance of a text detection system. Preliminary results produced a 20-fold performance enhancement using an ATI RADEON 9700 PRO board. The parallelism of a GPU is fully utilized by accumulating a lot of input feature vectors and weight vectors, then converting the many inner-product operations into one matrix operation. Further research areas include benchmarking the performance with various hardware and GPU-aware learning algorithms.     

Short-term memory circuit using hardware ring neural networks

A number of studies have recently been made on various neuron models and neural networks. This research is studied for applications to engineering problems and an understanding of the information processing functions of living organisms. We are studying an asynchronous neural network using a pulse-type hardware neuron model (P-HNM). Recently, we have been trying to construct a short-term memory circuit using hardware ring neural networks (RNN) with P-HNM. In this article, we discuss the construction of a short-term memory circuit using the hardware RNN, and conduct experiments that explain the characteristics of the network through circuit simulation using PSpice. As a result, we verify that the RNN which is proposed in this article can be used as the short-term memory circuit.This work was presented, in part, at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004     

An associative memory-based learning model with an efficient hardware implementation in FPGA

In this paper we propose a learning model based on a short- and long-term memory and a ranking mechanism which manages the transition of reference vectors between the two memories. Furthermore, an optimization algorithm is used to adjust the reference vectors components as well as their distribution, continuously. Comparing to other learning models like neural networks, the main advantage of the proposed model is that a pre-training phase is unnecessary and it has a hardware-friendly structure which makes it implementable by an efficient LSI architecture without requiring a large amount of resources. A prototype system is implemented on an FPGA platform and tested with real data of handwritten and printed English characters delivering satisfactory classification results.     

Information and topology in attractor neural networks

A wide range of networks, including those with small-world topology, can be modeled by the connectivity ratio and randomness of the links. Both learning and attractor abilities of a neural network can be measured by the mutual information (MI) as a function of the load and the overlap between patterns and retrieval states. In this letter, we use MI to search for the optimal topology with regard to the storage and attractor properties of the network in an Amari-Hopfield model. We find that while an optimal storage implies an extremely diluted topology, a large basin of attraction leads to moderate levels of connectivity. This optimal topology is related to the clustering and path length of the network. We also build a diagram for the dynamical phases with random or local initial overlap and show that very diluted networks lose their attractor ability.     

Integrating artificial neural networks with rule-based expert systems

The Rule-Based (RB) and the Artificial Neural Network (ANN) approaches to expert systems development have each demonstrated some specific advantages and disadvantages. These two approaches can be integrated to exploit the advantages and minimize the disadvantages of each method used alone. An RB/ANN integrated approach is proposed to facilitate the development of an expert system which provides a “high-performance” knowledge-based network, an explanation facility, and an input/output facility. In this case study an expert system designed to assist managers in forecasting the performance of stock prices is developed to demonstrate the advantages of this integrated approach and how it can enhance support for managerial decision making.