Artificial neural networks: a review of commercial hardware

Artificial neural networks (ANN) became a common solution for a wide variety of problems in many fields, such as control and pattern recognition to name but a few. Many solutions found in these and other ANN fields have reached a hardware implementation phase, either commercial or with prototypes. The most frequent solution for the implementation of ANN consists of training and implementing the ANN within a computer. Nevertheless this solution might be unsuitable because of its cost or its limited speed. The implementation might be too expensive because of the computer and too slow when implemented in software. In both cases dedicated hardware can be an interesting solution.

The necessity of dedicated hardware might not imply building the hardware since in the last two decades several commercial hardware solutions that can be used in the implementation have reached the market.

Unfortunately not every integrated circuit will fit the needs: some will use lower precision, some will implement only certain types of networks, some don’t have training built in and the information is not easy to find.

This article is confined to reporting the commercial chips that have been developed specifically for ANN, leaving out other solutions.

This option has been made because most of the other solutions are based on cards which are built either with these chips, Digital Signal Processors or Reduced Instruction Set Computers.     

Artificial neural networks for job shop simulation

This paper explores the use of artificial neural networks (ANNs) as a valid alternative to the traditional job-shop simulation approach. Feed forward, multi-layered neural network metamodels were trained through the back-error-propagation (BEP) learning algorithm to provide a versatile job-shop scheduling analysis framework. The constructed neural network architectures were capable of satisfactorily estimating the manufacturing lead times (MLT) for orders simultaneously processed in a four-machine job shop. The MLTs produced by the developed ANN models turned out to be as valid as the data generated from three well-known simulation packages, i.e. Arena, SIMAN, and ProModel. The ANN outputs proved not to be substantially different from the results provided by other valid models such as SIMAN and ProModel when compared against the adopted baseline, Arena. The ANN-based simulations were able to fairly capture the underlying relationship between jobs' machine sequences and their resulting average flowtimes, which proves that ANNs are a viable tool for stochastic simulation metamodeling.     

Artificial neural networks for vibration based inverse parametric identifications: A review

Vibration behavior of any solid structure reveals certain dynamic characteristics and property parameters of that structure. Inverse problems dealing with vibration response utilize the response signals to find out input factors and/or certain structural properties. Due to certain drawbacks of traditional solutions to inverse problems, ANNs have gained a major popularity in this field. This paper reviews some earlier researches where ANNs were applied to solve different vibration-based inverse parametric identification problems. The adoption of different ANN algorithms, input-output schemes and required signal processing were denoted in considerable detail. In addition, a number of issues have been reported, including the factors that affect ANNs’ prediction, as well as the advantage and disadvantage of ANN approaches with respect to general inverse methods Based on the critical analysis, suggestions to potential researchers have also been provided for future scopes.     

城市交通信号的ANN自校正预测控制

提出一种基于人工神经网络的城市交通信号的自校正预测控制方法.充分考虑相邻交叉路口之间交通流的强耦合性,在此基础上建立关于队长的交通模型;其中,受控路口下一周期到达的车辆数用人工神经网络(ANN)来预测;通过该ANN还可获得确定最佳周期长度所需要的交通参量,因此还可预测下一周期的长度;上述预测值均用实测信息进行反馈校正,在此基础上即可给出带约束的预测控制算法,从而确定下一周期的控制策略.仿真实例表明该方法具有较好的控制效果.     

Artificial neural networks for predicting dorsal pressures on the foot surface while walking

In this work, artificial neural networks (ANNs) are proposed to predict the dorsal pressure over the foot surface exerted by the shoe upper while walking. A model that is based on the multilayer perceptron (MLP) is used since it can provide a single equation to model the exerted pressure for all the materials used as shoe uppers. Five different models are produced, one model for each one of the four subjects under study and an overall model for the four subjects. The inputs to the neural model include the characteristics of the material and the positions during a whole step of 14 pressure sensors placed on the foot surface. The goal is to find models with good generalization capabilities, (i.e., models that work appropriately not only for the cases used to train the model but also for new cases) in order to have a useful predictor in routine practice. New cases may involve either new materials for the same subject or even new subjects and new materials. To accomplish this goal, two thirds of the patterns are trained to obtain the model (training data set) and the remaining third is kept for validation purposes. The achieved accuracy was very satisfactory since correlation coefficients between the predicted output and the actual pressure in the validation data were higher than 0.95 for those models developed for individual subjects. For the much more challenging problem of an overall prediction for all the subjects, the correlation coefficient was close to 0.9 in the validation data set (i.e., with data not previously seen by the model).     

Stable control of nonlinear systems using neural networks

A neural-network-based direct control architecture is presented that achieves output tracking for a class of continuous-time nonlinear plants, for which the nonlinearities are unknown. The controller employs neural networks to perform approximate input/output plant linearization. The network parameters are adapted according to a stability principle. The architecture is based on a modification of a method previously proposed by the authors, where the modification comprises adding a sliding control term to the controller. This modification serves two purposes: first, as suggested by Sanner and Slotine,¹ sliding control compensates for plant uncertainties outside the state region where the networks are used, thus providing global stability; second, the sliding control compensates for inherent network approximation errors, hence improving tracking performance. A complete stability and tracking error convergence proof is given and the setting of the controller parameters is discussed. It is demonstrated that as a result of using sliding control, better use of the network's approximation ability can be achieved, and the asymptotic tracking error can be made dependent only on inherent network approximation errors and the frequency range of unmodelled dynamical modes. Two simulations are provided to demonstrate the features of the control method.     

Classical and superposed learning for quantum weightless neural networks

A supervised learning algorithm for quantum neural networks (QNN) based on a novel quantum neuron node implemented as a very simple quantum circuit is proposed and investigated. In contrast to the QNN published in the literature, the proposed model can perform both quantum learning and simulate the classical models. This is partly due to the neural model used elsewhere which has weights and non-linear activations functions. Here a quantum weightless neural network model is proposed as a quantisation of the classical weightless neural networks (WNN). The theoretical and practical results on WNN can be inherited by these quantum weightless neural networks (qWNN). In the quantum learning algorithm proposed here patterns of the training set are presented concurrently in superposition. This superposition-based learning algorithm (SLA) has computational cost polynomial on the number of patterns in the training set.     

混沌系统的混合遗传神经网络控制

在小扰动控制技术基础上,将暂态误差预测方法和遗传算法结合起来,提出了一种混合遗传神经网络控制非线性混沌系统的新方法(简称HyGANN).通过增强学习训练,HyGANN可产生控制混沌状态的小扰动时间序列信号,Henon映射的计算机仿真结果表明,它不仅有效镇定混沌周期1,2等低周期轨道,还可成功将高周期混轨道变成期望周期行为.     

Review of pulse-coupled neural networks

This paper reviews the research status of pulse-coupled neural networks (PCNN) in the past decade. Considering there are too many publications about the PCNN, we summarize main approaches and point out interesting parts of the PCNN researches rather than contemplate to go into details of particular algorithms or describe results of comparative experiments. First, the current status of the PCNN and some modified models are briefly introduced. Second, we review the PCNN applications in the field of image processing (e.g. image segmentation, image enhancement, image fusion, object and edge detection, pattern recognition, etc.), then applications in other fields also are mentioned. Subsequently, some existing problems are summarized, while we give some suggestions for the solutions to some puzzles. Finally, the trend of the PCNN is pointed out.     

Inductive transfer with context-sensitive neural networks

Context-sensitive Multiple Task Learning, or csMTL, is presented as a method of inductive transfer which uses a single output neural network and additional contextual inputs for learning multiple tasks. Motivated by problems with the application of MTL networks to machine lifelong learning systems, csMTL encoding of multiple task examples was developed and found to improve predictive performance. As evidence, the csMTL method is tested on seven task domains and shown to produce hypotheses for primary tasks that are often better than standard MTL hypotheses when learning in the presence of related and unrelated tasks. We argue that the reason for this performance improvement is a reduction in the number of effective free parameters in the csMTL network brought about by the shared output node and weight update constraints due to the context inputs. An examination of IDT and SVM models developed from csMTL encoded data provides initial evidence that this improvement is not shared across all machine learning models.     

Continuous process control using neural networks

We present some adaptive control strategies based on neural networks that can be used for designing controllers for continuous process control problems. Specifically, a learning algorithm has been formulated based on reinforcement learning, a weakly supervised learning technique, to solve set-point control and control scheduling for continuous processes where the process cannot be modeled easily. It is shown how reinforcement learning can be used to learn the control strategy adaptively based on exploration of the control space without making assumptions about the process model. A new learning scheme, handicapped learning, was developed to learn a control schedule that specifies a schedule of set points. Applications studied include the control of a nonisothermal continuously stirred tank reactor at its unstable state and the learning of the daily time-temperature schedule for an environment controller. Experimental results demonstrate good learning performance, indicating that the learning algorithm can be used for solving transient startup and boundary value control problems.     

Learning from demonstration in robots: Experimental comparison of neural architectures

Robots have played an important role in the automation of computer aided manufacturing. The classical robot control implementation involves an expensive key step of model-based programming. An intuitive way to reduce this expensive exercise is to replace programming with machine learning of robot actions from demonstration where a (learner) robot learns an action by observing a demonstrator robot performing the same. To achieve this learning from demonstration (LFD) different machine learning techniques such as Artificial Neural Networks (ANN), Genetic Algorithms, Hidden Markov Models, Support Vector Machines, etc. can be used. This piece of work focuses exclusively on ANNs. Since ANNs have many standard architectural variations divided into two basic computational categories namely the recurrent networks and feed-forward networks, representative networks from each have been selected for study, i.e. Feed Forward Multilayer Perceptron (FF) network for feed-forward networks category and Elman (EL), and Nonlinear Autoregressive Exogenous Model (NARX) networks for the recurrent networks category. The main objective of this work is to identify the most suitable neural architecture for application of LFD in learning different robot actions. The sensor and actuator streams of demonstrated action are used as training data for ANN learning. Consequently, the learning capability is measured by comparing the error between demonstrator and corresponding learner streams. To achieve fairness in comparison three steps have been taken. First, Dynamic Time Warping is used to measure the error between demonstrator and learner streams, which gives resilience against translation in time. Second, comparison statistics are drawn between the best, instead of weight-equal, configurations of competing architectures so that learning capability of any architecture is not forced handicap. Third, each configuration's error is calculated as the average of ten trials of all possible learning sequences with random weight initialization so that the error value is independent of a particular sequence of learning or a particular set of initial weights. Six experiments are conducted to get a performance pattern of each architecture. In each experiment, a total of nine different robot actions were tested. Error statistics thus obtained have shown that NARX architecture is most suitable for this learning problem whereas Elman architecture has shown the worst suitability. Interestingly the computationally lesser MLP gives much lower and slightly higher error statistics compared to the computationally superior Elman and NARX neural architectures, respectively.     

The bridge relating process neural networks and traditional neural networks

The process neural network (PrNN) is an ANN model suited for solving the learning problems with signal inputs, whose elementary unit is the process neuron (PN), an emerging neuron model. There is an essential difference between the process neuron and traditional neurons, but there also exists a relation between them. The former can be approximated by the latter within any precision. First, the PN model and some PrNNs are introduced in brief. And then, two PN approximating theorems are presented and proved in detail. Each theorem gives an approximating model to the PN model, i.e., the time-domain feature expansion model and the orthogonal decomposition feature expansion model. Some corollaries are given for the PrNNs based on these two theorems. Thereafter, simulation studies are performed on some simulated signal sets and a real dataset. The results show that the PrNN can effectively suppress noises polluting the signals and generalize quite well. Finally some problems on PrNNs are discussed and further research directions are suggested.     

Artificial neural networks for infectious diarrhea prediction using meteorological factors in Shanghai (China)

Infectious diarrhea is an important public health problem around the world. Meteorological factors have been strongly linked to the incidence of infectious diarrhea. Therefore, accurately forecast the number of infectious diarrhea under the effect of meteorological factors is critical to control efforts. In recent decades, development of artificial neural network (ANN) models, as predictors for infectious diseases, have created a great change in infectious disease predictions. In this paper, a three layered feed-forward back-propagation ANN (BPNN) model trained by Levenberg–Marquardt algorithm was developed to predict the weekly number of infectious diarrhea by using meteorological factors as input variable. The meteorological factors were chosen based on the strongly relativity with infectious diarrhea. Also, as a comparison study, the support vector regression (SVR), random forests regression (RFR) and multivariate linear regression (MLR) also were applied as prediction models using the same dataset in addition to BPNN model. The 5-fold cross validation technique was used to avoid the problem of overfitting in models training period. Further, since one of the drawbacks of ANN models is the interpretation of the final model in terms of the relative importance of input variables, a sensitivity analysis is performed to determine the parametric influence on the model outputs. The simulation results obtained from the BPNN confirms the feasibility of this model in terms of applicability and shows better agreement with the actual data, compared to those from the SVR, RFR and MLR models. The BPNN model, described in this paper, is an efficient quantitative tool to evaluate and predict the infectious diarrhea using meteorological factors.     

An ensemble of neural networks for weather forecasting

This study presents the applicability of an ensemble of artificial neural networks (ANNs) and learning paradigms for weather forecasting in southern Saskatchewan, Canada. The proposed ensemble method for weather forecasting has advantages over other techniques like linear combination. Generally, the output of an ensemble is a weighted sum, which are weight-fixed, with the weights being determined from the training or validation data. In the proposed approach, weights are determined dynamically from the respective certainties of the network outputs. The more certain a network seems to be of its decision, the higher the weight. The proposed ensemble model performance is contrasted with multi-layered perceptron network (MLPN), Elman recurrent neural network (ERNN), radial basis function network (RBFN), Hopfield model (HFM) predictive models and regression techniques. The data of temperature, wind speed and relative humidity are used to train and test the different models. With each model, 24-h-ahead forecasts are made for the winter, spring, summer and fall seasons. Moreover, the performance and reliability of the seven models are then evaluated by a number of statistical measures. Among the direct approaches employed, empirical results indicate that HFM is relatively less accurate and RBFN is relatively more reliable for the weather forecasting problem. In comparison, the ensemble of neural networks produced the most accurate forecasts.     

Predicting peak pinch strength: Artificial neural networks vs. regression

Cumulative trauma disorders (CTDs) of the upper extremities are one of the major ergonomics areas of research. Pinching is a common risk factor associated with the development of hand/wrist CTDs. The capacity standards of peak pinch strength for various postures are needed to design the tasks in harmony with the workers. This paper describes the formulation, building and comparison of pinch strength prediction models that were obtained using two approaches: Statistical and artificial neural networks (ANN). Statistical and ANN models were developed to predict the peak chuck pinch strength as a function of different combinations of five elbow and seven shoulder flexion angles, and several anthropometric and physiological variables. The two modeling approaches were compared. The results indicate ANN models to provide more accurate predictions over the standard statistical models.     

Classification error of multilayer perceptron neural networks

In subject classification, artificial neural networks (ANNS) are efficient and objective classification methods. Thus, they have been successfully applied to the numerous classification fields. Sometimes, however, classifications do not match the real world, and are subjected to errors. These problems are caused by the nature of ANNS. We discuss these on multilayer perceptron neural networks. By studying of these problems, it helps us to have a better understanding on its classification.     

Artificial neural network approach for solving fuzzy differential equations

The current research attempts to offer a novel method for solving fuzzy differential equations with initial conditions based on the use of feed-forward neural networks. First, the fuzzy differential equation is replaced by a system of ordinary differential equations. A trial solution of this system is written as a sum of two parts. The first part satisfies the initial condition and contains no adjustable parameters. The second part involves a feed-forward neural network containing adjustable parameters (the weights). Hence by construction, the initial condition is satisfied and the network is trained to satisfy the differential equations. This method, in comparison with existing numerical methods, shows that the use of neural networks provides solutions with good generalization and high accuracy. The proposed method is illustrated by several examples.     

A learning automata-based algorithm for determination of the number of hidden units for three-layer neural networks

There is no method to determine the optimal topology for multi-layer neural networks for a given problem. Usually the designer selects a topology for the network and then trains it. Since determination of the optimal topology of neural networks belongs to class of NP-hard problems, most of the existing algorithms for determination of the topology are approximate. These algorithms could be classified into four main groups: pruning algorithms, constructive algorithms, hybrid algorithms and evolutionary algorithms. These algorithms can produce near optimal solutions. Most of these algorithms use hill-climbing method and may be stuck at local minima. In this article, we first introduce a learning automaton and study its behaviour and then present an algorithm based on the proposed learning automaton, called survival algorithm, for determination of the number of hidden units of three layers neural networks. The survival algorithm uses learning automata as a global search method to increase the probability of obtaining the optimal topology. The algorithm considers the problem of optimization of the topology of neural networks as object partitioning rather than searching or parameter optimization as in existing algorithms. In survival algorithm, the training begins with a large network, and then by adding and deleting hidden units, a near optimal topology will be obtained. The algorithm has been tested on a number of problems and shown through simulations that networks generated are near optimal.