A Survey of Accelerator Architectures for Deep Neural Networks

Recently, due to the availability of big data and the rapid growth of computing power, artificial intelligence (AI) has regained tremendous attention and investment. Machine learning (ML) approaches have been successfully applied to solve many problems in academia and in industry. Although the explosion of big data applications is driving the development of ML, it also imposes severe challenges of data processing speed and scalability on conventional computer systems. Computing platforms that are dedicatedly designed for AI applications have been considered, ranging from a complement to von Neumann platforms to a “must-have” and stand-alone technical solution. These platforms, which belong to a larger category named “domain-specific computing,” focus on specific customization for AI. In this article, we focus on summarizing the recent advances in accelerator designs for deep neural networks (DNNs)—that is, DNN accelerators. We discuss various architectures that support DNN executions in terms of computing units, dataflow optimization, targeted network topologies, architectures on emerging technologies, and accelerators for emerging applications. We also provide our visions on the future trend of AI chip designs.     

Acoustic Emission Recognition Based on a Two-Streams Convolutional Neural Network

The Convolutional Neural Network (CNN) is a widely used deep neural network. Compared with the shallow neural network, the CNN network has better performance and faster computing in some image recognition tasks. It can effectively avoid the problem that network training falls into local extremes. At present, CNN has been applied in many different fields, including fault diagnosis, and it has improved the level and efficiency of fault diagnosis. In this paper, a two-streams convolutional neural network (TCNN) model is proposed. Based on the short-time Fourier transform (STFT) spectral and Mel Frequency Cepstrum Coefficient (MFCC) input characteristics of two-streams acoustic emission (AE) signals, an AE signal processing and classification system is constructed and compared with the traditional recognition methods of AE signals and traditional CNN networks. The experimental results illustrate the effectiveness of the proposed model. Compared with single-stream convolutional neural network and a simple Long Short-Term Memory (LSTM) network, the performance of TCNN which combines spatial and temporal features is greatly improved, and the accuracy rate can reach 100% on the current database, which is 12% higher than that of single-stream neural network.     

Logic Computing with Stateful Neural Networks of Resistive Switches

Brain‐inspired neural networks can process information with high efficiency, thus providing a powerful tool for pattern recognition and other artificial intelligent tasks. By adopting binary inputs/outputs, neural networks can be used to perform Boolean logic operations, thus potentially surpassing complementary metal–oxide–semiconductor logic in terms of area efficiency, execution time, and computing parallelism. Here, the concept of stateful neural networks consisting of resistive switches, which can perform all logic functions with the same network topology, is introduced. The neural network relies on physical computing according to Ohm's law, Kirchhoff 's law, and the ionic migration within an output switch serving as the highly nonlinear activation function. The input and output are nonvolatile resistance states of the devices, thus enabling stateful and cascadable logic operations. Applied voltages provide the synaptic weights, which enable the convenient reconfiguration of the same circuit to serve various logic functions. The neural network can solve all two‐input logic operations with just one step, except for the exclusive‐OR (XOR) needing two sequential steps. 1‐bit full adder operation is shown to take place with just two steps and five resistive switches, thus highlighting the high efficiencies of space, time, and energy of logic computing with the stateful neural network.     

目标地震动信号的特征提取及识别研究

地面目标的地震动信号是目标识别的关键,本文研究了机动目标地震动特性,总结出目标的地震动信号的特征提取规律,并将神经网络方法用于目标的地震动信号的分类识别中。给出了识别结构及改进的BP算法,并将改进的BP算法用于实际目标的地震动信号的分类识别,得到令人满意的结果。     

Emerging Memristive Artificial Synapses and Neurons for Energy-Efficient Neuromorphic Computing

Memristors have recently attracted significant interest due to their applicability as promising building blocks of neuromorphic computing and electronic systems. The dynamic reconfiguration of memristors, which is based on the history of applied electrical stimuli, can mimic both essential analog synaptic and neuronal functionalities. These can be utilized as the node and terminal devices in an artificial neural network. Consequently, the ability to understand, control, and utilize fundamental switching principles and various types of device architectures of the memristor is necessary for achieving memristor-based neuromorphic hardware systems. Herein, a wide range of memristors and memristive-related devices for artificial synapses and neurons is highlighted. The device structures, switching principles, and the applications of essential synaptic and neuronal functionalities are sequentially presented. Moreover, recent advances in memristive artificial neural networks and their hardware implementations are introduced along with an overview of the various learning algorithms. Finally, the main challenges of the memristive synapses and neurons toward high-performance and energy-efficient neuromorphic computing are briefly discussed. This progress report aims to be an insightful guide for the research on memristors and neuromorphic-based computing.     

人工神经网络和机械故障诊断

智能化诊断是现代故障诊断技术发展的主要趋势,人工神经网络技术的出现为这种智能化提供了一个全新的途径。本文首先简单介绍了人工神经网络的基本性能及几个重要模型,着重探讨了人工神经网络技术在机械故障诊断领域中预测与控制、工况监测与故障分类诊断、模糊诊断和基于专家系统的故障诊断等几个主要方面的应用,指出人工神经网络技术与现有的信号处理、模式识别、模糊逻辑、专家系统等技术相结合,以解决故障信号分析与处理、故障模式识别以及故障论域专家知识的组织和推理等问题,必将加快智能化诊断发展的进程。可以预料:基于人工神经网络的故障诊断技术将具有广阔的发展与应用前景,并且随着VLsI 技术的发展,这一新技术必将广泛地应用于各种诊断实例。最后讨论了进一步值得研究的方向。     

人工神经网络多目标法用于辅助V－PTC材料研制

采用人工神经网络方法进行Ｖ－ＰＴＣ材料设计的多目标优化。应用偏相关指数和目标拟合变量的方法探寻具有较好性能样品的工艺条件和配方参数，并据此设计合成了数个优化样本，经实验测试达到了设计要求。     

Development of Al2O3/TiN Ceramic Cutting Tool Materials by Artificial Neural Networks

The artificial neural networks (ANN) which have broad application were proposed to develop multiphase ceramic cutting tool materials. Based on the back propagation algorithm of the forward multilayer perceptron, the models to predict volume content of composition in particle reinforced ceramics are established. The Al2O3/TiNl ceramic cutting tool material was developed by ANN, whose mechanical properties fully satisfy the cutting requirements.     

不规则零件优化排样的神经网络混合优化算法

 提出一种利用人工神经网络求解不规则件排样问题的混合优化方法.该方法首先把排样和制造工艺联系起来,将多边形各边向外扩充,为零件预留加工余量;然后采用自组织特征映射模型（SOM）和Hopfield人工神经网络相结合的方法,运用SOM神经网络对初始在板材内随机排布的不规则零件进行平移,逐步减小不规则零件之间的重叠面积,求得各零件的最优位置,再运用Hopfield神经网络对平移后的零件旋转,进行迭代运算,当能量函数达到稳定状态时,得到各排样零件的最优旋转角度组合,实现自动排样.算法可以解决不规则件和矩形件在规则板材以及不规则板材上的排样问题,实例证明了该算法的有效性和实用性.     

Development of Al_2O_3/TiN Ceramie Cutting Tool Materials by Artificial Neural Networks

The artificial neural networks (ANN) which have broad application were proposed to develop multiphase ceramie cutting tool materials. Based on the back propagation algorithm of the forward multilayer perceptron, the models to predict volume content of composition in particie reinforced ceramies are established. The Al2O3/TiN ceramie cutting tool material was developed by ANN, whose mechanicai properties fully satisfy the cutting requirements.     

Development of Al¬2O¬3/TiN Ceramic Cutting Tool Materials by Artificial Neural Networks

The artificial neural networks (ANN) which have broad application were proposed to develop multiphase ceramic cutting tool materials. Based on the back propagation algorithm of the forward multilayer perceptron, the models to predict volume content of composition in particle reinforced ceramics are established. The Al¬2O¬3/TiN ceramic cutting tool material was developed by ANN, whose mechanical properties fully satisfy the cutting requirements.     

Neuromorphic Engineering: From Biological to Spike-Based Hardware Nervous Systems

The human brain is a sophisticated, high-performance biocomputer that processes multiple complex tasks in parallel with high efficiency and remarkably low power consumption. Scientists have long been pursuing an artificial intelligence (AI) that can rival the human brain. Spiking neural networks based on neuromorphic computing platforms simulate the architecture and information processing of the intelligent brain, providing new insights for building AIs. The rapid development of materials engineering, device physics, chip integration, and neuroscience has led to exciting progress in neuromorphic computing with the goal of overcoming the von Neumann bottleneck. Herein, fundamental knowledge related to the structures and working principles of neurons and synapses of the biological nervous system is reviewed. An overview is then provided on the development of neuromorphic hardware systems, from artificial synapses and neurons to spike-based neuromorphic computing platforms. It is hoped that this review will shed new light on the evolution of brain-like computing.     

Artificial neural networks as alternative tool for minimizing error predictions in manufacturing ultradeformable nanoliposome formulations

This work was aimed at determining the feasibility of artificial neural networks (ANN) by implementing backpropagation algorithms with default settings to generate better predictive models than multiple linear regression (MLR) analysis. The study was hypothesized on timolol-loaded liposomes. As tutorial data for ANN, causal factors were used, which were fed into the computer program. The number of training cycles has been identified in order to optimize the performance of the ANN. The optimization was performed by minimizing the error between the predicted and real response values in the training step. The results showed that training was stopped at 10?000 training cycles with 80% of the pattern values, because at this point the ANN generalizes better. Minimum validation error was achieved at 12 hidden neurons in a single layer. MLR has great prediction ability, with errors between predicted and real values lower than 1% in some of the parameters evaluated. Thus, the performance of this model was compared to that of the MLR using a factorial design. Optimal formulations were identified by minimizing the distance among measured and theoretical parameters, by estimating the prediction errors. Results indicate that the ANN shows much better predictive ability than the MLR model. These findings demonstrate the increased efficiency of the combination of ANN and design of experiments, compared to the conventional MLR modeling techniques.     

A Novel Combinational Convolutional Neural Network for Automatic Food-Ingredient Classification

With the development of deep learning and Convolutional Neural Networks (CNNs), the accuracy of automatic food recognition based on visual data have significantly improved. Some research studies have shown that the deeper the model is, the higher the accuracy is. However, very deep neural networks would be affected by the overfitting problem and also consume huge computing resources. In this paper, a new classification scheme is proposed for automatic food-ingredient recognition based on deep learning. We construct an up-to-date combinational convolutional neural network (CBNet) with a subnet merging technique. Firstly, two different neural networks are utilized for learning interested features. Then, a well-designed feature fusion component aggregates the features from subnetworks, further extracting richer and more precise features for image classification. In order to learn more complementary features, the corresponding fusion strategies are also proposed, including auxiliary classifiers and hyperparameters setting. Finally, CBNet based on the well-known VGGNet, ResNet and DenseNet is evaluated on a dataset including 41 major categories of food ingredients and 100 images for each category. Theoretical analysis and experimental results demonstrate that CBNet achieves promising accuracy for multi-class classification and improves the performance of convolutional neural networks.     

Neural Networks in Applied Statistics

Artificial neural networks are computer algorithms or computer programs derived in part from attempts to model the activity of nerve cells. They have been applied to pattern recognition, classification, and optimization problems in the physical and chemical sciences, as well as in other fields. We introduce the principles of the multilayer feedforward network that is among the most commonly used neural networks in practical problems. The relevance of neural network models for the applied statistician is considered using a time series prediction problem as an example. The multilayer feedforward neural network uses a nonlinear function of the predictors to obtain predictions for future time series values. We illustrate the considerations involved in specifying a neural network model and evaluate the accuracy of neural network models relative to the accuracy obtained using other computer-intensive, nonmodel-based techniques.     

量子神经计算和量子遗传算法的理论分析和应用

经过比较研究发现,在量子计算与神经网络和遗传算法之间,不论在计算思想上还是模型表达上,都存在着许多相似之处,这些相似性启发人们去研究基于量子理论的神经网络和遗传算法模型,一方面探索神经网络和遗传算法在量子系统上的实现方法,另一方面研究量子理论启发下的新的神经网络与遗传算法模型。本文总结了本课题组近年来在量子计算与神经网络和遗传算法相结合领域的研究工作,包括量子系统实现神经计算的理论分析,量子神经网络物理模型的研究,基于量子概率表达的量子遗传算法及其应用研究等,并对今后的发展提出了展望。     

Application of Hybrid Learning Neural Fuzzy Systems in Reliability Prediction

This study presents a hybrid learning neural fuzzy system for accurately predicting system reliability. Neural fuzzy system learning with and without supervision has been successfully applied in control systems and pattern recognition problems. This investigation modifies the hybrid learning fuzzy systems to accept time series data and therefore examines the feasibility of reliability prediction. Two neural network systems are developed for solving different reliability prediction problems. Additionally, a scaled conjugate gradient learning method is applied to accelerate the training in the supervised learning phase. Several existing approaches, including feed‐forward multilayer perceptron (MLP) networks, radial basis function (RBF) neural networks and Box–Jenkins autoregressive integrated moving average (ARIMA) models, are used to compare the performance of the reliability prediction. The numerical results demonstrate that the neural fuzzy systems have higher prediction accuracy than the other methods. Copyright © 2005 John Wiley & Sons, Ltd.     

Neural networks and modelling in vacuum science

The paper is an attempt to describe how neural networks may be used as an approximation-modelling tool. A brief survey of the evolution of the approximation theory and neural networks is presented. Practical applications are based on modelling of vacuum science problems, especially the modelling of a cold cathode pressure gauge. The problem of approximation of wide range functions, that are one of the characteristics of vacuum science problems, is introduced. Parameters such as pressure or cathode current span over several decades and neural networks are not suitable for any approximation of such functions; therefore, two strategies need to be introduced, and these are described. The approximation made by the neural network is obtained by the training process. The models obtained by several independent repetitions of training processes performed on the same training set lead to slightly different results. Therefore the definition of training stability is introduced and described. Finally, some practical hints regarding the neural network synthesis (design) are given.     

Predictive modelling of turning operations using response surface methodology,artificial neural networks and support vector regression

This paper focuses on developing empirical models for predicting surface roughness, tool wear and power required in turning operations. These response parameters are mainly dependent upon cutting velocity, feed and cutting time. Three competing data mining techniques, response surface methodology (RSM), artificial neural networks (ANN) and support vector regression (SVR), are applied in developing the empirical models. The data of 27 experiments have been used to generate, compare and evaluate the proposed models of tool wear, power required and surface roughness for the selected tool/material combination. Testing results demonstrate that the models developed in this research are suitable for predicting the response parameters with a satisfactory goodness of fit. It has been found that ANN and SVR models are much better than regression and RSM models for predicting the three response parameters. Finally, some future research directions are outlined.     

基于智能元件和神经网络的复合材料损伤监测

介绍了基于智能元件和人工神经网络技术对复合材料损伤监测的原理，回顾了人工神经网络在复合材料健康监测中的应用，总结了国内外的主要成果，提出了现在所存在的问题，并对应用神经网络进行复合材料的健康监测技术进行了展望。