多核并行脉冲神经网络模拟器的设计

脉冲神经网络属于第三代人工神经网络,它是更具有生物可解释性的神经网络模型。随着人们对脉冲神经网络不断深入地研究,不仅神经元空间结构更为复杂,而且神经网络结构规模也随之增大。以串行计算的方式,难以在个人计算机上实现脉冲神经网络的模拟仿真。为此,设计了一个多核并行的脉冲神经网络模拟器,对神经元进行编码与映射,自定义路由表解决了多核间的网络通信,以时间驱动为策略,实现核与核间的动态同步,在模拟器上进行脉冲神经网络的并行计算。以Izhikevich脉冲神经元为模型,在模拟环境下进行仿真实验,结果表明多核并行计算相比传统的串行计算在效率方面约有两倍的提升,可为类似的脉冲神经网络的模拟并行化设计提供参考。     

Condition monitoring of the cutting process using a self-organizing spiking neural network map

This paper presents a new approach to sensor based condition monitoring using a self-organizing spiking neuron network map. Experimental evidence suggests that biological neural networks, which communicate through spikes, use the timing of these spikes to encode and compute information in a more efficient way. The paper introduces the basis of a simplified version of the Self-Organizing neural architecture based on Spiking Neurons. The fundamental steps for the development of this computational model are presented as well as some experimental evidence of its performance. It is shown that this computational architecture has a greater potential to unveil embedded information in tool wear monitoring data sets and that faster learning occurs if compared to traditional sigmoidal neural networks.     

Performance,optimization, and fitness: Connecting applications to architectures

Recent trends involving multicore processors and graphical processing units (GPUs) focus on exploiting task‐ and thread‐level parallelism. In this paper, we have analyzed various aspects of the performance of these architectures including NVIDIA GPUs, and multicore processors such as Intel Xeon, AMD Opteron, IBM's Cell Broadband Engine. The case study used in this paper is a biological spiking neural network (SNN), implemented with the Izhikevich, Wilson, Morris–Lecar, and Hodgkin–Huxley neuron models. The four SNN models have varying requirements for communication and computation making them useful for performance analysis of the hardware platforms. We report and analyze the variation of performance with network (problem size) scaling, available optimization techniques and execution configuration. A Fitness performance model, that predicts the suitability of the architecture for accelerating an application, is proposed and verified with the SNN implementation results. The Roofline model, another existing performance model, has also been utilized to determine the hardware bottleneck(s) and attainable peak performance of the architectures. Significant speedups for the four SNN neuron models utilizing these architectures are reported; the maximum speedup of 574x was observed in our GPU implementation. Our results and analysis show that a proper match of architecture with algorithm complexity provides the best performance. Copyright © 2010 John Wiley & Sons, Ltd.     

面向图像识别的多层脉冲神经网络学习算法综述

相较于第1代和第2代神经网络,第3代神经网络的脉冲神经网络是一种更加接近于生物神经网络的模型,因此更具有生物可解释性和低功耗性。基于脉冲神经元模型,脉冲神经网络可以通过脉冲信号的形式模拟生物信号在神经网络中的传播,通过脉冲神经元的膜电位变化来发放脉冲序列,脉冲序列通过时空联合表达不仅传递了空间信息还传递了时间信息。当前面向模式识别任务的脉冲神经网络模型性能还不及深度学习,其中一个重要原因在于脉冲神经网络的学习方法不成熟,深度学习中神经网络的人工神经元是基于实数形式的输出,这使得其可以使用全局性的反向传播算法对深度神经网络的参数进行训练,脉冲序列是二值性的离散输出,这直接导致对脉冲神经网络的训练存在一定困难,如何对脉冲神经网络进行高效训练是一个具有挑战的研究问题。本文首先总结了脉冲神经网络研究领域中的相关学习算法,然后对其中主要的方法:直接监督学习、无监督学习的算法以及ANN2SNN的转换算法进行分析介绍,并对其中代表性的工作进行对比分析,最后基于对当前主流方法的总结,对未来更高效、更仿生的脉冲神经网络参数学习方法进行展望。     

脉冲神经网络:模型、学习算法与应用

脉冲神经网络是目前最具有生物解释性的人工神经网络,是类脑智能领域的核心组成部分.首先介绍各类常用的脉冲神经元模型以及前馈和循环型脉冲神经网络结构;然后介绍脉冲神经网络的时间编码方式,在此基础上,系统地介绍脉冲神经网络的学习算法,包括无监督学习和监督学习算法,其中监督学习算法按照梯度下降算法、结合STDP规则的算法和基于脉冲序列卷积核的算法3大类别分别展开详细介绍和总结;接着列举脉冲神经网络在控制领域、模式识别领域和类脑智能研究领域的应用,并在此基础上介绍各国脑计划中,脉冲神经网络与神经形态处理器相结合的案例;最后分析脉冲神经网络目前所存在的困难和挑战.     

Spiking神经元输入脉冲扰动敏感性研究

脉冲神经网络是一种基于生物的网络模型,它的输入输出为具有时间特性的脉冲序列,其运行机制相比其他传统人工神经网络更加接近于生物神经网络。神经元之间通过脉冲序列传递信息,这些信息通过脉冲的激发时间编码能够更有效地发挥网络的学习性能。脉冲神经元的时间特性导致了其工作机制较为复杂,而spiking神经元的敏感性反映了当神经元输入发生扰动时输出的spike的变化情况,可以作为研究神经元内部工作机制的工具。不同于传统的神经网络,spiking神经元敏感性定义为输出脉冲的变化时刻个数与运行时间长度的比值,能直接反映出输入扰动对输出的影响程度。通过对不同形式的输入扰动敏感性的分析,可以看出spiking神经元的敏感性较为复杂,当全体突触发生扰动时,神经元为定值,而当部分突触发生扰动时,不同突触的扰动会导致不同大小的神经元敏感性。     

Delay learning and polychronization for reservoir computing

We propose a multi-timescale learning rule for spiking neuron networks, in the line of the recently emerging field of reservoir computing. The reservoir is a network model of spiking neurons, with random topology and driven by STDP (spike-time-dependent plasticity), a temporal Hebbian unsupervised learning mode, biologically observed. The model is further driven by a supervised learning algorithm, based on a margin criterion, that affects the synaptic delays linking the network to the readout neurons, with classification as a goal task. The network processing and the resulting performance can be explained by the concept of polychronization, proposed by Izhikevich [Polychronization: computation with spikes, Neural Comput. 18(2) (2006) 245–282], on physiological grounds. The model emphasizes that polychronization can be used as a tool for exploiting the computational power of synaptic delays and for monitoring the topology and activity of a spiking neuron network.     

通用的不带延迟的同质脉冲神经膜系统

脉冲神经膜系统是一种膜系统中吸收了脉冲神经网络特点的新型生物计算装置,具有强大的计算能力。同质脉冲神经膜系统是指一种所有神经元具有相同规则集合的脉冲神经膜系统的变体。研究了突触上带权值和突触上不带权值的两种同质脉冲神经膜系统在不使用具有延迟的激发规则情况下的计算通用性问题,并证明了这两种不带延迟的同质脉冲神经膜系统无论是工作在产生模式下,还是工作在接收模式下都是计算通用的。解决了曾湘祥、张兴义和潘林强提出的关于不带延迟的同质脉冲神经膜系统是否具有计算通用性的公开问题。     

基于卷积计算的多层脉冲神经网络的监督学习

针对脉冲神经元基于精确定时的多脉冲编码信息的特点,提出了一种基于卷积计算的多层脉冲神经网络监督学习的新算法。该算法应用核函数的卷积计算将离散的脉冲序列转换为连续函数,在多层前馈脉冲神经网络结构中,使用梯度下降的方法得到基于核函数卷积表示的学习规则,并用来调整神经元连接的突触权值。在实验部分,首先验证了该算法学习脉冲序列的效果,然后应用该算法对Iris数据集进行分类。结果显示,该算法能够实现脉冲序列复杂时空模式的学习,对非线性模式分类问题具有较高的分类正确率。     

不同重连概率的小世界脉冲神经网络抗扰功能研究

现今各种电磁干扰对电子系统造成的不良影响越来越严重,传统防护方式的局限性日益凸显。电磁仿生学由此被提出,目的是通过借鉴生物体的自适应抗扰的优良特性,以期建立新的防护模式。构建了以Izhikevich神经元模型为节点,兴奋性和抑制性突触可塑性模型共同调节基于小世界网络拓扑的小世界脉冲神经网络;基于复杂网络理论对比分析了不同重连概率的小世界网络的拓扑特性;对比分析了不同重连概率的小世界脉冲神经网络在高斯白噪声刺激下的抗扰功能。实验结果表明:小世界网络的平均路径长度和全局效率值受重连概率的影响较小,平均聚类系数和小世界属性受重连概率的影响较大;构建的不同重连概率的脉冲神经网络均具有一定抗扰功能且高聚类系数和低平均路径长度显著的小世界脉冲神经网络抗扰功能最优。     

Digital implementation of Hodgkin–Huxley neuron model for neurological diseases studies

Neurological disorders affect millions of people which influence their cognitive and/or motor capabilities. The realization of a prosthesis must consider the biological activity of the cells and the connection between machine and biological cells. Biomimetic neural network is one solution in front of neurological diseases. The neuron replacement should be processed by reproducing the timing and the shape of the spike. Several mathematical equations which model neural activities exist. The most biologically plausible one is the Hodgkin–Huxley (HH) model. The connection between electrical devices and living cells require a tunable real-time system. The field programmable gate array (FPGA) is a nice component including flexibility, speed and stability. Here, we propose an implementation of HH neurons in FPGA serving as a presage for a modulating network opening a large scale of possibilities such as damage cells replacement and the study of the effect of the cells disease on the neural network.     

基于脉冲频率与输入电流关系的SNN训练算法

脉冲神经网络(spiking neural network,SNN)以异步事件驱动,支持大规模并行计算,在改善同步模拟神经网络的计算效率方面具有巨大潜力.然而,目前SNN仍然面临无法直接训练的难题,为此,受到神经科学领域关于LIF(leaky integrate-and-fire)神经元响应机制研究启发,提出了一种新的...     

脉冲神经元环路振荡发放与环路选择

在Izhikevich提出的脉冲神经元模型中,引入随机变化的输入电流,使神经元的脉冲发放具有随机性,不同数量的神经元采用连接权值组成网络的脉冲发放。实验结果表明,选择适当的连接权值可以得到环路的持续振荡发放。通过脉冲发放,可以在网络中选择神经环路,完成环路记忆联想过程,并给出研究脉冲神经智能的新思路。     

脉冲神经膜系统在穷举使用规则下产生的二进制字符串语言

脉冲神经膜系统是基于大脑中神经元之间通过突触相瓦协作、处理脉冲的生物现象提出的一种新的模型,文中在穷举使用规则的情况下考虑将脉冲神经膜系统作为串语言产生器:当输出神经元发送出一个或多个神经脉冲时,用数字1表示,否则用数字0表示,当计算停止时,把产生的二进制串定义为系统的计算结果.在文中,作者让明了在穷举使用规则的情况下,具有一个神经元的脉冲神经膜系统可以刻画二进制有限语言,并且证明了在不限制神经元个数的情况下,该系统可以刻画递归可枚举语言.     

基于脉冲神经网络的迁移学习算法与软件框架

使用脉冲序列进行数据处理的脉冲神经网络具有优异的低功耗特性,但由于学习算法不成熟,多层网络训练存在收敛困难的问题。利用反向传播网络具有学习算法成熟和训练速度快的特点,设计一种迁移学习算法。基于反向传播网络完成训练过程,并通过脉冲编码规则和自适应的权值映射关系,将训练结果迁移至脉冲神经网络。实验结果表明,在多层脉冲神经网络中,迁移学习算法能够有效解决训练过程中收敛困难的问题,在MNIST数据集和CIFAR-10数据集上的识别准确率分别达到98.56%和56.00%,且具有微瓦级别的低功耗特性。     

Credit score classification using spiking extreme learning machine

Credit score classification is a prominent research problem in the banking or financial industry, and its predictive performance is responsible for the profitability of financial industry. This paper addresses how Spiking Extreme Learning Machine (SELM) can be effectively used for credit score classification. A novel spike-generating function is proposed in Leaky Nonlinear Integrate and Fire Model (LNIF). Its interspike period is computed and utilized in the extreme learning machine (ELM) for credit score classification. The proposed model is named as SELM and is validated on five real-world credit scoring datasets namely: Australian, German-categorical, German-numerical, Japanese, and Bankruptcy. Further, results obtained by SELM are compared with back propagation, probabilistic neural network, ELM, voting-based Q-generalized extreme learning machine, Radial basis neural network and ELM with some existing spiking neuron models in terms of classification accuracy, Area under curve (AUC), H-measure and computational time. From the experimental results, it has been noticed that improvement in accuracy and execution time for the proposed SELM is highly statistically important for all aforementioned credit scoring datasets. Thus, integrating a biological spiking function with ELM makes it more efficient for categorization.     

A two-variable silicon neuron circuit based on the Izhikevich model

The silicon neuron is an analog electronic circuit that reproduces the dynamics of a neuron. It is a useful element for artificial neural networks that work in real time. Silicon neuron circuits have to be simple, and at the same time they must be able to realize rich neuronal dynamics in order to reproduce the various activities of neural networks with compact, low-power consumption, and an easy-to-configure circuit. We have been developing a silicon neuron circuit based on the Izhikevich model, which has rich dynamics in spite of its simplicity. In our previous work, we proposed a simple silicon neuron circuit with low power consumption by reconstructing the mathematical structure in the Izhikevich model using an analog electronic circuit. In this article, we propose an improved circuit in which all of the MOSFETs are operated in the sub-threshold region.     

基于脉冲神经网络的红外目标提取

模拟生物信息处理机制,设计了一种用于红外目标提取的脉冲神经网络(SNN)。首先,利用输入层脉冲神经元将激励图像转化为脉冲序列;其次,采用中间层脉冲神经元输出脉冲的密度编码红外图像目标的轮廓像素和非目标轮廓像素;最后,根据输出层神经元输出脉冲的密度是否超过阈值提取红外目标。实验结果表明,设计的脉冲神经网络具有较好的红外目标提取性能,并且符合生物视觉信息处理机制。     

Spiking neural P systems with neuron division and budding

Spiking neural P systems are a class of distributed and parallel computing models inspired by spiking neurons.In this work,the features of neuron division and neuron budding are introduced into the framework of spiking neural P systems,which are processes inspired by neural stem cell division. With neuron division and neuron budding,a spiking neural P system can generate exponential work space in polynomial time as the case for P systems with active membranes.In this way,spiking neural P systems can efficie...     

仿生型脉冲神经网络学习算法和网络模型

为解决脉冲神经网络训练困难的问题,基于仿生学思路,提出脉冲神经网络的权值学习算法和结构学习算法,设计一种含有卷积结构的脉冲神经网络模型,搭建适合脉冲神经网络的软件仿真平台。实验结果表明,权值学习算法训练的网络对MNIST数据集识别准确率能够达到84.12%,具备良好的快速收敛能力和低功耗特点;结构学习算法能够自动生成网络结构,具有高度生物相似性。