Generalization in interactive networks: the benefits of inhibitory competition and Hebbian learning

Computational models in cognitive neuroscience should ideally use biological properties and powerful computational principles to produce behavior consistent with psychological findings. Error-driven backpropagation is computationally powerful and has proven useful for modeling a range of psychological data but is not biologically plausible. Several approaches to implementing backpropagation in a biologically plausible fashion converge on the idea of using bidirectional activation propagation in interactive networks to convey error signals. This article demonstrates two main points about these error-driven interactive networks: (1) they generalize poorly due to attractor dynamics that interfere with the network's ability to produce novel combinatorial representations systematically in response to novel inputs, and (2) this generalization problem can be remedied by adding two widely used mechanistic principles, inhibitory competition and Hebbian learning, that can be independently motivated for a variety of biological, psychological, and computational reasons. Simulations using the Leabra algorithm, which combines the generalized recirculation (GeneRec), biologically plausible, error-driven learning algorithm with inhibitory competition and Hebbian learning, show that these mechanisms can result in good generalization in interactive networks. These results support the general conclusion that cognitive neuroscience models that incorporate the core mechanistic principles of interactivity, inhibitory competition, and error-driven and Hebbian learning satisfy a wider range of biological, psychological, and computational constraints than models employing a subset of these principles.     

Advances in Design and Application of Spiking Neural Networks

This paper presents new findings in the design and application of biologically plausible neural networks based on spiking neuron models, which represent a more plausible model of real biological neurons where time is considered as an important feature for information encoding and processing in the brain. The design approach consists of an evolutionary strategy based supervised training algorithm, newly developed by the authors, and the use of different biologically plausible neuronal models. A dynamic synapse (DS) based neuron model, a biologically more detailed model, and the spike response model (SRM) are investigated in order to demonstrate the efficacy of the proposed approach and to further our understanding of the computing capabilities of the nervous system. Unlike the conventional synapse, represented as a static entity with a fixed weight, employed in conventional and SRM-based neural networks, a DS is weightless and its strength changes upon the arrival of incoming input spikes. Therefore its efficacy depends on the temporal structure of the impinging spike trains. In the proposed approach, the training of the network free parameters is achieved using an evolutionary strategy where, instead of binary encoding, real values are used to encode the static and DS parameters which underlie the learning process. The results show that spiking neural networks based on both types of synapse are capable of learning non-linearly separable data by means of spatio-temporal encoding. Furthermore, a comparison of the obtained performance with classical neural networks (multi-layer perceptrons) is presented.     

脉冲神经网络:模型、学习算法与应用

脉冲神经网络是目前最具有生物解释性的人工神经网络,是类脑智能领域的核心组成部分.首先介绍各类常用的脉冲神经元模型以及前馈和循环型脉冲神经网络结构;然后介绍脉冲神经网络的时间编码方式,在此基础上,系统地介绍脉冲神经网络的学习算法,包括无监督学习和监督学习算法,其中监督学习算法按照梯度下降算法、结合STDP规则的算法和基于脉冲序列卷积核的算法3大类别分别展开详细介绍和总结;接着列举脉冲神经网络在控制领域、模式识别领域和类脑智能研究领域的应用,并在此基础上介绍各国脑计划中,脉冲神经网络与神经形态处理器相结合的案例;最后分析脉冲神经网络目前所存在的困难和挑战.     

UML顺序图与状态图的一致性检查

用户可使用UML从不同角度对系统进行建模,但不同视图间存在信息冗余,可能导致视图不一致问题。该文针对具有多种逻辑语义的顺序图提出分析方法,为复杂层次结构的状态图引入有限状态自动机,利用自动机分解算法得到自动机树。制定新的顺序图和状态图一致性检查准则和Promela代码结构,用模型检验工具SPIN进行顺序图及其相关状态图的一致性检验。     

Petri网模型的扩展与检测

扩展了基本Petri网,提出了更加适合模型检测的MCPN方法,并将MCPN模型转换成模型检测工具SPIN的输入语言——PROMELA。使用SPIN完成对系统模型的检测,以提高软件设计的可靠性。在转换过程中,考虑了对当前情态下处于激活状态的多个变迁的同时激发;并提出了一种处理Petri网公平性问题的解决方案。     

Spatio–Temporal Adaptation in the Unsupervised Development of Networked Visual Neurons

There have been many computational models mimicking the visual cortex that are based on spatial adaptations of unsupervised neural networks. In this paper, we present a new model called neuronal cluster which includes spatial as well as temporal weights in its unified adaptation scheme. The “in-place” nature of the model is based on two biologically plausible learning rules, Hebbian rule and lateral inhibition. We present the mathematical demonstration that the temporal weights are derived from the delay in lateral inhibition. By training with the natural videos, this model can develop spatio–temporal features such as orientation selective cells, motion sensitive cells, and spatio–temporal complex cells. The unified nature of the adaption scheme allows us to construct a multilayered and task-independent attention selection network which uses the same learning rule for edge, motion, and color detection, and we can use this network to engage in attention selection in both static and dynamic scenes.      

UML模型检测方法的研究

统一建模语言（UML）是设计和分析软件系统最常用的方法，如何保证UML模型满足某些特性是一个非常重要的问题，而模型检测是一种能够有效提高系统可靠性的自动化技术。研究了使用简单进程元语言解释器（SPIN）对UML模型进行检测的方法。首先对UML模型进行形式化描述，使用层次自动机来描述状态图，然后根据层次自动机的操作语义将状态图和类图的部分信息转化为SPIN的输入语言PROMELA，使用SPIN来验证模型是否满足给定的线性时序逻辑所描述的系统约束，通过LTL公式描述顺序图的方式来验证与状态图之间的一致性问题。项目组基于此方法还开发了一套模型检测工具UMLChecker。     

A Stack-Slicing Algorithm for Multi-Core Model Checking

The broad availability of multi-core chips on standard desktop PCs provides strong motivation for the development of new algorithms for logic model checkers that can take advantage of the additional processing power. With a steady increase in the number of available processing cores, we would like the performance of a model checker to increase as well – ideally linearly. The new trend implies a change of focus away from cluster computers towards shared memory systems. In this paper we discuss the multi-core algorithms that are in development for the SPIN model checker.     

A SPIN-based approach for detecting vulnerabilities in C programs

An approach is presented for the detection of software vulnerabilities using the widely known SPIN model checker. Classes of vulnerabilities in C programs that can be detected using the presented approach are discussed. We present the results of experiments on detecting vulnerabilities in student-made software tools implementing array processing algorithms.     

有色噪声中非线性动态系统的BP网络辨识

前馈多层神经网络为复杂的非线性系统提供了一种极具吸引力的模型结构。本文不利用仅含一个隐层的前馈多层神经网络来拟合离散时间非线性动态系统的问题进行了探讨。由于有色噪声的存在会导致网络模型偏差产生，文中引入了一种对噪声建模的方案。借助于非线性模型检验技术，本文给出了在有色噪声存在的情况下，利用ＢＰ网络辨识离散时间非线性动态系统的一般方法，仿真结果亦表明该方法行之有效。     

Learning Temporally Encoded Patterns in Networks of Spiking Neurons

Networks of spiking neurons are very powerful and versatile models for biological and artificial information processing systems. Especially for modelling pattern analysis tasks in a biologically plausible way that require short response times with high precision they seem to be more appropriate than networks of threshold gates or models that encode analog values in average firing rates. We investigate the question how neurons can learn on the basis of time differences between firing times. In particular, we provide learning rules of the Hebbian type in terms of single spiking events of the pre- and postsynaptic neuron and show that the weights approach some value given by the difference between pre- and postsynaptic firing times with arbitrary high precision.     

Application of temporal difference learning rules in short‐term traffic flow prediction

Studying dynamic behaviours of a transportation system requires the use of the system mathematical models as well as prediction of traffic flow in the system. Therefore, traffic flow prediction plays an important role in today's intelligent transportation systems. This article introduces a new approach to short‐term daily traffic flow prediction based on artificial neural networks. Among the family of neural networks, multi‐layer perceptron (MLP), radial basis function (RBF) neural network and wavenets have been selected as the three best candidates for performing traffic flow prediction. Moreover, back‐propagation (BP) has been adapted as the most efficient learning scheme in all the cases. It is shown that the coefficients produced by temporal signals improve the performance of the BP learning (BPL) algorithm. Temporal signals provide researchers with a new model of temporal difference BP learning algorithm (TDBPL). The capability and performance of TDBPL algorithm are examined by means of simulation in order to prove that the wavelet theory, with its multi‐resolution ability in comparison to RBF neural networks, is a suitable algorithm in traffic flow forecasting. It is also concluded that despite MLP applications, RBF neural networks do not provide negative forecasts. In addition, the local minimum problems are inevitable in MLP algorithms, while RBF neural networks and wavenet networks do not encounter them.     

SABIO: A BIOLOGICALLY PLAUSIBLE CONNECTIONIST APPROACH TO AUTOMATIC TEXT SUMMARIZATION

An implementation of a computational tool to generate new summaries from new source texts is presented, by means of the connectionist approach (artificial neural networks). Among other contributions that this work intends to bring to natural language processing research, the use of a more biologically plausible connectionist architecture and training for automatic summarization is emphasized. The choice relies on the expectation that it may bring an increase in computational efficiency when compared to the so-called biologically implausible algorithms.     

Inferring causal networks using fuzzy cognitive maps and evolutionary algorithms with application to gene regulatory network reconstruction

Fuzzy cognitive maps have been widely used as abstract models for complex networks. Traditional ways to construct fuzzy cognitive maps rely on domain knowledge. In this paper, we propose to use fuzzy cognitive map learning algorithms to discover domain knowledge in the form of causal networks from data. More specifically, we propose to infer gene regulatory networks from gene expression data. Furthermore, a new efficient fuzzy cognitive map learning algorithm based on a decomposed genetic algorithm is developed to learn large scale networks. In the proposed algorithm, the simulation error is used as the objective function, while the model error is expected to be minimized. Experiments are performed to explore the feasibility of this approach. The high accuracy of the generated models and the approximate correlation between simulation errors and model errors suggest that it is possible to discover causal networks using fuzzy cognitive map learning. We also compared the proposed algorithm with ant colony optimization, differential evolution, and particle swarm optimization in a decomposed framework. Comparison results reveal the advantage of the decomposed genetic algorithm on datasets with small data volumes, large network scales, or the presence of noise.     

Formal Verification of Object-Oriented Graph Grammars Specifications

Concurrent object-oriented systems are ubiquitous due to the importance of networks and the current demands for modular, reusable, and easy to develop software. However, checking the correctness of such systems is a hard task, mainly due to concurrency and inheritance aspects. In this paper we present an approach to the verification of concurrent object-oriented systems. We use graph grammars equipped with object oriented features (including inheritance and polymorphism) as the specification formalism, and define a translation from such specifications to Promela, the input language of the SPIN model checker.     

Implementing spiking neural networks for real-time signal-processing and control applications: a model-validated FPGA approach.

In this paper, we present two versions of a hardware processing architecture for modeling large networks of leaky-integrate-and-fire (LIF) neurons; the second version provides performance enhancing features relative to the first. Both versions of the architecture use fixed-point arithmetic and have been implemented using a single field-programmable gate array (FPGA). They have successfully simulated networks of over 1000 neurons configured using biologically plausible models of mammalian neural systems. The neuroprocessor has been designed to be employed primarily for use on mobile robotic vehicles, allowing bio-inspired neural processing models to be integrated directly into real-world control environments. When a neuroprocessor has been designed to act as part of the closed-loop system of a feedback controller, it is imperative to maintain strict real-time performance at all times, in order to maintain integrity of the control system. This resulted in the reevaluation of some of the architectural features of existing hardware for biologically plausible neural networks (NNs). In addition, we describe a development system for rapidly porting an underlying model (based on floating-point arithmetic) to the fixed-point representation of the FPGA-based neuroprocessor, thereby allowing validation of the hardware architecture. The developmental system environment facilitates the cooperation of computational neuroscientists and engineers working on embodied (robotic) systems with neural controllers, as demonstrated by our own experience on the Whiskerbot project, in which we developed models of the rodent whisker sensory system.     

基于精细神经元的类脑感知学习模型

大脑如何实现学习以及感知功能对于人工智能和神经科学领域均是一个重要问题.现有人工神经网络由于结构和计算机制与真实大脑相差较大,无法直接用于理解真实大脑学习以及处理感知任务的机理.树突神经元模型是一种对大脑神经元树突信息处理过程进行建模仿真的计算模型,相比人工神经网络更接近生物真实.使用树突神经网络模型处理学习感知任务对理解真实大脑的学习过程有重要作用.然而,现有基于树突神经元网络的学习模型大都局限于简化树突模型,无法完整建模树突的信号处理过程.针对这一问题,提出一种基于精细中型多棘神经元网络的学习模型,使得精细神经网络可以通过学习完成相应感知任务.实验表明,在经典的图像分类任务上,所提模型可以达到很好的分类性能.此外,精细神经网络对于噪声干扰有很强的鲁棒性.对网络特性进行进一步分析,发现学习后网络中的神经元表现出了刺激选择性这种神经科学中的经典现象,表明所提模型具有一定的生物可解释性,同时也表明刺激选择特性可能是大脑通过学习完成感知任务的一种重要特性.     

Modeling and complexity in neural networks

In this paper, we study nonlinear spatio-temporal dynamics in synchronous and asynchronous chaotic neural networks from the viewpoint of the modeling and complexity of the dynamic brain. First, the possible roles and functions of spatio-temporal neurochaos are considered with a model of synchronous chaotic neural networks composed of a neuron model with a chaotic map. Second, deterministic point-process dynamics with spikes of action potentials is demonstrated with a biologically more plausible model of asynchronous chaotic neural networks. Last, the possibilities of inventing a new brain-type of computing system are discussed on the basis of these models of chaotic neural networks. This work was presented, in part, at the Third International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–21, 1998.     

Computation by Time

Over the last years, the amount of research performed in the field of spiking neural networks has been growing steadily. Spiking neurons are modeled to approximate the complex dynamic behavior of biological neurons. They communicate via discrete impulses called spikes with the actual information being encoded in the timing of these spikes. As already pointed out by Maass in his paper on the third generation of neural network models, this renders time a central factor for neural computation. In this paper, we investigate at different levels of granularity how absolute time and relative timing enable new ways of biologically inspired neural information processing. At the lowest level of single spiking neurons, we give an overview of coding schemes and learning techniques which rely on precisely timed neural spikes. A high-level perspective is provided in the second part of the paper which focuses on the role of time at the network level. The third aspect of time considered in this work is related to the interfacing of neural networks with real-time systems. In this context, we discuss how the concepts of computation by time can be implemented in computer simulations and on specialized neuromorphic hardware. The contributions of this paper are twofold: first, we show how the exact modeling of time in spiking neural networks serves as an important basis for powerful computation based on neurobiological principles. Second, by presenting a range of diverse learning techniques, we prove the biologically plausible applicability of spiking neural networks to real world problems like pattern recognition and path planning.     

Automated analysis of timed security: a case study on web privacy

This paper presents a case study on an automated analysis of real-time security models. The case study on a web system (originally proposed by Felten and Schneider) is presented that shows a timing attack on the privacy of browser users. Three different approaches are followed: LH-Timed Automata (analyzed using the model checker HyTech), finite-state automata (analyzed using the model checker NuSMV), and process algebras (analyzed using the model checker CWB-NC ). A comparative analysis of these three approaches is given.