Advancing the boundaries of high-connectivity network simulation with distributed computing

The availability of efficient and reliable simulation tools is one of the mission-critical technologies in the fast-moving field of computational neuroscience. Research indicates that higher brain functions emerge from large and complex cortical networks and their interactions. The large number of elements (neurons) combined with the high connectivity (synapses) of the biological network and the specific type of interactions impose severe constraints on the explorable system size that previously have been hard to overcome. Here we present a collection of new techniques combined to a coherent simulation tool removing the fundamental obstacle in the computational study of biological neural networks: the enormous number of synaptic contacts per neuron. Distributing an individual simulation over multiple computers enables the investigation of networks orders of magnitude larger than previously possible. The software scales excellently on a wide range of tested hardware, so it can be used in an interactive and iterative fashion for the development of ideas, and results can be produced quickly even for very large networks. In contrast to earlier approaches, a wide class of neuron models and synaptic dynamics can be represented.     

A Neural Network with Evolutionary Neurons

A neural network, combining evolution and learning is introduced. The novel feature of the proposed network is the evolutionary character of its neurons. The argument of the transfer function performed by the neurons in the network is neither a linear nor polynomial function of the inputs to the neuron, but an unknown general function P(·). The adequate functional form P(·) for each neuron, is achieved during the learning period by means of genetic programming. The proposed neural network is applied to the problem domain of time series prediction of the Mackey-Glass delay differential equation. Simulation results indicate that the new neural network is effective.     

基于精细神经元的类脑感知学习模型

大脑如何实现学习以及感知功能对于人工智能和神经科学领域均是一个重要问题.现有人工神经网络由于结构和计算机制与真实大脑相差较大,无法直接用于理解真实大脑学习以及处理感知任务的机理.树突神经元模型是一种对大脑神经元树突信息处理过程进行建模仿真的计算模型,相比人工神经网络更接近生物真实.使用树突神经网络模型处理学习感知任务对理解真实大脑的学习过程有重要作用.然而,现有基于树突神经元网络的学习模型大都局限于简化树突模型,无法完整建模树突的信号处理过程.针对这一问题,提出一种基于精细中型多棘神经元网络的学习模型,使得精细神经网络可以通过学习完成相应感知任务.实验表明,在经典的图像分类任务上,所提模型可以达到很好的分类性能.此外,精细神经网络对于噪声干扰有很强的鲁棒性.对网络特性进行进一步分析,发现学习后网络中的神经元表现出了刺激选择性这种神经科学中的经典现象,表明所提模型具有一定的生物可解释性,同时也表明刺激选择特性可能是大脑通过学习完成感知任务的一种重要特性.     

Bankruptcy prediction with neural logic networks by means of grammar-guided genetic programming

The paper demonstrates the efficient use of hybrid intelligent systems for solving the classification problem of bankruptcy. The aim of the study is to obtain classification schemes able to predict business failure. Previous attempts to form efficient classifiers for the same problem using intelligent or statistical techniques are discussed throughout the paper. The application of neural logic networks by means of genetic programming is proposed. This is an advantageous approach enabling the interpretation of the network structure through set of expert rules, which is a desirable feature for field experts. These evolutionary neural logic networks are consisted of an innovative hybrid intelligent methodology, by which evolutionary programming techniques are used for obtaining the best possible topology of a neural logic network. The genetic programming process is guided using a context-free grammar and indirect encoding of the neural logic networks into the genetic programming individuals. Indicative classification results are presented and discussed in detail in terms of both, classification accuracy and solution interpretability.     

泛函网络神经元构造理论与方法

泛函网络是近年提出的一种对神经网络的有效推广.与神经网络不同,它处理的是一般的泛函模型,它在各个神经元之间的连接没有权值,并且神经元函数不固定的,往往是一给定的基函数的组合,泛函网络学习的目的就是求出神经元函数的精确表达式或近似表达式. 迄今关于泛函网络神经元基函数的存在性和选取方法缺乏理论依据.文中基于Banach空间中偏序理论,分析了泛函网络神经元基函数的存在性,给出了泛函网络神经元基函数选取方法,对于完善泛函网络的基础理论具有参考价值.     

精细神经网络仿真方法研究进展

树突对大脑神经元实现不同的信息处理功能有着重要作用。精细神经元模型是一种对神经元树突以及离子通道的信息处理过程进行精细建模的模型,可以帮助科学家在实验条件的限制之外探索树突信息处理的特性。由精细神经元组成的精细神经网络模型可通过仿真对大脑的信息处理过程进行模拟,对于理解树突的信息处理机制、大脑神经网络功能背后的计算机理具有重要作用。然而,精细神经网络仿真需要进行大量计算,如何对精细神经网络进行高效仿真是一个具有挑战的研究问题。本文对精细神经网络仿真方法进行梳理,介绍了现有主流仿真平台与核心仿真算法,以及可进一步提升仿真效率的高效仿真方法。将具有代表性的高效仿真方法按照发展历程以及核心思路分为网络尺度并行方法、神经元尺度并行方法以及基于GPU(graphics processing unit)的并行仿真方法3类。对各类方法的核心思路进行总结,并对各类方法中代表性工作的细节进行分析介绍。随后对各类方法所具有的优劣势进行分析对比,对一些经典方法进行总结。最后根据高效仿真方法的发展趋势,对未来研究工作进行展望。     

基于卷积计算的多层脉冲神经网络的监督学习

针对脉冲神经元基于精确定时的多脉冲编码信息的特点,提出了一种基于卷积计算的多层脉冲神经网络监督学习的新算法。该算法应用核函数的卷积计算将离散的脉冲序列转换为连续函数,在多层前馈脉冲神经网络结构中,使用梯度下降的方法得到基于核函数卷积表示的学习规则,并用来调整神经元连接的突触权值。在实验部分,首先验证了该算法学习脉冲序列的效果,然后应用该算法对Iris数据集进行分类。结果显示,该算法能够实现脉冲序列复杂时空模式的学习,对非线性模式分类问题具有较高的分类正确率。     

Neural Network Based Algorithm for Dynamic System Optimization

A class of artificial neural networks with a two‐layer feedback topology to solve nonlinear discrete dynamic optimization problems is developed. Generalized recurrent neuron models are introduced. A direct method to assign the weights of neural networks is presented. The method is based on Bellmann's Optimality Principle and on the interchange of information which occurs during the synaptic chemical processing among neurons. A comparative analysis of the computational requirements is made. The analysis shows advantages of this approach as compared to the standard dynamic programming algorithm. The technique has been applied to several important optimization problems, such as shortest path and control optimal problems.     

Functional abilities of a stochastic logic neural network

The authors have studied the information processing ability of stochastic logic neural networks, which constitute one of the pulse-coded artificial neural network families. These networks realize pseudoanalog performance with local learning rules using digital circuits, and therefore suit silicon technology. The synaptic weights and the outputs of neurons in stochastic logic are represented by stochastic pulse sequences. The limited range of the synaptic weights reduces the coding noise and suppresses the degradation of memory storage capacity. To study the effect of the coding noise on an optimization problem, the authors simulate a probabilistic Hopfield model (Gaussian machine) which has a continuous neuron output function and probabilistic behavior. A proper choice of the coding noise amplitude and scheduling improves the network's solutions of the traveling salesman problem (TSP). These results suggest that stochastic logic may be useful for implementing probabilistic dynamics as well as deterministic dynamics.     

Self-organizing learning array and its application to economic and financial problems

A new self-organizing learning array (SOLAR) system has been implemented in software. It is an information theory based learning machine capable of handling a wide variety of classification problems. It has self-reconfigurable processing cells (neurons) and an evolvable system structure. Entropy based learning is performed locally at each neuron, where neural functions and connections that correspond to the minimum entropy are adaptively learned. By choosing connections for each neuron, the system sets up the wiring and completes its self-organization. SOLAR classifies input data based on weighted statistical information from all neurons. Unlike artificial neural networks, its multi-layer structure scales well to large systems capable of solving complex pattern recognition and classification tasks. This paper shows its application in economic and financial fields. A reference to influence diagrams is also discussed. Several prediction and classification cases are studied. The results have been compared with the existing methods.     

支持向量机理论与基于规划的神经网络学习算法

近年来支持向量机（SVM）理论得到国外学者高度的重视,普遍认为这是神经网络学习的新研究方向,近来也开始得到国内学者的注意。该文将研究SVM理论与神经网络的规划算法的关系,首先指出,Vapnik的基于SVM的算法与该文作者1994年提出的神经网络的基于规划的算法是等价的,即在样本集是线性可分的情况下,二者求得的均是最大边缘（maximal margin)解。不同的是,前者（通常用拉格郎日乘子法）求解的复杂性将随规模呈指数增长,而后者的复杂性是规模的多项式函数。其次,作者将规划算法化为求一点到某一凸集上的投影,利用这个几何的直观,给出一个构造性的迭代求解算法－－“单纯形迭代算法”。新算法有很强的几何直观性,这个直观性将加深对神经网络（线性可分情况下）学习的理解,并由此导出一个样本集是线性可分的充分必要条件。另外,新算法对知识扩充问题,给出一个非常方便的增量学习算法。最后指出,“将一些必须满足的条件,化成问题的约束条件,将网络的某一性能,作为目标函数,将网络的学习问题化为某种规划问题来求解”的原则,将是研究神经网络学习问题的一个十分有效的办法。     

Representing diverse mathematical problems using neural networks in hybrid intelligent systems

In recent years, artificial neural networks have attracted considerable attention as candidates for novel computational systems. Computer scientists and engineers are developing neural networks as representational and computational models for problem solving: neural networks are expected to produce new solutions or alternatives to existing models. This paper demonstrates the flexibility of neural networks for modeling and solving diverse mathematical problems including Taylor series expansion, Weierstrass's first approximation theorem, linear programming with single and multiple objectives, and fuzzy mathematical programming. Neural network representations of such mathematical problems may make it possible to overcome existing limitations, to find new solutions or alternatives to existing models, and to achieve synergistic effects through hybridization.     

Sequence learning in differentially activated dendrites

Differentially activated areas of a dendrite permit the existence of zones with distinct rates of synaptic modification, and such areas can be individually accessed using a reference signal which localizes synaptic plasticity and memory trace retrieval to certain subregions of the dendrite. It is proposed that the neural machinery required in such a learning/retrieval mechanism could involve the NMDA receptor, in conjunction with the ability of dendrites to maintain differentially activated regions. In particular, it is suggested that such a parcellation of the dendrite allows the neuron to participate in multiple sequences, which can be learned without suffering from the 'wash-out' of synaptic efficacy associated with superimposition of training patterns. This is a biologically plausible solution to the stability-plasticity dilemma of learning in neural networks.     

多聚合过程神经元网络及其学习算法研究

针对系统输入为多元过程函数以及多维过程信号的信息处理问题,提出了多聚合过程神经元和多聚合过程神经元网络模型.多聚合过程神经元的输入和连接权均可以是多元过程函数,其聚合运算包括对多个输入函数的空间加权聚集和对多维过程效应的累积,可同时反映多个多元过程输入信号在多维空间上的共同作用影响以及过程效应的累积结果.多聚合过程神经元网络是由多聚合过程神经元和其它类型的神经元按照一定的结构关系组成的网络模型,按照输出是否为多元过程函数建立了前馈多聚合过程神经元网络的一般模型和输入输出均为过程函数的多聚合过程神经元网络模型,具有对多元过程信号输入输出关系的直接映射和建模能力.文中给出了一种基于多元函数基展开的梯度下降与数值计算相结合的学习算法,仿真实验结果表明了模型和算法对多元过程信号分类和多维动态过程模拟问题的适应性.     

自反馈神经网络的椭球学习算法

本文讨论自反馈神经网络的学习问题，指出联想记忆的神经网络的学习可以化为某种规划（优化）的问题来解，于是可借用规划数学中发展得成熟的优化技术来解自反馈神经网络的学习问题，文中给出一种称为椭球算法的学习方法，其计算复杂性是多项式型。     

Programming based learning algorithms of neural networks withself-feedback connections

Discusses the learning problem of neural networks with self-feedback connections and shows that when the neural network is used as associative memory, the learning problem can be transformed into some sort of programming (optimization) problem. Thus, the rather mature optimization technique in programming mathematics can be used for solving the learning problem of neural networks with self-feedback connections. Two learning algorithms based on programming technique are presented. Their complexity is just polynomial. Then, the optimization of the radius of attraction of the training samples is discussed using quadratic programming techniques and the corresponding algorithm is given. Finally, the comparison is made between the given learning algorithm and some other known algorithms     

Incremental learning of collaborative classifier agents with new class acquisition: An incremental genetic algorithm approach

A number of soft computing approaches such as neural networks, evolutionary algorithms, and fuzzy logic have been widely used for classifier agents to adaptively evolve solutions on classification problems. However, most work in the literature focuses on the learning ability of the individual classifier agent. This article explores incremental, collaborative learning in a multiagent environment. We use the genetic algorithm (GA) and incremental GA (IGA) as the main techniques to evolve the rule set for classification and apply new class acquisition as a typical example to illustrate the incremental, collaborative learning capability of classifier agents. Benchmark data sets are used to evaluate proposed approaches. The results show that GA and IGA can be used successfully for collaborative learning among classifier agents. © 2003 Wiley Periodicals, Inc.     

EVOLVING NEURAL-SYMBOLIC SYSTEMS GUIDED BY ADAPTIVE TRAINING SCHEMES: APPLICATIONS IN FINANCE

The article presents a hybrid and adaptive intelligent methodology, based on neural logic networks and grammar-guided genetic programming. The aim of the study is to demonstrate how to generate efficient neural logic networks with the aid of genetic programming methods trained adaptively through an innovative scheme. The proposed adaptive training scheme of the genetic programming mechanism leads to the generation of high-diversity solutions and small-sized individuals. The overall methodology is advantageous due to the adaptive training scheme proposed for offering both accurate and interpretable results in the form of expert rules. Moreover, a sensitivity analysis study is provided within the article, comparing the performance of the proposed evolutionary neural logic networks methodology with well-known competitive inductive machine learning approaches. Two financial domains of application have been selected to demonstrate the capabilities of the proposed methodology: (a) classification of credit applicants for consumer loans of a German bank and (b) the credit-scoring decision-making process in an Australian bank. Results seem encouraging since the proposed methodology outperforms a number of competitive existing statistical and intelligent methodologies, while it also produces handy decision rules, short in length and transparent in meaning and use.     

Knowledge engineering

Summary Several new and emerging computational paradigms for engineering problem solving automation and processing of various types of knowledge are reviewed. They are artificial intelligence/sybmolic processing, object-oriented programming paradigm, machine learning, evolutionary computing/genetic algorithms, and neurocomputing. Examples of recent research efforts in each area are briefly described.     

基于模糊进化规划和分层方法的神经网络设计方法

本文提出一种模糊进化规划，用于前向神经网络的设计．该方法通过对神经元的部分解群体的进化，缩短了个体的编码长度，显著地减轻了计算量，同时这种方法不但能够在很大程度上简化适应值的计算，更重要的是能够降低适应值空间的复杂性，从而能够加速进化算法收敛到全局最优点．仿真结果显示，本文提出的算法能够有效抑制进化规划算法初期收敛的发生，有效地提高多层前向神经网络收敛精度，并可获得更为简洁的网络结构．