Time series prediction with single multiplicative neuron model

Single neuron models are typical functional replica of the biological neuron that are derived using their individual and group responses in networks. In recent past, a lot of work in this area has produced advanced neuron models for both analog and binary data patterns. Popular among these are the higher-order neurons, fuzzy neurons and other polynomial neurons. In this paper, we propose a new neuron model based on a polynomial architecture. Instead of considering all the higher-order terms, a simple aggregation function is used. The aggregation function is considered as a product of linear functions in different dimensions of the space. The functional mapping capability of the proposed neuron model is demonstrated through some well known time series prediction problems and is compared with the standard multilayer neural network.     

Hybrid integrate-and-fire model of a bursting neuron

We present a reduction of a Hodgkin-Huxley (HH)--style bursting model to a hybridized integrate-and-fire (IF) formalism based on a thorough bifurcation analysis of the neuron's dynamics. The model incorporates HH--style equations to evolve the subthreshold currents and includes IF mechanisms to characterize spike events and mediate interactions between the subthreshold and spiking currents. The hybrid IF model successfully reproduces the dynamic behavior and temporal characteristics of the full model over a wide range of activity, including bursting and tonic firing. Comparisons of timed computer simulations of the reduced model and the original model for both single neurons and moderately sized networks (n < or = 500) show that this model offers improvement in computational speed over the HH--style bursting model.     

Firing rate of the noisy quadratic integrate-and-fire neuron

We calculate the firing rate of the quadratic integrate-and-fire neuron in response to a colored noise input current. Such an input current is a good approximation to the noise due to the random bombardment of spikes, with the correlation time of the noise corresponding to the decay time of the synapses. The key parameter that determines the firing rate is the ratio of the correlation time of the colored noise, tau(s), to the neuronal time constant, tau(m). We calculate the firing rate exactly in two limits: when the ratio, tau(s)/tau(m), goes to zero (white noise) and when it goes to infinity. The correction to the short correlation time limit is O(tau(s)/tau(m)), which is qualita tively different from that of the leaky integrate-and-fire neuron, where the correction is O( radical tau(s)/tau(m)). The difference is due to the different boundary conditions of the probability density function of the membrane potential of the neuron at firing threshold. The correction to the long correlation time limit is O(tau(m)/tau(s)). By combining the short and long correlation time limits, we derive an expression that provides a good approximation to the firing rate over the whole range of tau(s)/tau(m) in the suprathreshold regime-that is, in a regime in which the average current is sufficient to make the cell fire. In the subthreshold regime, the expression breaks down somewhat when tau(s) becomes large compared to tau(m).     

Analytical integrate-and-fire neuron models with conductance-based dynamics for event-driven simulation strategies

Event-driven simulation strategies were proposed recently to simulate integrate-and-fire (IF) type neuronal models. These strategies can lead to computationally efficient algorithms for simulating large-scale networks of neurons; most important, such approaches are more precise than traditional clock-driven numerical integration approaches because the timing of spikes is treated exactly. The drawback of such event-driven methods is that in order to be efficient, the membrane equations must be solvable analytically, or at least provide simple analytic approximations for the state variables describing the system. This requirement prevents, in general, the use of conductance-based synaptic interactions within the framework of event-driven simulations and, thus, the investigation of network paradigms where synaptic conductances are important. We propose here a number of extensions of the classical leaky IF neuron model involving approximations of the membrane equation with conductance-based synaptic current, which lead to simple analytic expressions for the membrane state, and therefore can be used in the event-driven framework. These conductance-based IF (gIF) models are compared to commonly used models, such as the leaky IF model or biophysical models in which conductances are explicitly integrated. All models are compared with respect to various spiking response properties in the presence of synaptic activity, such as the spontaneous discharge statistics, the temporal precision in resolving synaptic inputs, and gain modulation under in vivo-like synaptic bombardment. Being based on the passive membrane equation with fixed-threshold spike generation, the proposed gIF models are situated in between leaky IF and biophysical models but are much closer to the latter with respect to their dynamic behavior and response characteristics, while still being nearly as computationally efficient as simple IF neuron models. gIF models should therefore provide a useful tool for efficient and precise simulation of large-scale neuronal networks with realistic, conductance-based synaptic interactions.     

PSO-based single multiplicative neuron model for time series prediction

Single multiplicative neuron model is a novel neural network model introduced recently, which has been used for time series prediction and function approximation. The model is based on a polynomial architecture that is the product of linear functions in different dimensions of the space. Particle swarm optimization (PSO), a global optimization method, is proposed to train the single neuron model in this paper. An improved version of the original PSO, cooperative random learning particle swarm optimization (CRPSO), is put forward to enhance the performance of the conventional PSO. The proposed CRPSO, PSO, back-propagation algorithm and genetic algorithm are employed to train the model for three well-known time series prediction problems. The experimental results demonstrate the superiority of CRPSO-based neuron model in efficiency and robustness over the other three algorithms.     

Time-series prediction using adaptive neuro-fuzzy networks

In this paper, we propose an Adaptive Neuro-Fuzzy Network (ANFN) to deal with forecasting problems. The ANFN model is inherently a modified Takagi–Sugeno–Kang-type fuzzy-rule-based model possessing a neural network's learning ability. We propose a hybrid learning algorithm which combines the Genetic Algorithm (GA) and the Least-Squares Estimate (LSE) method to construct the ANFN model. The GA is used to tune membership functions at the precondition part of fuzzy rules, while the LSE method is used to tune parameters at the consequent part of fuzzy rules. Simulations demonstrate that the proposed ANFN model has a good predictive capability.     

Time-series prediction based on pattern classification

In this paper, a new time-series predication method is proposed based on pattern analysis. In this method, basic patterns and their probabilities are extracted from a time series. A probabilistic relaxation method is employed to classify the probability vectors of the basic patterns. In order to verify the effectiveness of the proposed method, several experiments are carried out on a simulation signal and real data. The results show that the proposed method has advantages over existing methods in some applications.     

航空发动机的单神经元双变量解耦控制

针对航空发动机这样的多变量控制对象,要解决的突出问题是输入变量对输出变量的交叉影响,介绍了单神经元进行多变量系统解耦控制的基本方法,采用改进的Hebb学习算法以加速收敛。对某涡喷发动机的数学模型进行了双变量单神经元PID控制仿真研究,结果表明：采用此算法构成的神经网络PID控制对地面模型和高空模型都具有完全解耦、响应速度快、稳态误差小、算法简单的优点;用两个神经元作为双变量控制器,可以使整个飞行包线内的控制器数目明显减少。     

Analytical integrate-and-fire neuron models with conductance-based dynamics and realistic postsynaptic potential time course for event-driven simulation strategies

In a previous paper (Rudolph & Destexhe, 2006), we proposed various models, the gIF neuron models, of analytical integrate-and-fire (IF) neurons with conductance-based (COBA) dynamics for use in event-driven simulations. These models are based on an analytical approximation of the differential equation describing the IF neuron with exponential synaptic conductances and were successfully tested with respect to their response to random and oscillating inputs. Because they are analytical and mathematically simple, the gIF models are best suited for fast event-driven simulation strategies. However, the drawback of such models is they rely on a nonrealistic postsynaptic potential (PSP) time course, consisting of a discontinuous jump followed by a decay governed by the membrane time constant. Here, we address this limitation by conceiving an analytical approximation of the COBA IF neuron model with the full PSP time course. The subthreshold and suprathreshold response of this gIF4 model reproduces remarkably well the postsynaptic responses of the numerically solved passive membrane equation subject to conductance noise, while gaining at least two orders of magnitude in computational performance. Although the analytical structure of the gIF4 model is more complex than that of its predecessors due to the necessity of calculating future spike times, a simple and fast algorithmic implementation for use in large-scale neural network simulations is proposed.     

基于构造性神经网络的时间序列混合预测模型 *

针对传统时间序列预测模型不适应非线性预测而适应非线性预测的BP算法存在收敛速度慢,且容易陷入局部极小等问题,提出一种基于构造性神经网络的时间序列混合预测模型。采用构造性神经网络模型（覆盖算法）得出的类别值对统计时间序列模型的预测值进行修正,建立一种同时考虑时间序列自身周期变化和外生变量因子对时间序列未来变化趋势影响的混合预测模型,涵盖了实际问题的线性和非线性两方面,提高了预测精度。将该模型应用到粮食产量的预测中,取得了较好的预测效果。     

基于构造性神经网络的时间序列混合预测模型 *

针对传统时间序列预测模型不适应非线性预测而适应非线性预测的 BP算法存在收敛速度慢 ,且容易陷入局部极小等问题 ,提出一种基于构造性神经网络的时间序列混合预测模型。采用构造性神经网络模型 (覆盖算法 )得出的类别值对统计时间序列模型的预测值进行修正 ,建立一种同时考虑时间序列自身周期变化和外生变量因子对时间序列未来变化趋势影响的混合预测模型 ,涵盖了实际问题的线性和非线性两方面 ,提高了预测精度。将该模型应用到粮食产量的预测中 ,取得了较好的预测效果。     

Time-series prediction of hourly atmospheric pressure using ANFIS and LSTM approaches

Neural Computing and Applications - Atmospheric pressure (AP), which is an indicator of weather events, plays an important role in climatology, agriculture, meteorology, atmospheric and...     

改进单神经元自适应PID控制器应用于污水处理

结合无需辨识的自适应控制算法,提出一种动态调整增益系数和自适应学习率的改进型单神经元PID控制策略,进一步提高了控制器参数的自校正能力.利用ActiveX技术将改进型单神经元自适应PID控制算法封装在ActiveX控件中,并设计MFC应用程序对污水处理过程溶解氧的控制进行仿真.结果表明,改进型单神经元PID与改进前的单神经元PID控制方法相比,具有更好的自适应性和更强的鲁棒性.     

单神经元自适应PID控制器的性能优化设计

研究了单神经元自适应PID摔制器性能优化问题,阐述了该摔制器的特点、控制律;给出了一种控制灵敏度的快速近似求取方法,实现了PID参数的在线自学习;使单神经元控制器具有可调参数少、易于整定、控制输出平稳、鲁棒件强的独特优点,适用于大滞后且要求平稳控制输出的工业过程。     

Training a single sigmoidal neuron is hard

We first present a brief survey of hardness results for training feedforward neural networks. These results are then completed by the proof that the simplest architecture containing only a single neuron that applies a sigmoidal activation function sigma: kappa --> [alpha, beta], satisfying certain natural axioms (e.g., the standard (logistic) sigmoid or saturated-linear function), to the weighted sum of n inputs is hard to train. In particular, the problem of finding the weights of such a unit that minimize the quadratic training error within (beta - alpha)(2) or its average (over a training set) within 5(beta - alpha)(2)/ (12n) of its infimum proves to be NP-hard. Hence, the well-known backpropagation learning algorithm appears not to be efficient even for one neuron, which has negative consequences in constructive learning.     

Training integrate-and-fire neurons with the Informax principle II

For pt I see J. Phys. A, vol. 35, p. 2379-94 (2002).We develop neuron learning rules using the Informax principle together with the input-output relationship of the integrate-and-fire (IF) model with Poisson inputs. The learning rule is then tested with constant inputs, time-varying inputs and images. For constant inputs, it is found that, under the Informax principle, a network of IF models with initially all positive weights tends to disconnect some connections between neurons. For time-varying inputs and images, we perform signal separation tasks called independent component analysis. Numerical simulations indicate that some number of inhibitory inputs improves the performance of the system in both biological and engineering senses.     

Exact simulation of integrate-and-fire models with exponential currents

Neural networks can be simulated exactly using event-driven strategies, in which the algorithm advances directly from one spike to the next spike. It applies to neuron models for which we have (1) an explicit expression for the evolution of the state variables between spikes and (2) an explicit test on the state variables that predicts whether and when a spike will be emitted. In a previous work, we proposed a method that allows exact simulation of an integrate-and-fire model with exponential conductances, with the constraint of a single synaptic time constant. In this note, we propose a method, based on polynomial root finding, that applies to integrate-and-fire models with exponential currents, with possibly many different synaptic time constants. Models can include biexponential synaptic currents and spike-triggered adaptation currents.     

Exact simulation of integrate-and-fire models with synaptic conductances

Computational neuroscience relies heavily on the simulation of large networks of neuron models. There are essentially two simulation strategies: (1) using an approximation method (e.g., Runge-Kutta) with spike times binned to the time step and (2) calculating spike times exactly in an event-driven fashion. In large networks, the computation time of the best algorithm for either strategy scales linearly with the number of synapses, but each strategy has its own assets and constraints: approximation methods can be applied to any model but are inexact; exact simulation avoids numerical artifacts but is limited to simple models. Previous work has focused on improving the accuracy of approximation methods. In this article, we extend the range of models that can be simulated exactly to a more realistic model: an integrate-and-fire model with exponential synaptic conductances.     

Logic operations based on single neuron rational model

This paper focuses on phase analysis to explore the single neuron local arithmetic and logic operations on their input conductances. Based on the analysis of the rational function model of local spatial summation with the equivalent circuits for steady-state membrane potentials, the prototypes spatial summation with the equivalent circuits for steady-state membrane potentials, the prototypes of logic operations are constructed. A mapping from a partition of input conductance space into functionally distinct phases is described and the multiple mode models for logic operations are established. The transitions from output voltage to input conductance in logic operations are also discussed for the connections between neurons in different layers. Our theoretical studies and software simulations indicate that the single neuron local rational logic is programmable and the selection of these functional phases can be effectively instructed by presynaptic activities. This programmability makes the single neuron more flexible in processing the input information.     

On the nonlearnability of a single spiking neuron

We study the computational complexity of training a single spiking neuron N with binary coded inputs and output that, in addition to adaptive weights and a threshold, has adjustable synaptic delays. A synchronization technique is introduced so that the results concerning the nonlearnability of spiking neurons with binary delays are generalized to arbitrary real-valued delays. In particular, the consistency problem for N with programmable weights, a threshold, and delays, and its approximation version are proven to be NP-complete. It follows that the spiking neurons with arbitrary synaptic delays are not properly PAC learnable and do not allow robust learning unless RP = NP. In addition, the representation problem for N, a question whether an n-variable Boolean function given in DNF (or as a disjunction of O(n) threshold gates) can be computed by a spiking neuron, is shown to be coNP-hard.