Application of temporal difference learning rules in short‐term traffic flow prediction

Studying dynamic behaviours of a transportation system requires the use of the system mathematical models as well as prediction of traffic flow in the system. Therefore, traffic flow prediction plays an important role in today's intelligent transportation systems. This article introduces a new approach to short‐term daily traffic flow prediction based on artificial neural networks. Among the family of neural networks, multi‐layer perceptron (MLP), radial basis function (RBF) neural network and wavenets have been selected as the three best candidates for performing traffic flow prediction. Moreover, back‐propagation (BP) has been adapted as the most efficient learning scheme in all the cases. It is shown that the coefficients produced by temporal signals improve the performance of the BP learning (BPL) algorithm. Temporal signals provide researchers with a new model of temporal difference BP learning algorithm (TDBPL). The capability and performance of TDBPL algorithm are examined by means of simulation in order to prove that the wavelet theory, with its multi‐resolution ability in comparison to RBF neural networks, is a suitable algorithm in traffic flow forecasting. It is also concluded that despite MLP applications, RBF neural networks do not provide negative forecasts. In addition, the local minimum problems are inevitable in MLP algorithms, while RBF neural networks and wavenet networks do not encounter them.     

深度学习应用技术研究

本文针对深度学习应用技术进行了研究性综述。详细阐述了RBM（Restricted Boltzmann Machine）逐层预训练后再用BP（back-propagation）微调的深度学习贪婪层训练方法,对比分析了BP算法中三种梯度下降的方式,建议在线学习系统,采用随机梯度下降,静态离线学习系统采用随机小批量梯度下降;归纳总结了深度学习深层结构特征,并推荐了目前最受欢迎的5层深度网络结构设计方法。分析了前馈神经网络非线性激活函数的必要性及常用的激活函数优点,并推荐ReLU （rectified linear units）激活函数。最后简要概括了深度CNNs(Convolutional Neural Networks), 深度RNNs(recurrent neural networks), LSTM(long short-termmemory networks)等新型深度网络的特点及应用场景,并归纳总结了当前深度学习可能的发展方向。     

多层感知器的一种快速网络训练法及其应用

从多层感知器原理分析出发,提出一种自适应学习速率因子方法,用于对多层感知器中ＢＰ算法的改进,并将改进算法用于ＸＯＲ问题的学习及某分类器实例样本的学习。仿真结果表明,改进的ＢＰ算法可显著加速网络训练速度,学习过程具有较好的收敛性和较强的鲁棒性。     

神经网络中克服局部最小的BP—EP混合算法

人工神经网络在很多领域有着成功的应用,神经网络有许多学习算法,BP算法是前向多层神经网络的典型算法,但BP算法有时会陷入局部最小解,进化规划（EP）是一种随机优化技术,它可以发现全局成解,当网络学习过程陷入局部最小时,利用EP确定BP算法中的学习速率,使学习过程逸出局部最小,结合具体例子给出了算法实现的具体操作步骤和实验结果。     

人工神经网络BP算法的改进及其在无损检测中的应用

采用多层感知器（MLP）与误差反向传播算法（er-ror back-propagation algorithm)构造与监督训练人工祖辈 经网络,采用了改进的非线性激励函数与学习率的误差反向传播算法,超声无损检测的计算机模拟与实验结果表明,改进的BP算法收敛速度较之BP算法明显加快。     

Extreme learning machine for interval neural networks

Interval data offer a valuable way of representing the available information in complex problems where uncertainty, inaccuracy, or variability must be taken into account. Considered in this paper is the learning of interval neural networks, of which the input and output are vectors with interval components, and the weights are real numbers. The back-propagation (BP) learning algorithm is very slow for interval neural networks, just as for usual real-valued neural networks. Extreme learning machine (ELM) has faster learning speed than the BP algorithm. In this paper, ELM is applied for learning of interval neural networks, resulting in an interval extreme learning machine (IELM). There are two steps in the ELM for usual feedforward neural networks. The first step is to randomly generate the weights connecting the input and the hidden layers, and the second step is to use the Moore–Penrose generalized inversely to determine the weights connecting the hidden and output layers. The first step can be directly applied for interval neural networks. But the second step cannot, due to the involvement of nonlinear constraint conditions for IELM. Instead, we use the same idea as that of the BP algorithm to form a nonlinear optimization problem to determine the weights connecting the hidden and output layers of IELM. Numerical experiments show that IELM is much faster than the usual BP algorithm. And the generalization performance of IELM is much better than that of BP, while the training error of IELM is a little bit worse than that of BP, implying that there might be an over-fitting for BP.     

前向神经网络参数估计中的进化规划

人工神经网络在很多领域有着成功的应用。神经网络参数估计有许多训练算法,BP算法是前向多层神经网络的典型算法,但BP算法有时会陷入局部最小解。进化规划是一种随机优化技术,它可以发现全局最优解。文章介绍了进化规划在前向多层神经网络参数估计中的应用,结合具体例子给出了算法实现的具体操作步骤和实验结果。实验数据表明采用进化规划得到的网络参数是最优的,神经网络的性能优于基于BP算法的神经网络性能。     

Linear-least-squares initialization of multilayer perceptrons through backpropagation of the desired response

Training multilayer neural networks is typically carried out using descent techniques such as the gradient-based backpropagation (BP) of error or the quasi-Newton approaches including the Levenberg-Marquardt algorithm. This is basically due to the fact that there are no analytical methods to find the optimal weights, so iterative local or global optimization techniques are necessary. The success of iterative optimization procedures is strictly dependent on the initial conditions, therefore, in this paper, we devise a principled novel method of backpropagating the desired response through the layers of a multilayer perceptron (MLP), which enables us to accurately initialize these neural networks in the minimum mean-square-error sense, using the analytic linear least squares solution. The generated solution can be used as an initial condition to standard iterative optimization algorithms. However, simulations demonstrate that in most cases, the performance achieved through the proposed initialization scheme leaves little room for further improvement in the mean-square-error (MSE) over the training set. In addition, the performance of the network optimized with the proposed approach also generalizes well to testing data. A rigorous derivation of the initialization algorithm is presented and its high performance is verified with a number of benchmark training problems including chaotic time-series prediction, classification, and nonlinear system identification with MLPs.     

On-line learning algorithms for locally recurrent neural networks

This paper focuses on online learning procedures for locally recurrent neural nets with emphasis on multilayer perceptron (MLP) with infinite impulse response (IIR) synapses and its variations which include generalized output and activation feedback multilayer networks (MLN). We propose a new gradient-based procedure called recursive backpropagation (RBP) whose online version, causal recursive backpropagation (CRBP), has some advantages over other online methods. CRBP includes as particular cases backpropagation (BP), temporal BP, Back-Tsoi algorithm (1991) among others, thereby providing a unifying view on gradient calculation for recurrent nets with local feedback. The only learning method known for locally recurrent nets with no architectural restriction is the one by Back and Tsoi. The proposed algorithm has better stability and faster convergence with respect to the Back-Tsoi algorithm. The computational complexity of the CRBP is comparable with that of the Back-Tsoi algorithm, e.g., less that a factor of 1.5 for usual architectures and parameter settings. The superior performance of the new algorithm, however, easily justifies this small increase in computational burden. In addition, the general paradigms of truncated BPTT and RTRL are applied to networks with local feedback and compared with CRBP. CRBP exhibits similar performances and the detailed analysis of complexity reveals that CRBP is much simpler and easier to implement, e.g., CRBP is local in space and in time while RTRL is not local in space.     

Extraction of Logical Rules from Neural Networks

Three neural-based methods for extraction of logical rules from data are presented. These methods facilitate conversion of graded response neural networks into networks performing logical functions. MLP2LN method tries to convert a standard MLP into a network performing logical operations (LN). C-MLP2LN is a constructive algorithm creating such MLP networks. Logical interpretation is assured by adding constraints to the cost function, forcing the weights to ±1 or 0. Skeletal networks emerge ensuring that a minimal number of logical rules are found. In both methods rules covering many training examples are generated before more specific rules covering exceptions. The third method, FSM2LN, is based on the probability density estimation. Several examples of performance of these methods are presented.     

A successive projection method for binary pattern recognition with multilayer feedforward neural networks

Error back-propagation (BP) is one of the most popular ideas used in learning algorithms for multilayer neural networks. In BP algorithms, there are two types of learning schemes, online learning and batch learning. The online BP has been applied to various problems in practice, because of its simplicity of implementation. However, efficient implementation of the online BP usually requires an ad hoc rule for determining the learning rate of the algorithm. In this paper, we propose a new learning algorithm called SPM, which is derived from the successive projection method for solving a system of nonlinear inequalities. Although SPM can be regarded as a modification of online BP, the former algorithm determines the learning rate (step-size) adoptively based on the output for each input pattern. SPM may also be considered a modification of the globally guided back-propagation (GGBP) proposed by Tang and Koehler. Although no theoretical proof of the convergence for SPM is given, some simulation results on pattern classification problems indicate that SPM is more effective and robust than the standard online BP and GGBP     

连续搅拌反应釜( CSTR)的神经网络动态建模

基于广义性能指标,提出一种神经网络学习算法－广义递推预报误差学习算法（ＧＲＰＥ）,该算法具有二阶收敛阶次。同时讨论了学习速率的选择问题,利用所提出方法对ＣＳＴＲ动态建模结果表明,基于ＧＲＰＥ训练的ＤＲＮＮ比基于ＢＰ训练的ＭＬＰ模型精度高,收敛速度快。     

On-line neural training algorithm with sliding mode control and adaptive learning rate

This paper presents a new algorithm for on-line artificial neural networks (ANN) training. The network topology is a standard multilayer perceptron (MLP) and the training algorithm is based on the theory of variable structure systems (VSS) and sliding mode control (SMC). The main feature of this novel procedure is the adaptability of the gain (learning rate), which is obtained from sliding mode surface so that system stability is guaranteed.     

Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity

This paper has three main goals: (i) to employ two classes of algorithms: bio-inspired and gradient-based to train multi-layer perceptron (MLP) neural networks for pattern classification; (ii) to combine the trained neural networks into ensembles of classifiers; and (iii) to investigate the influence of diversity in the classification performance of individual and ensembles of classifiers. The optimization version of an artificial immune network, named opt-aiNet, particle swarm optimization (PSO) and an evolutionary algorithm (EA) are used as bio-inspired methods to train MLP networks. Besides, the standard backpropagation with momentum (BPM), a quasi-Newton method called DFP and a modified scaled-conjugate gradient (SCGM) are the gradient-based algorithms used to train MLP networks in this work. Comparisons among all the training methods are presented in terms of classification accuracy and diversity of the solutions found. The results obtained suggest that most bio-inspired algorithms deteriorate the diversity of solutions during the search, while immune-based methods, like opt-aiNet, and multiple initializations of standard gradient-based algorithms provide diverse solutions that result in good classification accuracy for the ensembles.     

ANASA-a stochastic reinforcement algorithm for real-valued neuralcomputation

This paper introduces ANASA (adaptive neural algorithm of stochastic activation), a new, efficient, reinforcement learning algorithm for training neural units and networks with continuous output. The proposed method employs concepts, found in self-organizing neural networks theory and in reinforcement estimator learning algorithms, to extract and exploit information relative to previous input pattern presentations. In addition, it uses an adaptive learning rate function and a self-adjusting stochastic activation to accelerate the learning process. A form of optimal performance of the ANASA algorithm is proved (under a set of assumptions) via strong convergence theorems and concepts. Experimentally, the new algorithm yields results, which are superior compared to existing associative reinforcement learning methods in terms of accuracy and convergence rates. The rapid convergence rate of ANASA is demonstrated in a simple learning task, when it is used as a single neural unit, and in mathematical function modeling problems, when it is used to train various multilayered neural networks.     

Nonlinear dynamic system identification using Chebyshev functionallink artificial neural networks

A computationally efficient artificial neural network (ANN) for the purpose of dynamic nonlinear system identification is proposed. The major drawback of feedforward neural networks, such as multilayer perceptrons (MLPs) trained with the backpropagation (BP) algorithm, is that they require a large amount of computation for learning. We propose a single-layer functional-link ANN (FLANN) in which the need for a hidden layer is eliminated by expanding the input pattern by Chebyshev polynomials. The novelty of this network is that it requires much less computation than that of a MLP. We have shown its effectiveness in the problem of nonlinear dynamic system identification. In the presence of additive Gaussian noise, the performance of the proposed network is found to be similar or superior to that of a MLP. A performance comparison in terms of computational complexity has also been carried out.     

Extraction of voltage harmonics using multi-layer perceptron neural network

This paper presents a harmonic extraction algorithm using artificial neural networks for Dynamic Voltage Restorers (DVRs). The suggested algorithm employs a feed forward Multi Layer Perceptron (MLP) Neural Network with error back propagation learning to effectively track and extract the 3rd and 5th voltage harmonics. For this purpose, two different MLP neural network structures are constructed and their performances compared. The effects of hidden layer, supervisors and learning rate are also presented. The proposed MLP Neural Network algorithm is trained and tested in MATLAB program environment. The results show that MLP neural network enable to extract each harmonic effectively.     

Handwritten characters recognition based on nature-inspired computing and neuro-evolution

The enormous services obtainable by bank and postal systems are not 100 % guaranteed due to variability of handwriting styles. Various methods based on neural networks have been suggested to address this issue. Unfortunately, they often fall into local optima that arises from the use of old learning methods. Global optimization methods provided new directions for neural networks evolution that may be useful in recognition. This paper develops efficient algorithms that compute globally optimal solutions by exploiting the benefits of both swarm intelligence and neuro-evolution in a way to improve the overall performance of a character recognition system. Various adaptations implied to both MLP and RBF networks have been suggested namely: particle swarm optimization (PSO) and the bees algorithm (BA) for characters classification, MLP training or RBF design by co-evolution and effective combinations of MLPs, RBFs or SVMs as an attempt to overcome the drawbacks of old recognition methods. Results proved that networks combination proposals ensure the highest improvement compared to either standard MLP and RBF networks, the co-evolutionary alternatives or other classifiers combination based on common combination rules namely majority voting, the fusion rules of min, max, sum, average, product and Bayes, Decision template and the Behavior Knowledge Space (BKS).     

A robust recurrent simultaneous perturbation stochastic approximation training algorithm for recurrent neural networks

Training of recurrent neural networks (RNNs) introduces considerable computational complexities due to the need for gradient evaluations. How to get fast convergence speed and low computational complexity remains a challenging and open topic. Besides, the transient response of learning process of RNNs is a critical issue, especially for online applications. Conventional RNN training algorithms such as the backpropagation through time and real-time recurrent learning have not adequately satisfied these requirements because they often suffer from slow convergence speed. If a large learning rate is chosen to improve performance, the training process may become unstable in terms of weight divergence. In this paper, a novel training algorithm of RNN, named robust recurrent simultaneous perturbation stochastic approximation (RRSPSA), is developed with a specially designed recurrent hybrid adaptive parameter and adaptive learning rates. RRSPSA is a powerful novel twin-engine simultaneous perturbation stochastic approximation (SPSA) type of RNN training algorithm. It utilizes three specially designed adaptive parameters to maximize training speed for a recurrent training signal while exhibiting certain weight convergence properties with only two objective function measurements as the original SPSA algorithm. The RRSPSA is proved with guaranteed weight convergence and system stability in the sense of Lyapunov function. Computer simulations were carried out to demonstrate applicability of the theoretical results.     

Using the EM algorithm to train neural networks: misconceptions and a new algorithm for multiclass classification

The expectation-maximization (EM) algorithm has been of considerable interest in recent years as the basis for various algorithms in application areas of neural networks such as pattern recognition. However, there exists some misconceptions concerning its application to neural networks. In this paper, we clarify these misconceptions and consider how the EM algorithm can be adopted to train multilayer perceptron (MLP) and mixture of experts (ME) networks in applications to multiclass classification. We identify some situations where the application of the EM algorithm to train MLP networks may be of limited value and discuss some ways of handling the difficulties. For ME networks, it is reported in the literature that networks trained by the EM algorithm using iteratively reweighted least squares (IRLS) algorithm in the inner loop of the M-step, often performed poorly in multiclass classification. However, we found that the convergence of the IRLS algorithm is stable and that the log likelihood is monotonic increasing when a learning rate smaller than one is adopted. Also, we propose the use of an expectation-conditional maximization (ECM) algorithm to train ME networks. Its performance is demonstrated to be superior to the IRLS algorithm on some simulated and real data sets.