A simulation study of artificial neural networks for nonlinear time-series forecasting

This study presents an experimental evaluation of neural networks for nonlinear time-series forecasting. The effects of three main factors — input nodes, hidden nodes and sample size, are examined through a simulated computer experiment. Results show that neural networks are valuable tools for modeling and forecasting nonlinear time series while traditional linear methods are not as competent for this task. The number of input nodes is much more important than the number of hidden nodes in neural network model building for forecasting. Moreover, large sample is helpful to ease the overfitting problem.Scope and purposeInterest in using artificial neural networks for forecasting has led to a tremendous surge in research activities in the past decade. Yet, mixed results are often reported in the literature and the effect of key modeling factors on performance has not been thoroughly examined. The lack of systematic approaches to neural network model building is probably the primary cause of inconsistencies in reported findings. In this paper, we present a systematic investigation of the application of neural networks for nonlinear time-series analysis and forecasting. The purpose is to have a detailed examination of the effects of certain important neural network modeling factors on nonlinear time-series modeling and forecasting.     

An empirical investigation of bias and variance in time series forecasting: modeling considerations and error evaluation

Bias and variance play an important role in understanding the fundamental issue of learning and generalization in neural network modeling. Several studies on bias and variance effects have been published in classification and regression related research of neural networks. However, little research has been done in this area for time-series modeling and forecasting. We consider modeling issues related to understanding error components given the common practices associated with neural-network time-series forecasting. We point out the key difference between classification and time-series problems in consideration of the bias-plus-variance decomposition. A Monte Carlo study on the role of bias and variance in neural networks time-series forecasting is conducted. We find that both input lag structure and hidden nodes are important in contributing to the overall forecasting performance. The results also suggest that overspecification of input nodes in neural network modeling does not impact the model bias, but has significant effect on the model variance. Methods such as neural ensembles that focus on reducing the model variance, therefore, can be valuable and effective in time-series forecasting modeling.     

HYBRID GREY RELATIONAL ARTIFICIAL NEURAL NETWORK AND AUTO REGRESSIVE INTEGRATED MOVING AVERAGE MODEL FOR FORECASTING TIME-SERIES DATA

The aim of this study is to develop a new hybrid model by combining a linear and nonlinear model for forecasting time-series data. The proposed model (GRANN_ARIMA) integrates nonlinear grey relational artificial neural network (GRANN) and a linear autoregressive integrated moving average (ARIMA) model by combining new features and grey relational analysis to select the appropriate inputs and hybridization succession. To validate the performance of the proposed model, small and large scale data sets are used. The forecasting performance is compared with several models, and these include: individual models (ARIMA, multiple regression, GRANN), several hybrid models (MARMA, MR_ANN, ARIMA_ANN), and an artificial neural network (ANN) trained using a Levenberg Marquardt algorithm. The experiments have shown that the proposed model has outperformed other models with 99.5% forecasting accuracy for small-scale data and 99.84% for large-scale data. The obtained empirical results have proven that the GRANN_ARIMA model can provide a better alternative for time-series forecasting due to its promising performance and capability in handling time-series data for both small- and large-scale data.     

基于自适应门控图神经网络的交通流预测

交通流预测是智能交通系统中的重要组成部分,由于交通数据的复杂性,长期而又准确的交通流预测一直是时间序列预测中最具挑战性的任务之一。近年来,研究人员将基于图神经网络的时空图建模方法应用于交通流预测任务,并取得了良好的预测性能。然而,现有的图建模方法仅通过预定义的邻接结构反映道路网络中的空间依赖关系,忽略了各节点之间的序列关联关系对预测的重要性。针对这一局限性,提出了一种自适应门控图神经网络（Ada-GGNN）,其核心为通过空间传递模块同时捕获道路网络的空间结构及自适应的时序相关性,并通过门控机制学习节点上的时间序列特征。在两个真实交通网络数据集PeMSD7和Los-loop上的实验结果证明了该模型具有更优越的性能。     

Trend time-series modeling and forecasting with neural networks.

Despite its great importance, there has been no general consensus on how to model the trends in time-series data. Compared to traditional approaches, neural networks (NNs) have shown some promise in time-series forecasting. This paper investigates how to best model trend time series using NNs. Four different strategies (raw data, raw data with time index, detrending, and differencing) are used to model various trend patterns (linear, nonlinear, deterministic, stochastic, and breaking trend). We find that with NNs differencing often gives meritorious results regardless of the underlying data generating processes (DGPs). This finding is also confirmed by the real gross national product (GNP) series.     

Trend Time–Series Modeling and Forecasting With Neural Networks

Despite its great importance, there has been no general consensus on how to model the trends in time-series data. Compared to traditional approaches, neural networks (NNs) have shown some promise in time-series forecasting. This paper investigates how to best model trend time series using NNs. Four different strategies (raw data, raw data with time index, detrending, and differencing) are used to model various trend patterns (linear, nonlinear, deterministic, stochastic, and breaking trend). We find that with NNs differencing often gives meritorious results regardless of the underlying data generating processes (DGPs). This finding is also confirmed by the real gross national product (GNP) series.     

Prediction of Road Traffic using a Neural Network Approach

A key component of the daily operation and planning activities of a traffic control centre is short-term forecasting, i.e. the prediction of daily to the next few days of traffic flow. Such forecasts have a significant impact on the optimal regulation of the road traffic on all kinds of freeways. They are increasingly important in an environment with increasing road traffic problems. The present paper aims at presenting the effectiveness of a neural network system for prediction based on time-series data. We only use one parameter, namely traffic volume for the forecasting. We employ artificial neural networks for traffic forecasting applied on a road section. Recurrent Jordan networks, popular in the modelling of time series, is examined in this study. Simulation results demonstrate that learning with this type of architecture has a good generalisation ability.     

A hybrid neural network and ARIMA model for water quality time series prediction

Accurate predictions of time series data have motivated the researchers to develop innovative models for water resources management. Time series data often contain both linear and nonlinear patterns. Therefore, neither ARIMA nor neural networks can be adequate in modeling and predicting time series data. The ARIMA model cannot deal with nonlinear relationships while the neural network model alone is not able to handle both linear and nonlinear patterns equally well. In the present study, a hybrid ARIMA and neural network model is proposed that is capable of exploiting the strengths of traditional time series approaches and artificial neural networks. The proposed approach consists of an ARIMA methodology and feed-forward, backpropagation network structure with an optimized conjugated training algorithm. The hybrid approach for time series prediction is tested using 108-month observations of water quality data, including water temperature, boron and dissolved oxygen, during 1996–2004 at Büyük Menderes river, Turkey. Specifically, the results from the hybrid model provide a robust modeling framework capable of capturing the nonlinear nature of the complex time series and thus producing more accurate predictions. The correlation coefficients between the hybrid model predicted values and observed data for boron, dissolved oxygen and water temperature are 0.902, 0.893, and 0.909, respectively, which are satisfactory in common model applications. Predicted water quality data from the hybrid model are compared with those from the ARIMA methodology and neural network architecture using the accuracy measures. Owing to its ability in recognizing time series patterns and nonlinear characteristics, the hybrid model provides much better accuracy over the ARIMA and neural network models for water quality predictions.     

A combination of artificial neural network and random walk models for financial time series forecasting

Properly comprehending and modeling the dynamics of financial data has indispensable practical importance. The prime goal of a financial time series model is to provide reliable future forecasts which are crucial for investment planning, fiscal risk hedging, governmental policy making, etc. These time series often exhibit notoriously haphazard movements which make the task of modeling and forecasting extremely difficult. As per the research evidence, the random walk (RW) is so far the best linear model for forecasting financial data. Artificial neural network (ANN) is another promising alternative with the unique capability of nonlinear self-adaptive modeling. Numerous comparisons of the performances of RW and ANN models have also been carried out in the literature with mixed conclusions. In this paper, we propose a combination methodology which attempts to benefit from the strengths of both RW and ANN models. In our proposed approach, the linear part of a financial dataset is processed through the RW model, and the remaining nonlinear residuals are processed using an ensemble of feedforward ANN (FANN) and Elman ANN (EANN) models. The forecasting ability of the proposed scheme is examined on four real-world financial time series in terms of three popular error statistics. The obtained results clearly demonstrate that our combination method achieves reasonably better forecasting accuracies than each of RW, FANN and EANN models in isolation for all four financial time series.     

人工神经网络在证券价格预测中的应用

证券市场中成功的交易模式是可以模仿及学习的.证券价格走势实质是一种复杂时序函数.人工神经网络是在模仿人脑处理问题过程中发展起来的新型智能信息处理系统,人工神经网络可以通过调节连接权值以任意精度逼近任何连续函数,因此也可以逼近证券价格随时间变换这种函数.文中采用基于BP模型的神经网络,用BP算法和遗传算法来训练网络权值,同时也采用了动量法和学习率自适应调整相结合的策略,对证券市场的价格进行建模和预测,结果表明,此模型具有较好的学习、泛化能力,对股票市场或其他类似的非线性经济系统的走势预测决策具有较好的效果.     

Application of neural networks in forecasting engine systems reliability

This paper presents a comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems. Traditionally, failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions, which are at times difficult to validate. On the other hand, the time series modeling technique using neural networks provides a promising alternative. Neural network modeling via feed-forward multilayer perceptron (MLP) suffers from local minima problems and long computation time. The radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time. Illustrative examples using reliability testing and field data showed that the proposed model results in comparable or better predictive performance than traditional MLP model and the linear benchmark based on Box–Jenkins autoregressive-integrated-moving average (ARIMA) models. The effects of input window size and hidden layer nodes are further investigated. Appropriate design topologies can be determined via sensitivity analysis.     

基于误差分治的神经网络验证

随着神经网络技术的快速发展,其在自动驾驶、智能制造、医疗诊断等安全攸关领域得到了广泛应用,神经网络的可信保障变得至关重要.然而,由于神经网络具有脆弱性,轻微的扰动经常会导致错误的结果,因此采用形式化验证的手段来保障神经网络安全可信是非常重要的.目前神经网络的验证方法主要关注分析的精度,而易忽略运行效率.在验证一些复杂网络的安全性质时,较大规模的状态空间可能会导致验证方法不可行或者无法求解等问题.为了减少神经网络的状态空间,提高验证效率,提出一种基于过近似误差分治的神经网络形式化验证方法.该方法利用可达性分析技术计算非线性节点的上下界,并采用一种改进的符号线性松弛方法减少了非线性节点边界计算过程中的过近似误差.通过计算节点过近似误差的直接和间接影响,将节点的约束进行细化,从而将原始验证问题划分为一组子问题,其混合整数规划(MILP)公式具有较少的约束数量.所提方法已实现为工具NNVerifier,并通过实验在经典的3个数据集上训练的4个基于ReLU的全连接基准网络进行性质验证和评估.实验结果表明, NNVerifier的验证效率比现有的完备验证技术提高了37.18%.     

A hybrid optimized error correction system for time series forecasting

Time series forecasting is a challenging task in machine learning. Real world time series are often composed by linear and nonlinear structures which need to be mapped by some forecasting method. Linear methods such as autoregressive integrated moving average (ARIMA) and nonlinear methods such as artificial neural networks (ANNs) could be employed to handle such problems, however model misspecification hinders the forecasting process producing inaccurate models. Hybrid models based on error forecasting and combination can reduce the misspecification of single models and improve the accuracy of the system. This work proposes a hybrid system that is composed of three parts: a) linear modeling of the time series, b) nonlinear modeling of the error series, and c) combination of the forecasts using three distinct approaches. The system performs a search for the best parameters of the linear and nonlinear components, and of the combination approaches. Particle swarm optimization is used to find suitable architecture and weights. Experiments show that the proposed technique achieved promising results in time series forecasting.     

A rapid learning and dynamic stepwise updating algorithm for flatneural networks and the application to time-series prediction

A fast learning algorithm is proposed to find an optimal weights of the flat neural networks (especially, the functional-link network). Although the flat networks are used for nonlinear function approximation, they can be formulated as linear systems. Thus, the weights of the networks can be solved easily using a linear least-square method. This formulation makes it easier to update the weights instantly for both a new added pattern and a new added enhancement node. A dynamic stepwise updating algorithm is proposed to update the weights of the system on-the-fly. The model is tested on several time-series data including an infrared laser data set, a chaotic time-series, a monthly flour price data set, and a nonlinear system identification problem. The simulation results are compared to existing models in which more complex architectures and more costly training are needed. The results indicate that the proposed model is very attractive to real-time processes.     

Comparison of TSCS regression and neural network models for panel data forecasting: debt policy

Empirical studies of variations in debt ratios across firms have analyzed important determinants of capital structure using statistical models. Researchers, however, rarely employ nonlinear models to examine the determinants and make little effort to identify a superior prediction model among competing ones. This paper reviews the time-series cross-sectional (TSCS) regression and the predictive abilities of neural network (NN) utilizing panel data concerning debt ratio of high-tech industries in Taiwan. We built models with these two methods using the same set of measurements as determinants of debt ratio and compared the forecasting performance of five models, namely, three TSCS regression models and two NN models. Models built with neural network obtained the lowest mean square error and mean absolute error. These results reveal that the relationships between debt ratio and determinants are nonlinear and that NNs are more competent in modeling and forecasting the test panel data. We conclude that NN models can be used to solve panel data analysis and forecasting problems.     

基于周期性建模的时间序列预测方法及电价预测研究

时间序列数据广泛存在于人类的生产生活中, 通常具有复杂的非线性动态和一定的周期性. 与传统的时间序列分析方法相比, 基于深度学习的方法更能捕捉数据的深层特性, 对具有复杂非线性的时间序列有较好的建模效果. 为了在神经网络中显式地建模时间序列数据的周期性和趋势性, 本文在循环神经网络的基础上引入了周期损失和趋势损失, 建立了基于周期性建模和多任务学习的时间序列预测模型. 将模型应用到欧洲能源交易所法国市场的能源市场价格预测中, 结果表明周期损失和趋势损失能够提高神经网络的泛化能力, 并提高预测时间序列趋势的精度.     

Representation of Nonlinear Random Transformations by Non-Gaussian Stochastic Neural Networks

The learning capability of neural networks is equivalent to modeling physical events that occur in the real environment. Several early works have demonstrated that neural networks belonging to some classes are universal approximators of input-output deterministic functions. Recent works extend the ability of neural networks in approximating random functions using a class of networks named stochastic neural networks (SNN). In the language of system theory, the approximation of both deterministic and stochastic functions falls within the identification of nonlinear no-memory systems. However, all the results presented so far are restricted to the case of Gaussian stochastic processes (SPs) only, or to linear transformations that guarantee this property. This paper aims at investigating the ability of stochastic neural networks to approximate nonlinear input-output random transformations, thus widening the range of applicability of these networks to nonlinear systems with memory. In particular, this study shows that networks belonging to a class named non-Gaussian stochastic approximate identity neural networks (SAINNs) are capable of approximating the solutions of large classes of nonlinear random ordinary differential transformations. The effectiveness of this approach is demonstrated and discussed by some application examples.     

Constructive neural networks with piecewise interpolationcapabilities for function approximations

This paper proposes a constructive neural network with a piecewise linear or nonlinear local interpolation capability to approximate arbitrary continuous functions. This neural network is devised by introducing a space tessellation which is a covering of the Euclidean space by nonoverlapping hyperpolyhedral convex cells. In the proposed neural network, a number of neural network granules (NNG's) are processed in parallel and repeated regularly with the same structures. Each NNG does a local mapping with an interpolation capability for a corresponding hyperpolyhedral convex cell in a tessellation. The plastic weights of the NNG can be calculated to implement the mapping for training data; consequently, this reduces training time and alleviates the difficulties of local minima in training. In addition, the interpolation capability of the NNG improves the generalization for the new data within the convex cell. The proposed network requires additional neurons for tessellation over the standard multilayer neural networks. This increases the network size but does not slow the retrieval response when implemented by parallel architecture.     

Type-2 fuzzy neural networks with fuzzy clustering and differential evolution optimization

In many real-world problems involving pattern recognition, system identification and modeling, control, decision making, and forecasting of time-series, available data are quite often of uncertain nature. An interesting alternative is to employ type-2 fuzzy sets, which augment fuzzy models with expressive power to develop models, which efficiently capture the factor of uncertainty. The three-dimensional membership functions of type-2 fuzzy sets offer additional degrees of freedom that make it possible to directly and more effectively account for model’s uncertainties. Type-2 fuzzy logic systems developed with the aid of evolutionary optimization forms a useful modeling tool subsequently resulting in a collection of efficient “If-Then” rules.The type-2 fuzzy neural networks take advantage of capabilities of fuzzy clustering by generating type-2 fuzzy rule base, resulting in a small number of rules and then optimizing membership functions of type-2 fuzzy sets present in the antecedent and consequent parts of the rules. The clustering itself is realized with the aid of differential evolution.Several examples, including a benchmark problem of identification of nonlinear system, are considered. The reported comparative analysis of experimental results is used to quantify the performance of the developed networks.     

股票价格预测的建模与仿真研究

研究股票价格准确预测问题,由于股票价格数据具非线性、随机性等变化规律,同时股票市场与国内外经济政治变化有关,传统股票价格预测方法只能对其线性变化规律进行准确预测,无法反映股票价格非线性部分进行有效建模,导致股价预测精度不高。为了提高股票价格预测精度,提出了一种遗传优化BP神经网络的股票价格预测模型。充分利用BP神经网络良好的非线性映射能力,对股票价格变化规律进行建模,并通过遗传算法对BP神经网络模型参数进行优化,从而获最优股票价格最优预测模型。实验结果表明,相对于传统股票价格预测模型,遗传算法优化BP神经网络的股票价格预测模型拟合程度更好,预测精度更高,为股票价格预测提供了依据。