Global and decomposition evolutionary support vector machine approaches for time series forecasting

Multi-step ahead time series forecasting (TSF) is a key tool for supporting tactical decisions (e.g., planning resources). Recently, the support vector machine (SVM) emerged as a natural solution for TSF due to its nonlinear learning capabilities. This paper presents two novel evolutionary SVM (ESVM) methods for multi-step TSF. Both methods are based on an estimation distribution algorithm search engine that automatically performs a simultaneous variable (number of inputs) and model (hyperparameters) selection. The global ESVM (GESVM) uses all past patterns to fit the SVM, while the decomposition ESVM (DESVM) separates the series into trended and stationary effects, using a distinct ESVM to forecast each effect and then summing both predictions into a single response. Several experiments were held, using six time series. The proposed approaches were analyzed under two criteria and compared against a recent evolutionary artificial neural network (EANN) and two classical forecasting methods, Holt–Winters and autoregressive integrated moving average. Overall, the DESVM and GESVM obtained competitive and high-quality results. Furthermore, both ESVM approaches consume much less computational effort when compared with EANN.     

Time series forecasting by evolving artificial neural networks with genetic algorithms, differential evolution and estimation of distribution algorithm

Time series forecasting is an important tool to support both individual and organizational decisions (e.g. planning production resources). In recent years, a large literature has evolved on the use of evolutionary artificial neural networks (EANN) in many forecasting applications. Evolving neural networks are particularly appealing because of their ability to model an unspecified nonlinear relationship between time series variables. In this work, two new approaches of a previous system, automatic design of artificial neural networks (ADANN) applied to forecast time series, are tackled. In ADANN, the automatic process to design artificial neural networks was carried out by a genetic algorithm (GA). This paper evaluates three methods to evolve neural networks architectures, one carried out with genetic algorithm, a second one carried out with differential evolution algorithm (DE) and the last one using estimation of distribution algorithms (EDA). A comparative study among these three methods with a set of referenced time series will be shown. In this paper, we also compare ADANN forecasting ability against a forecasting tool called Forecast Pro^® (FP) software, using five benchmark time series. The object of this study is to try to improve the final forecasting getting an accurate system.     

Multi-step time series prediction intervals using neuroevolution

Multi-step time series forecasting (TSF) is a crucial element to support tactical decisions (e.g., designing production or marketing plans several months in advance). While most TSF research addresses only single-point prediction, prediction intervals (PIs) are useful to reduce uncertainty related to important decision making variables. In this paper, we explore a large set of neural network methods for multi-step TSF and that directly optimize PIs. This includes multi-step adaptations of recently proposed PI methods, such as lower–upper bound estimation (LUBET), its ensemble extension (LUBEXT), a multi-objective evolutionary algorithm LUBE (MLUBET) and a two-phase learning multi-objective evolutionary algorithm (M2LUBET). We also explore two new ensemble variants for the evolutionary approaches based on two PI coverage–width split methods (radial slices and clustering), leading to the MLUBEXT, M2LUBEXT, MLUBEXT2 and M2LUBEXT2 methods. A robust comparison was held by considering the rolling window procedure, nine time series from several real-world domains and with different characteristics, two PI quality measures (coverage error and width) and the Wilcoxon statistic. Overall, the best results were achieved by the M2LUBET neuroevolution method, which requires a reasonable computational effort for time series with a few hundreds of observations.

     

A combination of artificial neural network and random walk models for financial time series forecasting

Properly comprehending and modeling the dynamics of financial data has indispensable practical importance. The prime goal of a financial time series model is to provide reliable future forecasts which are crucial for investment planning, fiscal risk hedging, governmental policy making, etc. These time series often exhibit notoriously haphazard movements which make the task of modeling and forecasting extremely difficult. As per the research evidence, the random walk (RW) is so far the best linear model for forecasting financial data. Artificial neural network (ANN) is another promising alternative with the unique capability of nonlinear self-adaptive modeling. Numerous comparisons of the performances of RW and ANN models have also been carried out in the literature with mixed conclusions. In this paper, we propose a combination methodology which attempts to benefit from the strengths of both RW and ANN models. In our proposed approach, the linear part of a financial dataset is processed through the RW model, and the remaining nonlinear residuals are processed using an ensemble of feedforward ANN (FANN) and Elman ANN (EANN) models. The forecasting ability of the proposed scheme is examined on four real-world financial time series in terms of three popular error statistics. The obtained results clearly demonstrate that our combination method achieves reasonably better forecasting accuracies than each of RW, FANN and EANN models in isolation for all four financial time series.     

Integration of supervised ART-based neural networks with a hybrid genetic algorithm

In this paper, two evolutionary artificial neural network (EANN) models that are based on integration of two supervised adaptive resonance theory (ART)-based artificial neural networks with a hybrid genetic algorithm (HGA) are proposed. The search process of the proposed EANN models is guided by a knowledge base established by ART with respect to the training data samples. The EANN models explore the search space for “coarse” solutions, and such solutions are then refined using the local search process of the HGA. The performances of the proposed EANN models are evaluated and compared with those from other classifiers using more than ten benchmark data sets. The applicability of the EANN models to a real medical classification task is also demonstrated. The results from the experimental studies demonstrate the effectiveness and usefulness of the proposed EANN models in undertaking pattern classification problems.     

A hybrid optimized error correction system for time series forecasting

Time series forecasting is a challenging task in machine learning. Real world time series are often composed by linear and nonlinear structures which need to be mapped by some forecasting method. Linear methods such as autoregressive integrated moving average (ARIMA) and nonlinear methods such as artificial neural networks (ANNs) could be employed to handle such problems, however model misspecification hinders the forecasting process producing inaccurate models. Hybrid models based on error forecasting and combination can reduce the misspecification of single models and improve the accuracy of the system. This work proposes a hybrid system that is composed of three parts: a) linear modeling of the time series, b) nonlinear modeling of the error series, and c) combination of the forecasts using three distinct approaches. The system performs a search for the best parameters of the linear and nonlinear components, and of the combination approaches. Particle swarm optimization is used to find suitable architecture and weights. Experiments show that the proposed technique achieved promising results in time series forecasting.     

Hybrid neural network models for hydrologic time series forecasting

The need for increased accuracies in time series forecasting has motivated the researchers to develop innovative models. In this paper, a new hybrid time series neural network model is proposed that is capable of exploiting the strengths of traditional time series approaches and artificial neural networks (ANNs). The proposed approach consists of an overall modelling framework, which is a combination of the conventional and ANN techniques. The steps involved in the time series analysis, e.g. de-trending and de-seasonalisation, can be carried out before gradually presenting the modified time series data to the ANN. The proposed hybrid approach for time series forecasting is tested using the monthly streamflow data at Colorado River at Lees Ferry, USA. Specifically, results from four time series models of auto-regressive (AR) type and four ANN models are presented. The results obtained in this study suggest that the approach of combining the strengths of the conventional and ANN techniques provides a robust modelling framework capable of capturing the non-linear nature of the complex time series and thus producing more accurate forecasts. Although the proposed hybrid neural network models are applied in hydrology in this study, they have tremendous scope for application in a wide range of areas for achieving increased accuracies in time series forecasting.     

Pareto evolutionary neural networks

For the purposes of forecasting (or classification) tasks neural networks (NNs) are typically trained with respect to Euclidean distance minimization. This is commonly the case irrespective of any other end user preferences. In a number of situations, most notably time series forecasting, users may have other objectives in addition to Euclidean distance minimization. Recent studies in the NN domain have confronted this problem by propagating a linear sum of errors. However this approach implicitly assumes a priori knowledge of the error surface defined by the problem, which, typically, is not the case. This study constructs a novel methodology for implementing multiobjective optimization within the evolutionary neural network (ENN) domain. This methodology enables the parallel evolution of a population of ENN models which exhibit estimated Pareto optimality with respect to multiple error measures. A new method is derived from this framework, the Pareto evolutionary neural network (Pareto-ENN). The Pareto-ENN evolves a population of models that may be heterogeneous in their topologies inputs and degree of connectivity, and maintains a set of the Pareto optimal ENNs that it discovers. New generalization methods to deal with the unique properties of multiobjective error minimization that are not apparent in the uni-objective case are presented and compared on synthetic data, with a novel method based on bootstrapping of the training data shown to significantly improve generalization ability. Finally experimental evidence is presented in this study demonstrating the general application potential of the framework by generating populations of ENNs for forecasting 37 different international stock indexes.     

Optimization of neural networks through grammatical evolution and a genetic algorithm

This paper proposes a hybrid neuro-evolutive algorithm (NEA) that uses a compact indirect encoding scheme (IES) for representing its genotypes (a set of ten production rules of a Lindenmayer System with memory), moreover has the ability to reuse the genotypes and automatically build modular, hierarchical and recurrent neural networks. A genetic algorithm (GA) evolves a Lindenmayer System (L-System) that is used to design the neural network’s architecture. This basic neural codification confers scalability and search space reduction in relation to other methods. Furthermore, the system uses a parallel genome scan engine that increases both the implicit parallelism and convergence of the GA. The fitness function of the NEA rewards economical artificial neural networks (ANNs) that are easily implemented. The NEA was tested on five real-world classification datasets and three well-known datasets for time series forecasting (TSF). The results are statistically compared against established state-of-the-art algorithms and various forecasting methods (ADANN, ARIMA, UCM, and Forecast Pro). In most cases, our NEA outperformed the other methods, delivering the most accurate classification and time series forecasting with the least computational effort. These superior results are attributed to the improved effectiveness and efficiency of NEA in the decision-making process. The result is an optimized neural network architecture for solving classification problems and simulating dynamical systems.     

Design of ANN Based Non-Linear Network Using Interconnection of Parallel Processor

Suspicious mass traffic constantly evolves, making network behaviour tracing and structure more complex. Neural networks yield promising results by considering a sufficient number of processing elements with strong interconnections between them. They offer efficient computational Hopfield neural networks models and optimization constraints used by undergoing a good amount of parallelism to yield optimal results. Artificial neural network (ANN) offers optimal solutions in classifying and clustering the various reels of data, and the results obtained purely depend on identifying a problem. In this research work, the design of optimized applications is presented in an organized manner. In addition, this research work examines theoretical approaches to achieving optimized results using ANN. It mainly focuses on designing rules. The optimizing design approach of neural networks analyzes the internal process of the neural networks. Practices in developing the network are based on the interconnections among the hidden nodes and their learning parameters. The methodology is proven best for nonlinear resource allocation problems with a suitable design and complex issues. The ANN proposed here considers more or less 46k nodes hidden inside 49 million connections employed on full-fledged parallel processors. The proposed ANN offered optimal results in real-world application problems, and the results were obtained using MATLAB.     

Time series forecasting based on wavelet decomposition and feature extraction

Time series forecasting is one of the most important issues in numerous applications in real life. The objective of this study was to propose a hybrid neural network model based on wavelet transform (WT) and feature extraction for time series forecasting. The motivation of the proposed model, which is called PCA-WCCNN, is to establish a single simplified model with shorter training time and satisfactory forecasting performance. This model combines the principal component analysis (PCA) and WT with artificial neural networks (ANNs). Given a forecasting sequence, order of the original forecasting model is determined firstly. Secondly, the original time series is decomposed into approximation and detail components by employing WT technique. Then, instead of using all the components as inputs, feature inputs are extracted from all the sub-series obtained from the above step. Finally, based on the extracted features and all the sub-series, a famous neural network construction method called cascade-correlation algorithm is applied to train neural network model to learn the dynamics. As an illustration, the proposed model is compared with two classical models and two hybrid models, respectively. They are the traditional cascade-correlation neural network, back-propagation neural network, wavelet-based cascade-correlation network using all the wavelet components as inputs to establish one model (WCCNN) and wavelet-based cascade-correlation network with combination of each sub-series model (WCCNN multi-models). Results obtained from this study indicate that the proposed method improves the accuracy of ANN and can yield better efficiency than other four neural network models.

     

Integration of genetic fuzzy systems and artificial neural networks for stock price forecasting

Stock market prediction is regarded as a challenging task in financial time-series forecasting. The central idea to successful stock market prediction is achieving best results using minimum required input data and the least complex stock market model. To achieve these purposes this article presents an integrated approach based on genetic fuzzy systems (GFS) and artificial neural networks (ANN) for constructing a stock price forecasting expert system. At first, we use stepwise regression analysis (SRA) to determine factors which have most influence on stock prices. At the next stage we divide our raw data into k clusters by means of self-organizing map (SOM) neural networks. Finally, all clusters will be fed into independent GFS models with the ability of rule base extraction and data base tuning. We evaluate capability of the proposed approach by applying it on stock price data gathered from IT and Airlines sectors, and compare the outcomes with previous stock price forecasting methods using mean absolute percentage error (MAPE). Results show that the proposed approach outperforms all previous methods, so it can be considered as a suitable tool for stock price forecasting problems.     

A new architecture selection method based on tabu search for artificial neural networks

Artificial neural networks (ANN) have been used in various applications in recent years. One of these applications is time series forecasting. Although ANN produces accurate forecasts in many time series implementations, there are still some problems with using ANN. ANN consist of some components such as architecture structure, learning algorithm and activation function. These components have important effect on the performance of ANN. An important decision is the selection of architecture structure that consists of determining the numbers of neurons in the layers of a network. Therefore, various approaches have been proposed to determine the best ANN architecture in the literature. However, the most preferred method is still trial and error method for finding a good architecture. In this study, a new architecture selection method based on tabu search algorithm is proposed. In the implementation, five real time series are analyzed by using ANN and the proposed method is employed to select the best architecture. For the comparison, these time series are also forecasted by using ANN when trial and error method is utilized to determine the best architecture. As a result of the implementation, it is clearly seen that better results are obtained when the proposed method is used for the selection of architecture.     

Ensembles of neural networks for forecasting of time series of spacecraft telemetry

We analyzed different approaches to developing ensembles of neural networks in respect to their forecasting accuracy. We describe a two level model of ensembles of neural networks for forecasting of telemetry time series of spacecraft’s subsystems. A possibility of additional training of these ensembles of neural networks is examined. Our results show that use of ensembles of neural networks with dynamic weighing allows us to reduce the forecasting error.     

An investigation and comparison of artificial neural network and time series models for Chinese food grain price forecasting

This paper compares the predictive performance of ARIMA, artificial neural network and the linear combination models for forecasting wheat price in Chinese market. Empirical results show that the combined model can improve the forecasting performance significantly in contrast with its counterparts in terms of the error evaluation measurements. However, as far as turning points and profit criterions are concerned, the ANN model is best as well as at capturing a significant number of turning points. The results are conflicting when implementing dissimilar forecasting criteria (the quantitative and the turning points measurements) to evaluate the performance of three models. The ANN model is overall the best model, and can be used as an alternative method to model Chinese future food grain price.     

A New Multiplicative Seasonal Neural Network Model Based on Particle Swarm Optimization

In recent years, artificial neural networks (ANNs) have been commonly used for time series forecasting by researchers from various fields. There are some types of ANNs and feed forward neural networks model is one of them. This type has been used to forecast various types of time series in many implementations. In this study, a novel multiplicative seasonal ANN model is proposed to improve forecasting accuracy when time series with both trend and seasonal patterns is forecasted. This neural networks model suggested in this study is the first model proposed in the literature to model time series which contain both trend and seasonal variations. In the proposed approach, the defined neural network model is trained by particle swarm optimization. In the training process, local minimum traps are avoided by using this population based heuristic optimization method. The performance of the proposed approach is examined by using two real seasonal time series. The forecasts obtained from the proposed method are compared to those obtained from other forecasting techniques available in the literature. It is seen that the proposed forecasting model provides high forecasting accuracy.     

Model dissection from earthquake time series: A comparative analysis using modern non-linear forecasting and artificial neural network approaches

This study utilizes two non-linear approaches to characterize model behavior of earthquake dynamics in the crucial tectonic regions of Northeast India (NEI). In particular, we have applied a (i) non-linear forecasting technique to assess the dimensionality of the earthquake-generating mechanism using the monthly frequency earthquake time series (magnitude ?4) obtained from NOAA and USGS catalogues for the period 1960–2003 and (ii) artificial neural network (ANN) methods—based on the back-propagation algorithm (BPA) to construct the neural network model of the same data set for comparing the two. We have constructed a multilayered feed forward ANN model with an optimum input set configuration specially designed to take advantage of more completely on the intrinsic relationships among the input and retrieved variables and arrive at the feasible model for earthquake prediction. The comparative analyses show that the results obtained by the two methods are stable and in good agreement and signify that the optimal embedding dimension obtained from the non-linear forecasting analysis compares well with the optimal number of inputs used for the neural networks.The constructed model suggests that the earthquake dynamics in the NEI region can be characterized by a high-dimensional chaotic plane. Evidence of high-dimensional chaos appears to be associated with “stochastic seasonal” bias in these regions and would provide some useful constraints for testing the model and criteria to assess earthquake hazards on a more rigorous and quantitative basis.     

Prediction of wind-induced pressures on a large gymnasium roof using artificial neural networks

The application of artificial neural networks (ANNs) to solve wind engineering problems has received increasing interests in recent years. This paper is concerned with developing two ANN approaches (a backpropagation neural network [BPNN] and a fuzzy neural network [FNN]) for the prediction of mean, root-mean-square (rms) pressure coefficients and time series of wind-induced pressures on a large gymnasium roof. In this study, simultaneous pressure measurements are made on a large gymnasium roof model in a boundary layer wind tunnel and parts of the model test data are used as the training sets for developing two ANN models to recognize the input–output patterns. Comparisons of the prediction results by the two ANN approaches and those from the wind tunnel test are made to examine the performance of the two ANN models, which demonstrates that the two ANN approaches can successfully predict the pressures on the entire surfaces of the large roof on the basis of wind tunnel pressure measurements from a certain number of pressure taps. Moreover, the FNN approach is found to be superior to the BPNN approach. It is shown through this study that the developed ANN approaches can be served as an effective tool for the design and analysis of wind effects on large roof structures.     

Combination of artificial neural-network forecasters for predictionof natural gas consumption

The focus of this paper is on combination of artificial neural-network (ANN) forecasters with application to the prediction of daily natural gas consumption needed by gas utilities. ANN forecasters can model the complex relationship between weather parameters and previous gas consumption with the future consumption. A two-stage system is proposed with the first stage containing two ANN forecasters, a multilayer feedforward ANN and a functional link ANN. These forecasters are initially trained with the error backpropagation algorithm, but an adaptive strategy is employed to adjust their weights during online forecasting. The second stage consists of a combination module to mix the two individual forecasts produced in the first stage. Eight different combination algorithms are examined, they are based on: averaging, recursive least squares, fuzzy logic, feedforward ANN, functional link ANN, temperature space approach, Karmarkar's linear programming algorithm (1984) and adaptive mixture of local experts (modular neural networks). The performance is tested on real data from six different gas utilities. The results indicate that combination strategies based on a single ANN outperform the other approaches.     

混合模型神经网络在短期负荷预测中的应用

提出了可应用于电力系统负荷预测的混合模型神经网络方法,该方法同时具有电力系统负荷预测的传统方法的优点及人工视网络方法的优点,该方法中,不同的负荷分量采用不同类型的预测方法,并采用基本绵谐振分量作神经网络的输入,神经网络的训练采用快速的学习算法进行,该方法具有很强的实时性和适应性,适用于没有气象资料的应用场合,仿真计算的结果表明,预测精度较传统来得高。