A moving-average filter based hybrid ARIMA–ANN model for forecasting time series data

A suitable combination of linear and nonlinear models provides a more accurate prediction model than an individual linear or nonlinear model for forecasting time series data originating from various applications. The linear autoregressive integrated moving average (ARIMA) and nonlinear artificial neural network (ANN) models are explored in this paper to devise a new hybrid ARIMA–ANN model for the prediction of time series data. Many of the hybrid ARIMA–ANN models which exist in the literature apply an ARIMA model to given time series data, consider the error between the original and the ARIMA-predicted data as a nonlinear component, and model it using an ANN in different ways. Though these models give predictions with higher accuracy than the individual models, there is scope for further improvement in the accuracy if the nature of the given time series is taken into account before applying the models. In the work described in this paper, the nature of volatility was explored using a moving-average filter, and then an ARIMA and an ANN model were suitably applied. Using a simulated data set and experimental data sets such as sunspot data, electricity price data, and stock market data, the proposed hybrid ARIMA–ANN model was applied along with individual ARIMA and ANN models and some existing hybrid ARIMA–ANN models. The results obtained from all of these data sets show that for both one-step-ahead and multistep-ahead forecasts, the proposed hybrid model has higher prediction accuracy.     

Predicting river daily flow using wavelet-artificial neural networks based on regression analyses in comparison with artificial neural networks and support vector machine models

This study investigates the ability of wavelet-artificial neural networks (WANN) for the prediction of short-term daily river flow. The WANN model is improved by conjunction of two methods, discrete wavelet transform and artificial neural networks (ANN) based on regression analyses, respectively. The proposed WANN models are applied to the daily flow data of Vanyar station, on the Ajichai River in the northwest region of Iran, and compared with the ANN and support vector machine (SVM) techniques. Mean square error (MSE), mean absolute error (MAE) and correlation coefficient (R) statistics are used for evaluating precision of the WANN, ANN and SVM models. Comparison results demonstrate that the WANN model performs better than the ANN and SVM models in short-term (1-, 2- and 3-day ahead) daily river flow prediction.

     

Hybrid neural modeling for groundwater level prediction

The accurate prediction of groundwater level is important for the efficient use and management of groundwater resources, particularly in sub-humid regions where water surplus in monsoon season and water scarcity in non-monsoon season is a common phenomenon. In this paper, an attempt has been made to develop a hybrid neural model (ANN-GA) employing an artificial neural network (ANN) model in conjunction with famous optimization strategy called genetic algorithms (GA) for accurate prediction of groundwater levels in the lower Mahanadi river basin of Orissa State, India. Three types of functionally different algorithm-based ANN models (viz. back-propagation (GDX), Levenberg–Marquardt (LM) and Bayesian regularization (BR)) were used to compare the strength of proposed hybrid model in the efficient prediction of groundwater fluctuations. The ANN-GA hybrid modeling was carried out with lead-time of 1 week and study mainly aimed at November and January months of a year. Overall, simulation results suggest that the Bayesian regularization model is the most efficient of the ANN models tested for the study period. However, a strong correlation between the observed and predicted groundwater levels was observed for all the models. The results reveal that the hybrid GA-based ANN algorithm is able to produce better accuracy and performance in medium and high groundwater level predictions compared to conventional ANN techniques including Bayesian regularization model. Furthermore, the study shows that hybrid neural models can offer significant implications for improving groundwater management and water supply planning in semi-arid areas where aquifer information is not available.     

A Study of Hybrid Neural Network Approaches and the Effects of Missing Data on Traffic Forecasting

In this paper we present an application of hybrid neural network approaches and an assessment of the effects of missing data on motorway traffic flow forecasting. Two hybrid approaches are developed using a Self-Organising Map (SOM) to initially classify traffic into different states. The first hybrid approach includes four Auto-Regressive Integrated Moving Average (ARIMA) models, whilst the second uses two Multi-Layer Perception (MLP) models. It was found that the SOM/ARIMA hybrid approach out-performs all individual ARIMA models, whilst the SOM/MLP hybrid approach achieves superior forecasting performance to all models used in this study, including three na?ve models. The effects of different proportions of missing data on Neural Network (NN) performance when forecasting traffic flow are assessed and several initial substitution options to replace missing data are discussed. Over-all, it is shown that ARIMA models are more sensitive to the percentage of missing data than neural networks in this context.       

Modelling of diesel engine performance using advanced machine learning methods under scarce and exponential data set

Traditional methods on creating diesel engine models include the analytical methods like multi-zone models and the intelligent based models like artificial neural network (ANN) based models. However, those analytical models require excessive assumptions while those ANN models have many drawbacks such as the tendency to overfitting and the difficulties to determine the optimal network structure. In this paper, several emerging advanced machine learning techniques, including least squares support vector machine (LS-SVM), relevance vector machine (RVM), basic extreme learning machine (ELM) and kernel based ELM, are newly applied to the modelling of diesel engine performance. Experiments were carried out to collect sample data for model training and verification. Limited by the experiment conditions, only 24 sample data sets were acquired, resulting in data scarcity. Six-fold cross-validation is therefore adopted to address this issue. Some of the sample data are also found to suffer from the problem of data exponentiality, where the engine performance output grows up exponentially along the engine speed and engine torque. This seriously deteriorates the prediction accuracy. Thus, logarithmic transformation of dependent variables is utilized to pre-process the data. Besides, a hybrid of leave-one-out cross-validation and Bayesian inference is, for the first time, proposed for the selection of hyperparameters of kernel based ELM. A comparison among the advanced machine learning techniques, along with two traditional types of ANN models, namely back propagation neural network (BPNN) and radial basis function neural network (RBFNN), is conducted. The model evaluation is made based on the time complexity, space complexity, and prediction accuracy. The evaluation results show that kernel based ELM with the logarithmic transformation and hybrid inference is far better than basic ELM, LS-SVM, RVM, BPNN and RBFNN, in terms of prediction accuracy and training time.     

Performance evaluation of hybrid Wavelet-ANN and Wavelet-ANFIS models for estimating evapotranspiration in arid regions of India

This paper evaluates the ability of wavelet transform in improving the accuracy of artificial neural network (ANN) and adaptive neuro-fuzzy interface systems (ANFIS) models. In this study, the performance of hybrid Wavelet-ANN and Wavelet-ANFIS models for estimating daily evapotranspiration in arid regions was evaluated. Prior to the development of models, gamma test was used to identify the best input combinations that could be used under limited data scenario. Performance of the proposed hybrid models was compared to ANN, ANFIS, and conventionally used Hargreaves equation. The results revealed that use of wavelet transform as data preprocessing technique enhanced the efficiency of ANN and ANFIS models. Wavelet-ANN and Wavelet-ANFIS performed reasonably better than other models. Better handling of wavelet-decomposed input variables enabled Wavelet-ANN models to perform slightly better than the Wavelet-ANFIS models. W-ANN2 (RMSE = 0.632 mm/day and R = 0.96) was found to be the best model for estimating daily evapotranspiration in arid regions. The proposed W-ANN2 model used second-level db3 wavelet-decomposed subseries of temperature and previous day evapotranspiration values as inputs. The study concludes that hybrid Wavelet-ANN and Wavelet-ANFIS models can be effectively used for modeling evapotranspiration.

     

Forecasting of Software Reliability Using Neighborhood Fuzzy Particle Swarm Optimization Based Novel Neural Network

This paper proposes an artificial neural network (ANN) based software reliability model trained by novel particle swarm optimization (PSO) algorithm for enhanced forecasting of the reliability of software. The proposed ANN is developed considering the fault generation phenomenon during software testing with the fault complexity of different levels. We demonstrate the proposed model considering three types of faults residing in the software. We propose a neighborhood based fuzzy PSO algorithm for competent learning of the proposed ANN using software failure data. Fitting and prediction performances of the neighborhood fuzzy PSO based proposed neural network model are compared with the standard PSO based proposed neural network model and existing ANN based software reliability models in the literature through three real software failure data sets. We also compare the performance of the proposed PSO algorithm with the standard PSO algorithm through learning of the proposed ANN. Statistical analysis shows that the neighborhood fuzzy PSO based proposed neural network model has comparatively better fitting and predictive ability than the standard PSO based proposed neural network model and other ANN based software reliability models. Faster release of software is achievable by applying the proposed PSO based neural network model during the testing period.      

Credit rating by hybrid machine learning techniques

It is very important for financial institutions to develop credit rating systems to help them to decide whether to grant credit to consumers before issuing loans. In literature, statistical and machine learning techniques for credit rating have been extensively studied. Recent studies focusing on hybrid models by combining different machine learning techniques have shown promising results. However, there are various types of combination methods to develop hybrid models. It is unknown that which hybrid machine learning model can perform the best in credit rating. In this paper, four different types of hybrid models are compared by ‘Classification + Classification’, ‘Classification + Clustering’, ‘Clustering + Classification’, and ‘Clustering + Clustering’ techniques, respectively. A real world dataset from a bank in Taiwan is considered for the experiment. The experimental results show that the ‘Classification + Classification’ hybrid model based on the combination of logistic regression and neural networks can provide the highest prediction accuracy and maximize the profit.     

A comparative study of artificial neural networks,and decision trees for digital game content stocks price prediction

Precise prediction of stock prices is difficult chiefly because of the many intervening factors. Unpredictability is particularly notable in the aftermath of the global financial crisis. Data mining may however be used to discover highly correlated estimation models. This study looks at artificial neural networks (ANN), decision trees and the hybrid model of ANN and decision trees (hybrid model), the three common algorithm methods used for numerical analysis, to forecast stock prices. The author compared the stock price forecasting models derived from the three methods, and applied the models on 10 different stocks in 320 data sets in an empirical forecast. Average accuracy of ANN is 15.31%, the highest, in terms of match with real market stock prices, followed by decision trees, at 14.06%; hybrid model is 13.75%. The study also discovers that compared to the other two methods, ANN is a more stable method for predicting stock prices in the volatile post-crisis stock market.     

Support vector regression with reduced training sets for air temperature prediction: a comparison with artificial neural networks

Sudden changes in weather, in particular extreme temperatures, can result in increased energy expenditures, depleted agricultural resources, and even loss of life. However, these ill effects can be reduced with accurate air temperature predictions that provide adequate advance warning. Support vector regression (SVR) was applied to meteorological data collected across the state of Georgia in order to produce short-term air temperature predictions. A method was proposed for reducing the number of training patterns of massively large data sets that does not require lengthy pre-processing of the data. This method was demonstrated on two large data sets: one containing 300,000 cold-weather training patterns collected during the winter months and one containing 1.25 million training patterns collected throughout the year. These patterns were used to produce predictions from 1 to 12 h ahead. The mean absolute error (MAE) for the evaluation set of winter-only patterns ranged from 0.514°C for the 1-h prediction horizon to 2.303°C for the 12-h prediction horizon. For the evaluation set of year-round patterns, the MAE ranged from 0.513°C for the 1-h prediction horizon to 1.922°C for the 12-h prediction horizon. These results were competitive with previously developed artificial neural network (ANN) models that were trained on the full data sets. For the winter-only evaluation data, the SVR models were slightly more accurate than the ANN models for all twelve of the prediction horizons. For the year-round evaluation data, the SVR models were slightly more accurate than the ANN models for three of the twelve prediction horizons.     

Combining cluster analysis with classifier ensembles to predict financial distress

The ability to accurately predict business failure is a very important issue in financial decision-making. Incorrect decision-making in financial institutions is very likely to cause financial crises and distress. Bankruptcy prediction and credit scoring are two important problems facing financial decision support. As many related studies develop financial distress models by some machine learning techniques, more advanced machine learning techniques, such as classifier ensembles and hybrid classifiers, have not been fully assessed. The aim of this paper is to develop a novel hybrid financial distress model based on combining the clustering technique and classifier ensembles. In addition, single baseline classifiers, hybrid classifiers, and classifier ensembles are developed for comparisons. In particular, two clustering techniques, Self-Organizing Maps (SOMs) and k-means and three classification techniques, logistic regression, multilayer-perceptron (MLP) neural network, and decision trees, are used to develop these four different types of bankruptcy prediction models. As a result, 21 different models are compared in terms of average prediction accuracy and Type I & II errors. By using five related datasets, combining Self-Organizing Maps (SOMs) with MLP classifier ensembles performs the best, which provides higher predication accuracy and lower Type I & II errors.     

An effective hybrid learning system for telecommunication churn prediction

Customer churn has emerged as a critical issue for Customer Relationship Management and customer retention in the telecommunications industry, thus churn prediction is necessary and valuable to retain the customers and reduce the losses. Moreover, high predictive accuracy and good interpretability of the results are two key measures of a classification model. More studies have shown that single model-based classification methods may not be good enough to achieve a satisfactory result. To obtain more accurate predictive results, we present a novel hybrid model-based learning system, which integrates the supervised and unsupervised techniques for predicting customer behaviour. The system combines a modified k-means clustering algorithm and a classic rule inductive technique (FOIL).Three sets of experiments were carried out on telecom datasets. One set of the experiments is for verifying that the weighted k-means clustering can lead to a better data partitioning results; the second set of experiments is for evaluating the classification results, and comparing it to other well-known modelling techniques; the last set of experiment compares the proposed hybrid-model system with several other recently proposed hybrid classification approaches. We also performed a comparative study on a set of benchmarks obtained from the UCI repository. All the results show that the hybrid model-based learning system is very promising and outperform the existing models.     

Application of artificial neural network (ANN)–self-organizing map (SOM) for the categorization of water,soil and sediment quality in petrochemical regions

The utilization of mathematical and computational tools for pollutant assessment frameworks has become increasingly valuable due to the capability to interpret integrated variable measurements. Artificial neural networks (ANNs) are considered as dependable and inexpensive techniques for data interpretation and prediction. The self-organizing map (SOM) is an unsupervised ANN used for data training to classify and effectively recognize patterns embedded in the input data space. Application of SOM–ANN is useful for recognizing spatial patterns in contaminated zones by integrating chemical, physical, ecotoxicological and toxicokinetic variables in the identification of pollution sources and similarities in the quality of the samples. Water (n = 11), soil (n = 38) and sediment (n = 54) samples from four areas in the Niger Delta (Nigeria) were classified based on their chemical, toxicological and physical variables applying the SOM. The results obtained in this study provided valuable assessment using the SOM visualization capabilities and highlighted zones of priority that might require additional investigations and also provide productive pathway for effective decision making and remedial actions.     

The influence of ARIMA-GARCH parameters in feed forward neural networks prediction

The objective of this article is to find out the influence of the parameters of the ARIMA-GARCH models in the prediction of artificial neural networks (ANN) of the feed forward type, trained with the Levenberg–Marquardt algorithm, through Monte Carlo simulations. The paper presents a study of the relationship between ANN performance and ARIMA-GARCH model parameters, i.e. the fact that depending on the stationarity and other parameters of the time series, the ANN structure should be selected differently. Neural networks have been widely used to predict time series and their capacity for dealing with non-linearities is a normally outstanding advantage. However, the values of the parameters of the models of generalized autoregressive conditional heteroscedasticity have an influence on ANN prediction performance. The combination of the values of the GARCH parameters with the ARIMA autoregressive terms also implies in ANN performance variation. Combining the parameters of the ARIMA-GARCH models and changing the ANN’s topologies, we used the Theil inequality coefficient to measure the prediction of the feed forward ANN.     

Hybrid modeling of spatial continuity for application to numerical inverse problems

A novel two-step modeling approach is presented to obtain optimal starting values and geostatistical constraints for numerical inverse problems otherwise characterized by spatially-limited field data. First, a type of unsupervised neural network, called the self-organizing map (SOM), is trained to recognize nonlinear relations among environmental variables (covariates) occurring at various scales. The values of these variables are then estimated at random locations across the model domain by iterative minimization of SOM topographic error vectors. Cross-validation is used to ensure unbiasedness and compute prediction uncertainty for select subsets of the data. Second, analytical functions are fit to experimental variograms derived from original plus resampled SOM estimates producing model variograms. Sequential Gaussian simulation is used to evaluate spatial uncertainty associated with the analytical functions and probable range for constraining variables. The hybrid modeling of spatial continuity is demonstrated using spatially-limited hydrologic measurements at different scales in Brazil: (1) physical soil properties (sand, silt, clay, hydraulic conductivity) in the 42 km² Vargem de Caldas basin; (2) well yield and electrical conductivity of groundwater in the 132 km² fractured crystalline aquifer; and (3) specific capacity, hydraulic head, and major ions in a 100,000 km² transboundary fractured-basalt aquifer. These results illustrate the benefits of exploiting nonlinear relations among sparse and disparate data sets for modeling spatial continuity, but the actual application of these spatial data to improve numerical inverse modeling requires testing.     

Proposing two new metaheuristic algorithms of ALO-MLP and SHO-MLP in predicting bearing capacity of circular footing located on horizontal multilayer soil

In this study, for the issue of shallow circular footing’s bearing capacity (also shown as F_ult), we used the merits of artificial neural network (ANN), while optimized it by two metaheuristic algorithms (i.e., ant lion optimization (ALO) and the spotted hyena optimizer (SHO)). Several studies demonstrated that ANNs have significant results in terms of predicting the soil’s bearing capacity. Nevertheless, most models of ANN learning consist of different disadvantages. Accordantly, we focused on the application of two hybrid models of ALO–MLP and SHO–MLP for predicting the F_ult placed in layered soils. Moreover, we performed an Extensive Finite Element (FE) modeling on 16 sets of soil layer (soft soil placed onto stronger soil and vice versa) considering a database that consists of 703 testing and 2810 training datasets for preparing the training and testing datasets. The independent variables in terms of ALO and SHO algorithms have been optimized by taking into account a trial and error process. The input data layers consisted of (i) upper layer foundation/thickness width (h/B) ratio, (ii) bottom and topsoil layer properties (for example, six of the most important properties of soil), (iii) vertical settlement (s), (iv) footing width (B), where the main target was taken F_ult. According to RMSE and R², values of (0.996 and 0.034) and (0.994 and 0.044) are obtained for training dataset and values of (0.994 and 0.040) and (0.991 and 0.050) are found for the testing dataset of proposed SHO–MLP and ALO–MLP best-fit prediction network structures, respectively. This proves higher reliability of the proposed hybrid model of SHO–MLP in approximating shallow circular footing bearing capacity.

     

Artificial neural networks for predicting maximum wave runup on rubble mound structures

Accurate prediction of maximum wave runup on breakwaters is a vital issue for determining crest level of coastal structures. In practice, traditional regression-based empirical model, recommended by the “Coastal Engineering Manual”, as well as the “Manual on the use of rock in hydraulic engineering”, is widely used. However, use of these approaches brings additional restrictive assumptions such as linearity, normality (Gaussian distributed variables), variance constancy (homoscedasticity) etc. This paper focuses on the prediction of maximum wave runup elevation through artificial neural networks (ANNs), which has no restrictive assumptions. Out of 261 irregular wave runup data of Van der Meer and Stam, 100 randomly chosen data points are used for training the model. The remaining data are exploited for testing purposes. This study has two objectives: (1) to develop ANN models and search their applicability to estimate maximum wave runup elevation on breakwaters; (2) to compare widely used empirical model with these models. For these purposes, different ANN models are constructed and trained with their own topology. The performance of the ANN models is tested against the same testing data, none of which is employed in the training. It is found that ANN technique gives more accurate results and the extent of accuracy can be affected by the structure of ANNs.     

The effect of ICA and PSO on ANN results in approximating elasticity modulus of rock material

Reliable determination/evaluation of the rock deformation can be useful prior any structural design application. Young’s modulus (E) affords great insight into the characteristics of the rock. However, its direct determination in the laboratory is costly and time-consuming. Therefore, rock deformation prediction through indirect techniques is greatly suggested. This paper describes hybrid particle swarm optimization (PSO)–artificial neural network (ANN) and imperialism competitive algorithm (ICA)–ANN to solve shortcomings of ANN itself. In fact, the influence of PSO and ICA on ANN results in predicting E was studied in this research. By investigating the related studies, the most important parameters of PSO and ICA were identified and a series of parametric studies for their determination were conducted. All models were built using three inputs (Schmidt hammer rebound number, point load index and p-wave velocity) and one output which is E. To have a fair comparison and to show the capability of the hybrid models, a pre-developed ANN model was also constructed to estimate E. Evaluation of the obtained results demonstrated that a higher ability of E prediction is received developing a hybrid ICA–ANN model. Coefficient of determination (R²) values of (0.952, 0.943 and 0.753) and (0.955, 0.949 and 0.712) were obtained for training and testing of ICA–ANN, PSO–ANN and ANN models, respectively. In addition, VAF values near to 100 (95.182 and 95.143 for train and test) were achieved for a developed ICA–ANN hybrid model. The results indicated that the proposed ICA–ANN model can be implemented better in improving performance capacity of ANN model compared to another implemented hybrid model.

     

Web Navigation Prediction Using Multiple Evidence Combination and Domain Knowledge

Predicting users' future requests in the World Wide Web can be applied effectively in many important applications, such as web search, latency reduction, and personalization systems. Such application has traditional tradeoffs between modeling complexity and prediction accuracy. In this paper, we study several hybrid models that combine different classification techniques, namely, Markov models, artificial neural networks (ANNs), and the All-Kth-Markov model, to resolve prediction using Dempster's rule. Such fusion overcomes the inability of the Markov model in predicting beyond the training data, as well as boosts the accuracy of ANN, particularly, when dealing with a large number of classes. We also employ a reduction technique, which uses domain knowledge, to reduce the number of classifiers to improve the predictive accuracy and the prediction time of ANNs. We demonstrate the effectiveness of our hybrid models by comparing our results with widely used techniques, namely, the Markov model, the All-Kth-Markov model, and association rule mining, based on a benchmark data set.     

An optimized ANN model based on genetic algorithm for predicting ripping production

Due to the environmental constraints and the limitations on blasting, ripping as a ground loosening and breaking method has become more popular in both mining and civil engineering applications. As a result, a more applicable rippability model is required to predict ripping production (Q) before conducting such tests. In this research, a hybrid genetic algorithm (GA) optimized by artificial neural network (ANN) was developed to predict ripping production results obtained from three sites in Johor state, Malaysia. It should be noted that the mentioned hybrid model was first time applied in this field. In this regard, 74 ripping tests were investigated in the studied areas and the relevant parameters were also measured. A series of GA–ANN models were conducted in order to propose a hybrid model with a higher accuracy level. To demonstrate the performance capacity of the hybrid GA–ANN model, a pre-developed ANN model was also proposed and results of predictive models were compared using several performance indices. The results revealed higher accuracy of the proposed hybrid GA–ANN model in estimating Q compared to ANN technique. As an example, root-mean-square error values of 0.092 and 0.131 for testing datasets of GA–ANN and ANN techniques, respectively, express the superiority of the newly developed model in predicting ripping production.