Prediction of dissolved oxygen in River Calder by noise elimination time series using wavelet transform

Prediction of dissolved oxygen (DO) plays an important role in water resources especially in surface waters such as rivers. The oxygen affects a vast number of other water indicators. In this study, the artificial neural network (ANN) and a hybrid wavelet-ANN (WANN) models were considered to predict thirty minutes dissolved oxygen in the River Calder at the Methley Bridge Station was located in the UK. For the proposed WANN model, the discrete wavelet transform (DWT) was linked to the ANN model for DO prediction. To achieve this aim, the original time series of thirty minutes DO and temperature (T) were decomposed in several sub-time series by DWT, and these new sub-series were imposed to the ANN model. The results were compared with single ANN model. The comparisons were done by some of the widely used relevant physical statistic indices. The Nash–Sutcliffe coefficient values were 0.998 and 0.740 for the WANN and ANN models, respectively. The model computed values of DO by the WANN model were in close agreement with respective measured values in the river water. Elimination noise by DWT model during pre-processing data is one of the abilities of the WANN model to better prediction. Since the results indicate closer approximation of the peak DO values by the WANN model, this model could be used for the simulation of cumulative DO data prediction in thirty minutes ahead.     

Predicting direction of stock price index movement using artificial neural networks and support vector machines: The sample of the Istanbul Stock Exchange

Prediction of stock price index movement is regarded as a challenging task of financial time series prediction. An accurate prediction of stock price movement may yield profits for investors. Due to the complexity of stock market data, development of efficient models for predicting is very difficult. This study attempted to develop two efficient models and compared their performances in predicting the direction of movement in the daily Istanbul Stock Exchange (ISE) National 100 Index. The models are based on two classification techniques, artificial neural networks (ANN) and support vector machines (SVM). Ten technical indicators were selected as inputs of the proposed models. Two comprehensive parameter setting experiments for both models were performed to improve their prediction performances. Experimental results showed that average performance of ANN model (75.74%) was found significantly better than that of SVM model (71.52%).     

Flood simulation using parallel genetic algorithm integrated wavelet neural networks

The conventional means of flood simulation and prediction using conceptual hydrological model or artificial neural network (ANN) has provided promising results in recent years. However, it is usually difficult to obtain ideal flood reproducing due to the structure of hydrological model. Back propagation (BP) algorithm of ANN may also reach local optimum when training nodal weights. To improve the mapping capability of neural networks, wavelet function was adopted (WANN) to strengthen the non-linear simulation accuracy and generality. In addition, genetic algorithm is integrated with WANN (GAWANN) to avoid reaching local optimum. Meanwhile, Message Passing Interface (MPI) subroutines are introduced for distributed implement considering the time consumption during nodal weights training. The GAWANN was applied in the flood simulation and prediction in arid area. The test results of 4 independent cases were compared to reveal the relations between historical rainfall and runoff under different time lags. The simulation was also carried out with Xinanjiang model to demonstrate the capability of GAWANN. The numerical experiments in this paper indicated that the parallel GAWANN has strong capability of rain-runoff mapping as well as computational efficiency and is suitable for applications of flood simulation in arid areas.     

Generalization performance of support vector machines and neural networks in runoff modeling

Effective one-day lead runoff prediction is one of the significant aspects of successful water resources management in arid region. For instance, reservoir and hydropower systems call for real-time or on-line site-specific forecasting of the runoff. In this research, we present a new data-driven model called support vector machines (SVMs) based on structural risk minimization principle, which minimizes a bound on a generalized risk (error), as opposed to the empirical risk minimization principle exploited by conventional regression techniques (e.g. ANNs). Thus, this stat-of-the-art methodology for prediction combines excellent generalization property and sparse representation that lead SVMs to be a very promising forecasting method. Further, SVM makes use of a convex quadratic optimization problem; hence, the solution is always unique and globally optimal. To demonstrate the aforementioned forecasting capability of SVM, one-day lead stream flow of Bakhtiyari River in Iran was predicted using the local climate and rainfall data. Moreover, the results were compared with those of ANN and ANN integrated with genetic algorithms (ANN-GA) models. The improvements in root mean squared error (RMSE) and squared correlation coefficient (R²) by SVM over both ANN models indicate that the prediction accuracy of SVM is at least as good as that of those models, yet in some cases actually better, as well as forecasting of high-value discharges.     

Customer churn prediction by hybrid neural networks

As churn management is a major task for companies to retain valuable customers, the ability to predict customer churn is necessary. In literature, neural networks have shown their applicability to churn prediction. On the other hand, hybrid data mining techniques by combining two or more techniques have been proved to provide better performances than many single techniques over a number of different domain problems. This paper considers two hybrid models by combining two different neural network techniques for churn prediction, which are back-propagation artificial neural networks (ANN) and self-organizing maps (SOM). The hybrid models are ANN combined with ANN (ANN + ANN) and SOM combined with ANN (SOM + ANN). In particular, the first technique of the two hybrid models performs the data reduction task by filtering out unrepresentative training data. Then, the outputs as representative data are used to create the prediction model based on the second technique. To evaluate the performance of these models, three different kinds of testing sets are considered. They are the general testing set and two fuzzy testing sets based on the filtered out data by the first technique of the two hybrid models, i.e. ANN and SOM, respectively. The experimental results show that the two hybrid models outperform the single neural network baseline model in terms of prediction accuracy and Types I and II errors over the three kinds of testing sets. In addition, the ANN + ANN hybrid model significantly performs better than the SOM + ANN hybrid model and the ANN baseline model.     

Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions

Over the past 15 years, artificial neural networks (ANNs) have been used increasingly for prediction and forecasting in water resources and environmental engineering. However, despite this high level of research activity, methods for developing ANN models are not yet well established. In this paper, the steps in the development of ANN models are outlined and taxonomies of approaches are introduced for each of these steps. In order to obtain a snapshot of current practice, ANN development methods are assessed based on these taxonomies for 210 journal papers that were published from 1999 to 2007 and focus on the prediction of water resource variables in river systems. The results obtained indicate that the vast majority of studies focus on flow prediction, with very few applications to water quality. Methods used for determining model inputs, appropriate data subsets and the best model structure are generally obtained in an ad-hoc fashion and require further attention. Although multilayer perceptrons are still the most popular model architecture, other model architectures are also used extensively. In relation to model calibration, gradient based methods are used almost exclusively. In conclusion, despite a significant amount of research activity on the use of ANNs for prediction and forecasting of water resources variables in river systems, little of this is focused on methodological issues. Consequently, there is still a need for the development of robust ANN model development approaches.     

Artificial neural networks for infant mortality modelling

This work aims to investigate a simple to use and easy to interpret methodology for assessing the relative importance of input variables in artificial neural networks (ANNs) applied to epidemiological modelling. The independent variables were 43 variables of the social, economic, environmental and health sector of 59 Brazilian municipalities, and the outcomes were infant mortality rates from these municipalities. Two assays were developed for the ANN modelling. On the first, all 43 variables were taken as input; and on the second, input variables were chosen with the help of factor analysis (FA). The relative importance of the input variables was investigated by means of bootstrap replications of the ANN model on the second assay. Further, multiple linear regression models (LRMs) were developed with the same data set and compared to the ANN models. The FA analysis allowed the selection of eight variables for the second assay. The percent of explained variance R(2) on the ANNs was in the range 0.74-0.80, while linear models had R(2)=0.4-0.5. These findings were validated by the bootstrap replications, in which the ANN models remained with higher R(2) and lower mean square error than the LRMs. The analysis of the best (second) ANN model indicated the highest ranking of importance for the variables literacy, agricultural and livestock sector jobs, number of commercial establishments and telephones. The approach presented here successfully integrated a data-oriented model with expert knowledge, indicating the potentiality of ANN modelling in the prediction, planning and assessment of public health actions.     

Optimization of self-organizing polynomial neural networks

The main disadvantage of self-organizing polynomial neural networks (SOPNN) automatically structured and trained by the group method of data handling (GMDH) algorithm is a partial optimization of model weights as the GMDH algorithm optimizes only the weights of the topmost (output) node. In order to estimate to what extent the approximation accuracy of the obtained model can be improved the particle swarm optimization (PSO) has been used for the optimization of weights of all node-polynomials. Since the PSO is generally computationally expensive and time consuming a more efficient Levenberg–Marquardt (LM) algorithm is adapted for the optimization of the SOPNN. After it has been optimized by the LM algorithm the SOPNN outperformed the corresponding models based on artificial neural networks (ANN) and support vector method (SVM). The research is based on the meta-modeling of the thermodynamic effects in fluid flow measurements with time-constraints. The outstanding characteristics of the optimized SOPNN models are also demonstrated in learning the recurrence relations of multiple superimposed oscillations (MSO).     

Daily prediction of short-term trends or crude oil prices using neural networks exploiting multimarket dynamics

This paper documents a systematic investigation on the predictability of short-term trends of crude oil prices on a daily basis. In stark contrast with longer-term predictions of crude oil prices, short-term prediction with time horizons of 1-3 days posits an important problem that is quite different from what has been studied in the literature. The problem of such short-term predicability is tackled through two aspects. The first is to examine the existence of linear or nonlinear dynamic processes in crude oil prices. This sub-problem is addressed with statistical analysis involving the Brock-Dechert-Scheinkman test for nonlinearity. The second aspect is to test the capability of artificial neural networks (ANN) for modeling the implicit nonlinearity for prediction. Four experimental models are designed and tested with historical data: (1) using only the lagged returns of filtered crude oil prices as input to predict the returns of the next days; this is used as the benchmark, (2) using only the information set of filtered crude oil futures price as input, (3) combining the inputs from the benchmark and second models, and (4) combing the inputs from the benchmark model and the intermarket information. In order to filter out the noise in the original price data, the moving averages of prices are used for all the experiments. The results provided sufficient evidence to the predictability of crude oil prices using ANN with an out-of-sample hit rate of 80%, 70%, and 61% for each of the next three days’ trends.     

Non-tuned machine learning approach for hydrological time series forecasting

Stream-flow forecasting is a crucial task for hydrological science. Throughout the literature, traditional and artificial intelligence models have been applied to this task. An attempt to explore and develop better expert models is an ongoing endeavor for this hydrological application. In addition, the accuracy of modeling, confidence and practicality of the model are the other significant problems that need to be considered. Accordingly, this study investigates modern non-tuned machine learning data-driven approach, namely extreme learning machine (ELM). This data-driven approach is containing single layer feedforward neural network that selects the input variables randomly and determine the output weights systematically. To demonstrate the reliability and the effectiveness, one-step-ahead stream-flow forecasting based on three time-scale pattern (daily, mean weekly and mean monthly) for Johor river, Malaysia, were implemented. Artificial neural network (ANN) model is used for comparison and evaluation. The results indicated ELM approach superior the ANN model level accuracies and time consuming in addition to precision forecasting in tropical zone. In measureable terms, the dominance of ELM model over ANN model was indicated in accordance with coefficient determination (R ²) root-mean-square error (RMSE) and mean absolute error (MAE). The results were obtained for example the daily time scale R ² = 0.94 and 0.90, RMSE = 2.78 and 11.63, and MAE = 0.10 and 0.43, for ELM and ANN models respectively.

     

Application of artificial neural networks for water quality prediction

The term “water quality” is used to describe the condition of water, including its chemical, physical, and biological characteristics. Modeling water quality parameters is a very important aspect in the analysis of any aquatic systems. Prediction of surface water quality is required for proper management of the river basin so that adequate measure can be taken to keep pollution within permissible limits. Accurate prediction of future phenomena is the life blood of optimal water resources management. The artificial neural network is a new technique with a flexible mathematical structure that is capable of identifying complex non-linear relationships between input and output data when compared to other classical modeling techniques. Johor River Basin located in Johor state, Malaysia, which is significantly degrading due to human activities and development along the river. Accordingly, it is very important to implement and adopt a water quality prediction model that can provide a powerful tool to implement better water resource management. Several modeling methods have been applied in this research including: linear regression models (LRM), multilayer perceptron neural networks and radial basis function neural networks (RBF-NN). The results showed that the use of neural networks and more specifically RBF-NN models can describe the behavior of water quality parameters more accurately than linear regression models. In addition, we observed that the RBF finds a solution faster than the MLP and is the most accurate and most reliable tool in terms of processing large amounts of non-linear, non-parametric data.

     

The influence of ARIMA-GARCH parameters in feed forward neural networks prediction

The objective of this article is to find out the influence of the parameters of the ARIMA-GARCH models in the prediction of artificial neural networks (ANN) of the feed forward type, trained with the Levenberg–Marquardt algorithm, through Monte Carlo simulations. The paper presents a study of the relationship between ANN performance and ARIMA-GARCH model parameters, i.e. the fact that depending on the stationarity and other parameters of the time series, the ANN structure should be selected differently. Neural networks have been widely used to predict time series and their capacity for dealing with non-linearities is a normally outstanding advantage. However, the values of the parameters of the models of generalized autoregressive conditional heteroscedasticity have an influence on ANN prediction performance. The combination of the values of the GARCH parameters with the ARIMA autoregressive terms also implies in ANN performance variation. Combining the parameters of the ARIMA-GARCH models and changing the ANN’s topologies, we used the Theil inequality coefficient to measure the prediction of the feed forward ANN.     

Daily prediction of short-term trends of crude oil prices using neural networks exploiting multimarket dynamics

This paper documents a systematic investigation on the predictability of short-term trends of crude oil prices on a daily basis. In stark contrast with longer-term predictions of crude oil prices, short-term prediction with time horizons of 1–3 days posits an important problem that is quite different from what has been studied in the literature. The problem of such short-term predicability is tackled through two aspects. The first is to examine the existence of linear or nonlinear dynamic processes in crude oil prices. This sub-problem is addressed with statistical analysis involving the Brock-Dechert-Scheinkman test for nonlinearity. The second aspect is to test the capability of artificial neural networks (ANN) for modeling the implicit nonlinearity for prediction. Four experimental models are designed and tested with historical data: (1) using only the lagged returns of filtered crude oil prices as input to predict the returns of the next days; this is used as the benchmark, (2) using only the information set of filtered crude oil futures price as input, (3) combining the inputs from the benchmark and second models, and (4) combing the inputs from the benchmark model and the intermarket information. In order to filter out the noise in the original price data, the moving averages of prices are used for all the experiments. The results provided sufficient evidence to the predictability of crude oil prices using ANN with an out-of-sample hit rate of 80%, 70%, and 61% for each of the next three days’ trends.     

Long short-term memory for predicting daily suspended sediment concentration

Frequent and accurate estimation of suspended sediment concentration (SSC) in surface waters and hydraulic schemes is of prime importance for proper design, operation and management of many hydraulic projects. in the present study, a long short-term memory (LSTM) was considered for predicting daily suspended sediment concentration in a river. The LSTM extends recurrent neural network with memory cells, instead of recurrent units, to store and output information, easing the learning of temporal relationships on long time scales. To build the model, daily observed time series of river discharge (Q) and SSC in the Schuylkill River in the United States were used. The results of the proposed model were evaluated and compared with the feedforward neural network and the adaptive neuro fuzzy inference system models which were trained using three different learning algorithms and widely used in the literature for prediction of daily SSC. The comparison of prediction accuracy of the models demonstrated that the LSTM model could satisfactory predict SSC time series, and adequately estimate cumulative suspended sediment load (SSL).

     

Short-term Maharashtra state electrical power load prediction with special emphasis on seasonal changes using a novel focused time lagged recurrent neural network based on time delay neural network model

In this paper, the parameter-wise optimization training process is implemented to achieve an optimal configuration of focused time lagged recurrent neural network (FTLRNN) models by embedding the gamma, laguarre, and multi-channel tapped delay line memory structure. The aim is to examine the prediction ability of the proposed models in order to predict one-day-ahead electric power load simultaneously as usual to oppose 1–24 h forecast in sequel with a special emphasis on seasonal changes over a year. An improved delta-bar-delta algorithm is used to accelerate the training of neural networks and to improve the stability of the convergence.Experimental results indicate that the FTLRNN with time delay neural network (TDNN) clearly outperformed the gamma and laguarre based short-term memory structure in various performance metrics such as mean square error (MSE), normalized MSE, correlation coefficient (r) and mean absolute percentage error (MAPE) during evaluation process. Empirical results show that the proposed dynamic NN model consistently performs well on daily, weekly, and monthly average basis in terms of prediction accuracy. It is noticed from the literature review that an optimally configured FTLRNN with multi-channel tapped delay line memory structure is not currently available to solve short-term electrical power load prediction. The proposed method gives acceptable errors in all seasons, months and on daily basis. The average prediction error on three weeks is obtained as low as 1.67%.     

Comparisons between two types of neural networks for manufacturing cost estimation of piping elements

The objective of this paper is to develop and test a model of manufacturing cost estimating of piping elements during the early design phase through the application of artificial neural networks (ANN). The developed model can help designers to make decisions at the early phases of the design process. An ANN model would allow obtaining a fairly accurate prediction, even when enough and adequate information is not available in the early stages of the design process. The developed model is compared with traditional neural networks and conventional regression models. This model proved that neural networks are capable of reducing uncertainties related to the cost estimation of shell and tube heat exchangers.     

Performance of radial basis and LM-feed forward artificial neural networks for predicting daily watershed runoff

This study investigated the effects of upstream stations’ flow records on the performance of artificial neural network (ANN) models for predicting daily watershed runoff. As a comparison, a multiple linear regression (MLR) analysis was also examined using various statistical indices. Five streamflow measuring stations on the Cahaba River, Alabama, were selected as case studies. Two different ANN models, multi layer feed forward neural network using Levenberg–Marquardt learning algorithm (LMFF) and radial basis function (RBF), were introduced in this paper. These models were then used to forecast one day ahead streamflows. The correlation analysis was applied for determining the architecture of each ANN model in terms of input variables. Several statistical criteria (RMSE, MAE and coefficient of correlation) were used to check the model accuracy in comparison with the observed data by means of K-fold cross validation method. Additionally, residual analysis was applied for the model results. The comparison results revealed that using upstream records could significantly increase the accuracy of ANN and MLR models in predicting daily stream flows (by around 30%). The comparison of the prediction accuracy of both ANN models (LMFF and RBF) and linear regression method indicated that the ANN approaches were more accurate than the MLR in predicting streamflow dynamics. The LMFF model was able to improve the average of root mean square error (RMSE_ave) and average of mean absolute percentage error (MAPE_ave) values of the multiple linear regression forecasts by about 18% and 21%, respectively. In spite of the fact that the RBF model acted better for predicting the highest range of flow rate (flood events, RMSE_ave/RBF = 26.8 m³/s vs. RMSE_ave/LMFF = 40.2 m³/s), in general, the results suggested that the LMFF method was somehow superior to the RBF method in predicting watershed runoff (RMSE/LMFF = 18.8 m³/s vs. RMSE/RBF = 19.2 m³/s). Eventually, statistical differences between measured and predicted medians were evaluated using Mann-Whitney test, and differences in variances were evaluated using the Levene's test.     

Division-based rainfall-runoff simulations with BP neural networks and Xinanjiang model

The application of artificial neural network (ANN) to rainfall-runoff simulations has provided promising results in recent years. However, it is difficult to obtain satisfying results by using raw data for the direct prediction of the time series of streamflows. To improve simulating daily streamflow with back-propagation (BP) neural networks, the whole data set in this study is divided into two independent groups, flood period and non-flood period. The approaches and techniques of applying the division-based BP (DBP) in runoff simulation are presented in this paper. A comparison of the DBP model to the primitive BP model and the Xinanjiang model was also conducted to evaluate the effectiveness of the improvement. The numerical experimental results indicate that DBP model still overestimated flow peak, but improved considerably the streamflow simulation in the non-flood period.     

Performance of artificial neural networks in nearshore wave power prediction

In this paper the assessment of the wave energy potential in nearshore coastal areas is investigated by means of artificial neural networks (ANNs). The performance of the ANNs is compared with in situ measurements and spectral numerical modelling (the conventional tool for wave energy assessment). For this purpose, 13 years of records of two buoys, one offshore and one inshore, with an hourly frequency are used to develop an ANN model for predicting the nearshore wave power. The best suited architecture was selected after assessing the performance of 480 ANN models involving twelve different architectures. The results predicted by the ANN model were compared with the measured data and those obtained by means of the SWAN (Simulating Waves Nearshore) spectral model. The quality in the predictions of the ANN model shows that this type of artificial intelligence models constitutes a powerful tool to forecast the wave energy potential at particular coastal site with great accuracy, and one that overcomes some of the disadvantages of the conventional tools for nearshore wave power prediction.