Neural-network metamodelling for the prediction of Caulerpa taxifolia development in the Mediterranean sea

This paper addresses the use of neural networks as a metamodelling technique for discrete event stochastic simulation to reduce significantly the computational burden involved by the simulations. A sophisticated computer model has been developed to anticipate the propagation of the green alga Caulerpa taxifolia in the northwestern Mediterranean sea. The simulation model provides reliable predictions, a couple of years in advance, of the covered surfaces. To reduce the heavy computational burden involved by the simulation, a neural network was successfully trained on artificially generated data provided by the simulation runs to provide accurate forecasts 12 years in advance, along with associated confidence intervals. The neural-network metamodel is competitive in accuracy when compared to the simulation itself and, once trained, can operate in nearly real time.     

Web traffic demand forecasting using wavelet‐based multiscale decomposition

In this paper we propose an experimental forecasting strategy taking into account the long‐range dependence of aggregate network traffic, and we apply it to provide one‐minute‐ahead World‐Wide Web (Web) traffic demand forecasts in terms of average number of bytes transferred. Recently, statistical examination of Web traces have shown evidence that Web traffic arising from file transfers exhibits a behavior that is consistent with the notion of self‐similarity. Essentially, self‐similarity indicates that significant burstiness is present on a wide range of time scales (i.e., the process is long‐range dependent). Hence the idea of exploiting this multiscale property with a view towards discovering and capturing regularities underlying the time series which may prove useful for short‐term traffic load forecasting. We carry out a wavelet transform decomposition of the original series to decompose the traffic time series into varying scales of temporal resolution, with the aim of making the underlying temporal structures more tractable. In a second step, each individual wavelet series—supposed to capture some features of the series—is fitted with a dynamical recurrent neural network (DRNN) model to output the wavelet forecast. The latter are afterwards recombined to form the next‐minute Web Traffic demand. The method is applied on a large set of HTTP logs and is shown to yield good results. © 2001 John Wiley & Sons, Inc.     

An extensive empirical comparison of ensemble learning methods for binary classification

We present an extensive empirical comparison between nineteen prototypical supervised ensemble learning algorithms, including Boosting, Bagging, Random Forests, Rotation Forests, Arc-X4, Class-Switching and their variants, as well as more recent techniques like Random Patches. These algorithms were compared against each other in terms of threshold, ranking/ordering and probability metrics over nineteen UCI benchmark data sets with binary labels. We also examine the influence of two base learners, CART and Extremely Randomized Trees, on the bias–variance decomposition and the effect of calibrating the models via Isotonic Regression on each performance metric. The selected data sets were already used in various empirical studies and cover different application domains. The source code and the detailed results of our study are publicly available.     

Unsupervised feature selection with ensemble learning

Machine Learning - In this paper, we show that the way internal estimates are used to measure variable importance in Random Forests are also applicable to feature selection in unsupervised...     

Sufficient conditions for error backflow convergence in dynamical recurrent neural networks

This article extends previous analysis of the gradient decay to a class of discrete-time fully recurrent networks, called dynamical recurrent neural networks, obtained by modeling synapses as finite impulse response (FIR) filters instead of multiplicative scalars. Using elementary matrix manipulations, we provide an upper bound on the norm of the weight matrix, ensuring that the gradient vector, when propagated in a reverse manner in time through the error-propagation network, decays exponentially to zero. This bound applies to all recurrent FIR architecture proposals, as well as fixed-point recurrent networks, regardless of delay and connectivity. In addition, we show that the computational overhead of the learning algorithm can be reduced drastically by taking advantage of the exponential decay of the gradient.     

Building bagging on critical instances

The ensemble method is a powerful data mining paradigm, which builds a classification model by integrating multiple diversified component learners. Bagging is one of the most successful ensemble methods. It is made of bootstrap-inspired classifiers and uses these classifiers to get an aggregated classifier. However, in bagging, bootstrapped training sets become more and more similar as redundancy is increasing. Besides redundancy, any training set is usually subject to noise. Moreover, the training set might be imbalanced. Thus, each training instance has a different impact on the learning process. This paper explores some properties of the ensemble margin and its use in improving the performance of bagging. We introduce a new approach to measure the importance of training data in learning, based on the margin theory. Then, a new bagging method concentrating on critical instances is proposed. This method is more accurate than bagging and more robust than boosting. Compared to bagging, it reduces the bias while generally keeping the same variance. Our findings suggest that (a) examples with low margins tend to be more critical for the classifier performance; (b) examples with higher margins tend to be more redundant; (c) misclassified examples with high margins tend to be noisy examples. Our experimental results on 15 various data sets show that the generalization error of bagging can be reduced up to 2.5% and its resilience to noise strengthened by iteratively removing both typical and noisy training instances, reducing the training set size by up to 75%.     

Neural network modelling for environmental prediction

We describe the choice and assessment of neural network and statistical methods for data modelling, feature selection and forecasting. We deal in particular with how empirical environmental and Earth observation data can be used in conjunction with physical simulation models.