Modifying genetic programming for artificial neural network development for data mining

The development of artificial neural networks (ANNs) is usually a slow process in which the human expert has to test several architectures until he finds the one that achieves best results to solve a certain problem. However, there are some tools that provide the ability of automatically developing ANNs, many of them using evolutionary computation (EC) tools. One of the main problems of these techniques is that ANNs have a very complex structure, which makes them very difficult to be represented and developed by these tools. This work presents a new technique that modifies genetic programming (GP) so as to correctly and efficiently work with graph structures in order to develop ANNs. This technique also allows the obtaining of simplified networks that solve the problem with a small group of neurons. In order to measure the performance of the system and to compare the results with other ANN development methods by means of evolutionary computation (EC) techniques, several tests were performed with problems based on some of the most used test databases in the Data Mining domain. These comparisons show that the system achieves good results that are not only comparable to those of the already existing techniques but, in most cases, improve them.     

Fuzzy reliability analysis of repairable industrial systems using soft-computing based hybridized techniques

The purpose of the present study is to analyze the fuzzy reliability of a repairable industrial system utilizing historical vague, imprecise and uncertain data which reflects its components’ failure and repair pattern. Soft-computing based two different hybridized techniques named as Genetic Algorithms Based Lambda–Tau (GABLT) and Neural Network and Genetic Algorithms Based Lambda–Tau (NGABLT) along with a traditional Fuzzy Lambda–Tau (FLT) technique are used to evaluate some important reliability indices of the system in the form of fuzzy membership functions. As a case study, all the three techniques are applied to analyse the fuzzy reliability of the washing system in a paper mill and results are compared. Sensitivity analysis has also been performed to analyze the effect of variation of different reliability parameters on system performance. The analysis can help maintenance personnel to understand and plan suitable maintenance strategy to improve the overall performance of the system. Based on results some important suggestions are given for future course of action in maintenance planning.     

Classification of the cardiotocogram data for anticipation of fetal risks using machine learning techniques

The aim of the research is evaluating the classification performances of eight different machine-learning methods on the antepartum cardiotocography (CTG) data. The classification is necessary to predict newborn health, especially for the critical cases. Cardiotocography is used for assisting the obstetricians’ to obtain detailed information during the pregnancy as a technique of measuring fetal well-being, essentially in pregnant women having potential complications. The obstetricians describe CTG shortly as a continuous electronic record of the baby's heart rate took from the mother's abdomen. The acquired information is necessary to visualize unhealthiness of the embryo and gives an opportunity for early intervention prior to happening a permanent impairment to the embryo. The aim of the machine learning methods is by using attributes of data obtained from the uterine contraction (UC) and fetal heart rate (FHR) signals to classify as pathological or normal. The dataset contains 1831 instances with 21 attributes, examined by applying the methods. In the paper, the highest accuracy displayed as 99.2%.     

On using reservoir computing for sensing applications: exploring environment-sensitive memristor networks

Abstract

Recently, the SWEET sensing setup has been proposed as a way of exploiting reservoir computing for sensing. The setup features three components: an input signal (the drive), the environment and a reservoir, where the reservoir and the environment are treated as one dynamical system, a super-reservoir. Due to the reservoir-environment interaction, the information about the environment is encoded in the state of the reservoir. This information can be inferred (decoded) by analysing the reservoir state. The decoding is done by using an external drive signal. This signal is optimised to achieve a separation in the space of the reservoir states: Under different environmental conditions, the reservoir should visit distinct regions of the configuration space. We examined this approach theoretically by using an environment-sensitive memristor as a reservoir, where the memristance is the state variable. The goal has been to identify a suitable drive that can achieve the phase space separation, which was formulated as an optimization problem, and solved by a genetic optimization algorithm developed in this study. For simplicity reasons, only two environmental conditions were considered (describing a static and a varying environment). A suitable drive signal has been identified based on intuitive analysis of the memristor dynamics, and by solving the optimization problem. Under both drives the memristance is driven to two different regions of the one-dimensional state space under the influence of the two environmental conditions, which can be used to infer about the environment. The separation occurs if there is a synchronisation between the drive and the environmental signals. To quantify the magnitude of the separation, we introduced a quality of sensing index: The ability to sense depends critically on the synchronisation between the drive and environmental conditions. If this synchronisation is not maintained the quality of sensing deteriorates.     

A comparison of classification accuracy of four genetic programming-evolved intelligent structures

We investigate the effectiveness of GP-generated intelligent structures in classification tasks. Specifically, we present and use four context-free grammars to describe (1) decision trees, (2) fuzzy rule-based systems, (3) feedforward neural networks and (4) fuzzy Petri-nets with genetic programming. We apply cellular encoding in order to express feedforward neural networks and fuzzy Petri-nets with arbitrary size and topology. The models then are examined thoroughly in six well-known real world data sets. Results are presented in detail and the competitive advantages and drawbacks of the selected methodologies are discussed, in respect to the nature of each application domain. Conclusions are drawn on the effectiveness and efficiency of the presented approach.     

Evaluating the sources of water to wells: Three techniques for metamodeling of a groundwater flow model

For decision support, the insights and predictive power of numerical process models can be hampered by insufficient expertise and computational resources required to evaluate system response to new stresses. An alternative is to emulate the process model with a statistical “metamodel.” Built on a dataset of collocated numerical model input and output, a groundwater flow model was emulated using a Bayesian Network, an Artificial neural network, and a Gradient Boosted Regression Tree. The response of interest was surface water depletion expressed as the source of water-to-wells. The results have application for managing allocation of groundwater. Each technique was tuned using cross validation and further evaluated using a held-out dataset. A numerical MODFLOW-USG model of the Lake Michigan Basin, USA, was used for the evaluation. The performance and interpretability of each technique was compared pointing to advantages of each technique. The metamodel can extend to unmodeled areas.     

Artificial neural network-based techniques for the retrieval of SWE and snow depth from SSM/I data

The retrieval of snow water equivalent (SWE) and snow depth is performed by inverting Special Sensor Microwave Imager (SSM/I) brightness temperatures at 19 and 37 GHz using artificial neural network ANN-based techniques. The SSM/I used data, which consist of Pathfinder Daily EASE-Grid brightness temperatures, were supplied by the National Snow and Ice Data Centre (NSIDC). They were gathered during the period of time included between the beginning of 1996 and the end of 1999 all over Finland. A ground snow data set based on observations of the Finnish Environment Institute (SYKE) and the Finnish Meteorological Institute (FMI) was used to estimate the performances of the technique. The ANN results were confronted with those obtained using the spectral polarization difference (SPD) algorithm, the HUT model-based iterative inversion and the Chang algorithm, by comparing the RMSE, the R², and the regression coefficients. In general, it was observed that the results obtained through ANN-based technique are better than, or comparable to, those obtained through other approaches, when trained with simulated data. Performances were very good when the ANN were trained with experimental data.     

Investigating the relationship between adverse events and infrastructure development in an active war theater using soft computing techniques

The purpose of this paper is to investigate the relationship between adverse events and infrastructure development investments in an active war theater by using soft computing techniques including fuzzy inference systems (FIS), artificial neural networks (ANNs), and adaptive neuro-fuzzy inference systems (ANFIS) where the accuracy of the predictions is directly beneficial from an economic and humanistic point of view. Fourteen developmental and economic improvement projects were selected as independent variables. A total of four outputs reflecting the adverse events in terms of the number of people killed, wounded or hijacked, and the total number of adverse events has been estimated.The results obtained from analysis and testing demonstrate that ANN, FIS, and ANFIS are useful modeling techniques for predicting the number of adverse events based on historical development or economic project data. When the model accuracy was calculated based on the mean absolute percentage error (MAPE) for each of the models, ANN had better predictive accuracy than FIS and ANFIS models, as demonstrated by experimental results. For the purpose of allocating resources and developing regions, the results can be summarized by examining the relationship between adverse events and infrastructure development in an active war theater, with emphasis on predicting the occurrence of events. We conclude that the importance of infrastructure development projects varied based on the specific regions and time period.     

Evaluation of prediction capability, robustness, and sensitivity in non-linear landslide susceptibility models, Guantánamo, Cuba

This paper describes a procedure for landslide susceptibility assessment based on artificial neural networks, and focuses on the estimation of the prediction capability, robustness, and sensitivity of susceptibility models. The study is carried out in the Guantanamo Province of Cuba, where 186 landslides were mapped using photo-interpretation. Twelve conditioning factors were mapped including geomorphology, geology, soils, landuse, slope angle, slope direction, internal relief, drainage density, distance from roads and faults, rainfall intensity, and ground peak acceleration.A methodology was used that subdivided the database in 3 subsets. A training set was used for updating the weights. A validation set was used to stop the training procedure when the network started losing generalization capability, and a test set was used to calculate the performance of the network. A 10-fold cross-validation was performed in order to show that the results are repeatable. The prediction capability, the robustness analysis, and the sensitivity analysis were tested on 10 mutually exclusive datasets. The results show that by means of artificial neural networks it is possible to obtain models with high prediction capability and high robustness, and that an exploration of the effect of the individual variables is possible, even if they are considered as a black-box model.     

Application of integrated soft computing techniques for optimisation of hybrid CO2 laser–MIG welding process

In this paper, artificial neural networks (ANNs), genetic algorithm (GA), simulated annealing (SA) and Quasi Newton line search techniques have been combined to develop three integrated soft computing based models such as ANN–GA, ANN–SA and ANN–Quasi Newton for prediction modelling and optimisation of welding strength for hybrid CO₂ laser–MIG welded joints of aluminium alloy. Experimental dataset employed for the purpose has been generated through full factorial experimental design. Laser power, welding speeds and wires feed rate are considered as controllable input parameters. These soft computing models employ a trained ANN for calculation of objective function value and thereby eliminate the need of closed form objective function. Among 11 tested networks, the ANN with best prediction performance produces maximum percentage error of only 3.21%. During optimisation ANN–GA is found to show best performance with absolute percentage error of only 0.09% during experimental validation. Low value of percentage error indicates efficacy of models. Welding speed has been found as most influencing factor for welding strength.     

An evolutionary gray gradient algorithm for multilevel thresholding of brain MR images using soft computing techniques

The conventional two dimensional (2-D) histogram based Otsu’s method gives unreliable results while considering multilevel thresholding of brain magnetic resonance (MR) images, because the edges of the brain regions are not preserved due to the local averaging process involved. Moreover, some of the useful pixels present inside the off-diagonal regions are ignored in the calculation. This article presents an evolutionary gray gradient algorithm (EGGA) for optimal multilevel thresholding of brain MR images. In this paper, more edge information is preserved by computing 2-D histogram based gray gradient. The key to our success is the use of the gray gradient information between the pixel values and the pixel average values to minimize the information loss. In addition, the speed improvement is achieved. Theoretical formulations are derived for computing the maximum between class variance from the 2-D histogram of the brain image. A first-hand fitness function is suggested for the EGGA. A novel adaptive swallow swarm optimization (ASSO) algorithm is introduced to optimize the fitness function. The performance of ASSO is validated using twenty three standard Benchmark test functions. The performance of ASSO is better than swallow swarm optimization (SSO). The optimum threshold value is obtained by maximizing the between class variance using ASSO. Our method is tested using the standard axial T₂ − weighted brain MRI database of Harvard medical education using 100 slices. Performance of our method is compared to the Otsu’s method based on the one dimensional (1-D) and the 2-D histogram. The results are also compared among four different soft computing techniques. It is observed that results obtained using our method is better than the other methods, both qualitatively and quantitatively. Benefits of our method are – (i) the EGGA exhibits better objective function values; (ii) the EGGA provides us significantly improved results; and (iii) more computational speed is achieved.     

Bio-inspired computational heuristics to study the boundary layer flow of the Falkner-Scan system with mass transfer and wall stretching

In the present study, a bio-inspired computational intelligence technique is developed for finding the solutions of the celebrated Falkner-Skan equation arising in fluid mechanics problems using feed-forward Artificial Neural Networks (ANNs), Genetic Algorithms (GA), the Active-Set (AS) method and their combination namely a GA-AS approach. The differential equations based ANNs modeling of the Falkner-Skan system is constructed by defining an unsupervised error function. The training of the design parameters of ANNs is carried out with the help of viable global search through GAs and fine tuning of the results is achieved with an efficient local search using the AS method. The proposed scheme is applied to a number of scenarios for the Falkner-Skan system based on boundary layer flow over a moving wall with mass transfer in the presence of a free stream with power-law velocity distributions. The dynamics of the system are investigated for different cases of mass transfer and wall stretching. The proposed results are compared with analytical and numerical solutions to verify the correctness of the approach. The accuracy and convergence of the proposed solver are validated through sufficient large numbers of independent runs in terms of different performance indices based on mean absolute error, Thail’s inequality coefficient and Nash-Suitcliff efficiency. These solutions greatly enrich possible approaches for stochastic numerical solution of the celebrated Falkner-Skan system.     

Optimized scenario for rainfall forecasting using genetic algorithm coupled with artificial neural network

Rainfall forecasting plays many important role in water resources studies such as river training works and design of flood warning systems. Recent advancement in artificial intelligence and in particular techniques aimed at converting input to output for highly nonlinear, non-convex and dimensionalized processes such as rainfall field, provide an alternative approach for developing rainfall forecasting model. Artificial neural networks (ANNs), which perform a nonlinear mapping between inputs and outputs, are such a technique. Current literatures on artificial neural networks show that the selection of network architecture and its efficient training procedure are major obstacles for their daily usage. In this paper, feed-forward type networks will be developed to simulate the rainfall field and a so-called back propagation (BP) algorithm coupled with genetic algorithm (GA) will be used to train and optimize the networks. The technique will be implemented to forecast rainfall for a number of times using rainfall hyetograph of recording rain gauges in the Upper Parramatta catchment in the western suburbs of Sydney, Australia. Results of the study showed the structuring of ANN network with the input parameter selection, when coupled with GA, performed better compared to similar work of using ANN alone.     

Reliable numerical treatment of nonlinear singular Flierl–Petviashivili equations for unbounded domain using ANN,GAs, and SQP

In this paper, a novel intelligent computational approach is developed for finding the solution of nonlinear singular system governed by boundary value problems of Flierl–Petviashivili equations using artificial neural networks optimized with genetic algorithms, sequential quadratic programming technique, and their combinations. The competency of artificial neural network for universal function approximation is exploited in formulation of mathematical modelling of the equation based on an unsupervised error with specialty of satisfying boundary conditions at infinity. The training of the weights of the networks is carried out with memetic computing based on genetic algorithm used as a tool for reliable global search method, hybridized with sequential quadratic programming technique used as a tool for rapid local convergence. The proposed scheme is evaluated on three variants of the two boundary problems by taking different values of nonlinearity operators and constant coefficients. The reliability and effectiveness of the design approaches are validated through the results of statistical analyses based on sufficient large number of independent runs in terms of accuracy, convergence, and computational complexity. Comparative studies of the proposed results are made with state of the art analytical solvers, which show a good agreement mostly and even better in few cases as well. The intrinsic worth of the schemes is simplicity in the concept, ease in implementation, to avoid singularity at origin, to deal with strong nonlinearity effectively, and their ability to handle exactly traditional initial conditions along with boundary condition at infinity.     

Prediction of elastic compressibility of rock material with soft computing techniques

Mechanical and physical properties of sandstone are interesting scientifically and have great practical significance as well as their relations to the mineralogy and pore features. These relations are however highly nonlinear and cannot be easily formulated by conventional methods. This paper investigates the potential of the technique named as the relevance vector machine (RVM) for prediction of the elastic compressibility of sandstone based on its characteristics of physical properties. Based on the fact that the hyper-parameters may have effects on the RVM performance, an iteration method is proposed in this paper to search for optimal hyper-parameter value so that it can produce best predictions. Also, the qualitative sensitivity of the physical properties is investigated by the backward regression analysis. Meanwhile, the hyper-parameter effect of the RVM approach is discussed in the prediction of the elastic compressibility of sandstone. The predicted results of the RVM demonstrate that hyper-parameter values have evident effects on the RVM performance. Comparisons on the results of the RVM, the artificial neural network and the support vector machine prove that the proposed strategy is feasible and reliable for prediction of the elastic compressibility of sandstone based on its physical properties.     

A genetic algorithm optimized Morlet wavelet artificial neural network to study the dynamics of nonlinear Troesch’s system

In this work, a new stochastic computing technique is developed to study the nonlinear dynamics of Troesch’s problem by designing the mathematical models of Morlet Wavelets Artificial Neural Networks (MW-ANNs) optimized with Genetic Algorithm (GA) integrated with Sequential Quadratic Programming (SQP). The differential equation mathematical model for MW-ANNs are designed for Troesch’s system by incorporating a windowing kernel based on Morlet Wavelets as an activation function and these networks are constructed to define a fitness function of Troesch’s system in the mean squared sense. The unknown adjustable parameters of MW-ANNs are trained initially by an effective global search using GAs hybridized with SQP for rapid local refinement of the results. The proposed scheme is evaluated to solve the Troesch’s problems for small and large values of the critical parameter in the system. Comparison of the proposed results with standard reference solutions of Adams method shows good agreement. Validation of accuracy and convergence of the proposed scheme is made using statistical analysis based on a sufficiently large number of independent runs, this is done in terms of performance measures of mean absolute deviation and root mean squared error.     

Fusion of soft and hard computing: multi-dimensional categorization of computationally intelligent hybrid systems

The concept of fusion of soft computing and hard computing has rapidly gained importance over the last few years. Soft computing is known as a complementary set of techniques such as neural networks, fuzzy systems, or evolutionary computation which are able to deal with uncertainty, partial truth, and imprecision. Hard computing, i.e., the huge set of traditional techniques, is usually seen as the antipode of soft computing. Fusion of soft and hard computing techniques aims at exploiting the particular advantages of both realms. This article introduces a multi-dimensional categorization scheme for fusion techniques and applies it by analyzing several fusion techniques where the soft computing part is realized by a neural network. The categorization scheme facilitates the discussion of advantages or drawbacks of certain fusion approaches, thus supporting the development of novel fusion techniques and applications.     

Evaluating evapotranspiration and water-use efficiency of terrestrial ecosystems in the conterminous United States using MODIS and AmeriFlux data

In this study, we used the remotely-sensed data from the Moderate Resolution Imaging Spectrometer (MODIS), meteorological and eddy flux data and an artificial neural networks (ANNs) technique to develop a daily evapotranspiration (ET) product for the period of 2004-2005 for the conterminous U.S. We then estimated and analyzed the regional water-use efficiency (WUE) based on the developed ET and MODIS gross primary production (GPP) for the region. We first trained the ANNs to predict evapotranspiration fraction (EF) based on the data at 28 AmeriFlux sites between 2003 and 2005. Five remotely-sensed variables including land surface temperature (LST), normalized difference vegetation index (NDVI), normalized difference water index (NDWI), leaf area index (LAI) and photosynthetically active radiation (PAR) and ground-measured air temperature and wind velocity were used. The daily ET was calculated by multiplying net radiation flux derived from remote sensing products with EF. We then evaluated the model performance by comparing modeled ET with the data at 24 AmeriFlux sites in 2006. We found that the ANNs predicted daily ET well (R² = 0.52-0.86). The ANNs were applied to predict the spatial and temporal distributions of daily ET for the conterminous U.S. in 2004 and 2005. The ecosystem WUE for the conterminous U.S. from 2004 to 2005 was calculated using MODIS GPP products (MOD17) and the estimated ET. We found that all ecosystems' WUE-drought relationships showed a two-stage pattern. Specifically, WUE increased when the intensity of drought was moderate; WUE tended to decrease under severe drought. These findings are consistent with the observations that WUE does not monotonously increase in response to water stress. Our study suggests a new water-use efficiency mechanism should be considered in ecosystem modeling. In addition, this study provides a high spatial and temporal resolution ET dataset, an important product for climate change and hydrological cycling studies for the MODIS era.     

The effects of scheduling, workload type and consolidation scenarios on virtual machine performance and their prediction through optimized artificial neural networks

The aim of this paper is to study and predict the effect of a number of critical parameters on the performance of virtual machines (VMs). These parameters include allocation percentages, real-time scheduling decisions and co-placement of VMs when these are deployed concurrently on the same physical node, as dictated by the server consolidation trend and the recent advances in the Cloud computing systems. Different combinations of VM workload types are investigated in relation to the aforementioned factors in order to find the optimal allocation strategies. What is more, different levels of memory sharing are applied, based on the coupling of VMs to cores on a multi-core architecture. For all the aforementioned cases, the effect on the score of specific benchmarks running inside the VMs is measured. Finally, a black box method based on genetically optimized artificial neural networks is inserted in order to investigate the degradation prediction ability a priori of the execution and is compared to the linear regression method.     

Modeling the permeability of carbonate reservoir using type-2 fuzzy logic systems

In this work, the use of type-2 fuzzy logic systems as a novel approach for predicting permeability from well logs has been investigated and implemented. Type-2 fuzzy logic system is good in handling uncertainties, including uncertainties in measurements and data used to calibrate the parameters. In the formulation used, the value of a membership function corresponding to a particular permeability value is no longer a crisp value; rather, it is associated with a range of values that can be characterized by a function that reflects the level of uncertainty. In this way, the model will be able to adequately account for all forms of uncertainties associated with predicting permeability from well log data, where uncertainties are very high and the need for stable results are highly desirable. Comparative studies have been carried out to compare the performance of the proposed type-2 fuzzy logic system framework with those earlier used methods, using five different industrial reservoir data. Empirical results from simulation show that type-2 fuzzy logic approach outperformed others in general and particularly in the area of stability and ability to handle data in uncertain situations, which are common characteristics of well logs data. Another unique advantage of the newly proposed model is its ability to generate, in addition to the normal target forecast, prediction intervals as its by-products without extra computational cost.