Drought estimation with neural networks

An artificial neural network model is presented to derive streamflow precipitation data. It is tested with actual data coming from a nearby river, referred to a basin area of 356 km² and a time period of 11 years. A feedforward multilayer perception with linear output has been built to deal with this problem. The dynamics are caught by the filter structure of the input layer.

A special study on crossing properties, based on training sample selection,is made to measure the performance of the network for drought analysis. Sample selection leads to increased accuracy within the sample range and degraded performance for points that are clearly out. Predicted number of droughts, average drought length and deficit are compared with the actual data. The results show that very simple neural network models can give fine results.     

Confidence estimation methods for neural networks: a practicalcomparison

Feedforward neural networks, particularly multilayer perceptrons, are widely used in regression and classification tasks. A reliable and practical measure of prediction confidence is essential. In this work three alternative approaches to prediction confidence estimation are presented and compared. The three methods are the maximum likelihood, approximate Bayesian, and the bootstrap technique. We consider prediction uncertainty owing to both data noise and model parameter misspecification. The methods are tested on a number of controlled artificial problems and a real, industrial regression application, the prediction of paper "curl". Confidence estimation performance is assessed by calculating the mean and standard deviation of the prediction interval coverage probability. We show that treating data noise variance as a function of the inputs is appropriate for the curl prediction task. Moreover, we show that the mean coverage probability can only gauge confidence estimation performance as an average over the input space, i.e., global performance and that the standard deviation of the coverage is unreliable as a measure of local performance. The approximate Bayesian approach is found to perform better in terms of global performance.     

Speech recognition with artificial neural networks

In this paper, artificial neural networks were used to accomplish isolated speech recognition. The topic was investigated in two steps, consisting of the pre-processing part with Digital Signal Processing (DSP) techniques and the post-processing part with Artificial Neural Networks (ANN). These two parts were briefly explained and speech recognizers using different ANN architectures were implemented on Matlab. Three different neural network models; multi layer back propagation, Elman and probabilistic neural networks were designed. Performance comparisons with similar studies found in the related literature indicated that our proposed ANN structures yield satisfactory results.     

Empirical prediction limit estimation methods for feed-forward neural networks

Feed-forward neural networks (FFNs) have gained a lot of interest in the last decade as empirical models for their powerful representational capacity, non-parametric nature and multivariate characteristics. While these neural network models focus primarily on accurate prediction of output values, often outperforming their statistical counterparts in dealing with sparse date sets, they usually do not provide any information regarding the confidence with which they make these predictions. Since prediction limits (PLs) indicate the extent to which one can rely on predictions for making future decisions, it is of paramount importance to estimate these limits. Two empirical PL estimation methods for FFNs are presented. The two methods differ in one fundamental aspect: the method employed for modeling the properties of the neural network model residuals. While one method uses a local approximation scheme, the other utilizes a global approximation scheme. Simulation results reveal that both methods have their relative strengths and weaknesses.     

Compartmental modelling with artificial neural networks

In this paper, we introduce an abbreviated compartmental modelling scheme which may be of interest to those in neuron- based adaptive systems because of the additional scope it provides for studying biologically-inspired learning mechanisms. The scheme, although not as flexible and precise as the general compartmental approach, allows one to design Hodgkin-Huxley style cells, and passive dendritic trees with an arbitrary number of synaptic connections. The trade-offs made for computational performance, may make the modelling scheme more appropriate for practical applications. The modelling scheme is based upon artificial neural networks, which we have used to represent cylindrical compartments (both passive and active) of different lengths, two types of voltage-dependent channels, and basic chemical synapses with variable time constants.     

Multi-objective design of post-tensioned concrete road bridges using artificial neural networks

In order to minimize the total expected cost, bridges have to be designed for safety and durability. This paper considers the cost, the safety, and the corrosion initiation time to design post-tensioned concrete box-girder road bridges. The deck is modeled by finite elements based on problem variables such as the cross-section geometry, the concrete grade, and the reinforcing and post-tensioning steel. An integrated multi-objective harmony search with artificial neural networks (ANNs) is proposed to reduce the high computing time required for the finite-element analysis and the increment in conflicting objectives. ANNs are trained through the results of previous bridge performance evaluations. Then, ANNs are used to evaluate the constraints and provide a direction towards the Pareto front. Finally, exact methods actualize and improve the Pareto set. The results show that the harmony search parameters should be progressively changed in a diversification-intensification strategy. This methodology provides trade-off solutions that are the cheapest ones for the safety and durability levels considered. Therefore, it is possible to choose an alternative that can be easily adjusted to each need.     

Rule-based artificial neural networks

Neural networks that are integrated with rule-based systems having a knowledge base offer more capabilities than networks not integrated with such systems.     

Knowledge-based artificial neural networks

Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN (Knowledge-Based Artificial Neural Networks) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.     

Extracting reduced logic programs from artificial neural networks

Artificial neural networks can be trained to perform excellently in many application areas. Whilst they can learn from raw data to solve sophisticated recognition and analysis problems, the acquired knowledge remains hidden within the network architecture and is not readily accessible for analysis or further use: Trained networks are black boxes. Recent research efforts therefore investigate the possibility to extract symbolic knowledge from trained networks, in order to analyze, validate, and reuse the structural insights gained implicitly during the training process. In this paper, we will study how knowledge in form of propositional logic programs can be obtained in such a way that the programs are as simple as possible—where simple is being understood in some clearly defined and meaningful way.     

Atrial fibrillation classification with artificial neural networks

The acquired 72 normal sinus rhythm ECGs and 80 ECGs with atrial fibrillation (AF) are decomposed with ‘db10’ Daebauchies wavelets at level 6 and power spectral density was calculated for each decomposed signal with Welch method. Average power spectral density was calculated for six subbands and normalized to be used as input to the neural network. Levenberg-Marquart backpropagation feed forward neural network was built from logarithmic sigmoid transfer functions in three-layer form. The trained network was tested on 24 normal and 28 AF state ECGs. The classification performance was accomplished as 100% accurate.     

Ear recognition with feed-forward artificial neural networks

Biometrics has become one of the most important techniques in recognizing a person’s identity. A person’s face, iris and fingerprint are mostly used in biometrics today. It has been established that there are no two ears exactly alike, even in the cases of identical twins. In this paper, we define a 7-element ear feature set and design and train a feed-forward artificial neural network to recognize a human ear. We train and test the network with 51 ear pictures from 51 different persons. Simulation experiments with various networks with various number of layers and number of neurons per layer and with and without noise are conducted. Results indicate that a 95 % ear recognition accuracy is achieved with a simple 3-layer feed-forward neural network with only a total of 18 neurons even in the presence of some noise. This design outstands previous work in simplicity and implementation cost.     

Globally optimal bounding ellipsoid algorithm for parameter estimation using artificial neural networks

This paper develops a real-time implementation of a globally optimal bounding ellipsoid (GOBE) algorithm for parameter estimation of linear-in-parameter models with unknown but bounded (UBB)errors. A recently proposed recursively optimal bounding ellipsoid (ROBE) algorithm is introduced, and a GOBE algorithm is derived through repeating this ROBE algorithm. An analogue artificial neural network (ANN) is provided to implement the GOBE algorithm in real time. Convergence analyses on the ROBE, the GOBE algorithms, and the analogue ANN implementation of the GOBE algorithm are presented. No persistent excitation condition is required to ensure the convergence. Simulation results show the good performances of these algorithms and the ANN implementation.     

Soil-moisture estimation from TerraSAR-X data using neural networks

Early prediction of natural disasters like floods and landslides is essential for reasons of public safety. This can be attained by processing Synthetic-Aperture Radar (SAR) images and retrieving soil-moisture parameters. In this article, TerraSAR-X product images are investigated in combination with a water-cloud model based on the Shi semi-empirical model to determine the accuracy of soil-moisture parameter retrieval. SAR images were captured between January 2008 and September 2010 in the vicinity of the city Maribor, Slovenia, at different incidence angles. The water-cloud model provides acceptable estimated soil-moisture parameters at bare or scarcely vegetated soil areas. However, this model is too sensitive to speckle noise; therefore, a pre-processing step for speckle-noise reduction is carried out. Afterwards, self-organizing neural networks (SOM) are used to segment the areas at which the performance of this model is poor, and at the same time neural networks are also used for a more accurate approximation of model parameters’ values. Ground-truth is measured using the Pico64 sensor located on the field, simultaneously with capturing SAR images, in order to enable the comparison and validation of the obtained results. Experimental results show that the proposed method outperforms the water-cloud model accuracy over all incidence angles.     

Logistic model tree extraction from artificial neural networks.

Artificial neural networks (ANNs) are a powerful and widely used pattern recognition technique. However, they remain "black boxes" giving no explanation for the decisions they make. This paper presents a new algorithm for extracting a logistic model tree (LMT) from a neural network, which gives a symbolic representation of the knowledge hidden within the ANN. Landwehr's LMTs are based on standard decision trees, but the terminal nodes are replaced with logistic regression functions. This paper reports the results of an empirical evaluation that compares the new decision tree extraction algorithm with Quinlan's C4.5 and ExTree. The evaluation used 12 standard benchmark datasets from the University of California, Irvine machine-learning repository. The results of this evaluation demonstrate that the new algorithm produces decision trees that have higher accuracy and higher fidelity than decision trees created by both C4.5 and ExTree.     

Extraction of rules from artificial neural networks for nonlinearregression

Neural networks (NNs) have been successfully applied to solve a variety of application problems including classification and function approximation. They are especially useful as function approximators because they do not require prior knowledge of the input data distribution and they have been shown to be universal approximators. In many applications, it is desirable to extract knowledge that can explain how Me problems are solved by the networks. Most existing approaches have focused on extracting symbolic rules for classification. Few methods have been devised to extract rules from trained NNs for regression. This article presents an approach for extracting rules from trained NNs for regression. Each rule in the extracted rule set corresponds to a subregion of the input space and a linear function involving the relevant input attributes of the data approximates the network output for all data samples in this subregion. Extensive experimental results on 32 benchmark data sets demonstrate the effectiveness of the proposed approach in generating accurate regression rules.     

Top-of-atmosphere flux retrievals from CERES using artificial neural networks

The Clouds and the Earth's Radiant Energy System (CERES) instruments on the Terra spacecraft provide accurate shortwave (SW), longwave (LW) and window (WN) region top-of-atmosphere (TOA) radiance measurements from which TOA radiative flux values are obtained by applying Angular Distribution Models (ADMs). These models are developed empirically as functions of the surface and cloud properties provided by coincident high-resolution imager measurements over CERES field-of-view. However, approximately 5.6% of the CERES/Terra footprints lack sufficient imager information for a reliable scene identification. To avoid any systematic biases in regional mean radiative fluxes, it is important to provide TOA fluxes for these footprints. For this purpose, we apply a feedforward error-backpropagation Artificial Neural Network (ANN) technique to reproduce CERES/Terra ADMs relying only on CERES measurements. All-sky ANN-based angular distribution models are developed for 10 surface types separately for shortwave, longwave and window TOA flux retrievals. To optimize the ANN performance, we use a partially connected first hidden neuron layer and compact training sets with reduced data noise. We demonstrate the performance of the ANN-based ADMs by comparing TOA fluxes inferred from ANN and CERES anisotropic factors. The global annual average bias in ANN-derived fluxes relative to CERES is less than 0.5% for all ANN scene types. The maximum bias occurs over sea ice and permanent snow surfaces. For all surface types, instantaneous ANN-derived TOA fluxes are self-consistent in viewing zenith angle to within 9% for shortwave, 3.5% and 3% longwave daytime and nighttime, respectively.     

Controlling fast spring-legged locomotion with artificial neural networks

Controlling the model of an one-legged robot is investigated. The model consists merely of a mass less spring attached to a point mass. The motion of this system is characterised by repeated changes between ground contact and flight phases. It can be kept in motion by active control only. Robots that are suited for fast legged locomotion require different hardware layouts and control approaches in contrast to slow moving ones. The spring mass system is a simple model that describes this principle movement of a spring-legged robot. Multi-Layer-Perceptrons (MLPs), Radial Basis Functions (RBFs) and Self-Organising Motoric Maps (SOMMs) were used to implement neurocontrollers for such a movement system. They all prove to be suitable for control of the movement. This is also shown by an experiment where the environment of the spring-mass system is changed from even to uneven ground. The neurocontroller is performing well with this additional complexity without being trained for it.     

Clustering with artificial neural networks and traditional techniques

In this article, two clustering techniques based on neural networks are introduced. The two neural network models are the Harmony theory network (HTN) and the self‐organizing logic neural network (SOLNN), both of which are characterized by parallel processing, a distributed architecture, and a large number of nodes. After describing their clustering characteristics and potential, a comparison to classical statistical techniques is performed. This comparison allows the creation of a correspondence between each neural network clustering technique and particular metrics as used by the corresponding statistical methods, which reflect the affinity of the clustered patterns. In particular, the HTN is found to perform the clustering task with an accuracy similar to the best statistical methods, while it is further capable of proposing an optimal number of groups into which the patterns may be clustered. On the other hand, the SOLNN combines a high clustering accuracy with the ability to cluster higher‐dimensional patterns without a considerable increase in the processing time. © 2003 Wiley Periodicals, Inc.