Effective data mining using neural networks

Classification is one of the data mining problems receiving great attention recently in the database community. The paper presents an approach to discover symbolic classification rules using neural networks. Neural networks have not been thought suited for data mining because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. With the proposed approach, concise symbolic rules with high accuracy can be extracted from a neural network. The network is first trained to achieve the required accuracy rate. Redundant connections of the network are then removed by a network pruning algorithm. The activation values of the hidden units in the network are analyzed, and classification rules are generated using the result of this analysis. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of standard data mining test problems     

Acoustic-to-phonetic mapping using recurrent neural networks

This paper describes the application of artificial neural networks to acoustic-to-phonetic mapping. The experiments described are typical of problems in speech recognition in which the temporal nature of the input sequence is critical. The specific task considered is that of mapping formant contours to the corresponding CVC' syllable. We performed experiments on formant data extracted from the acoustic speech signal spoken at two different tempos (slow and normal) using networks based on the Elman simple recurrent network model. Our results show that the Elman networks used in these experiments were successful in performing the acoustic-to-phonetic mapping from formant contours. Consequently, we demonstrate that relatively simple networks, readily trained using standard backpropagation techniques, are capable of initial and final consonant discrimination and vowel identification for variable speech rates     

Financial volatility trading using recurrent neural networks

We simulate daily trading of straddles on financial indexes. The straddles are traded based on predictions of daily volatility differences in the indexes. The main predictive models studied are recurrent neural nets (RNN). Such applications have often been studied in isolation. However, due to the special character of daily financial time-series, it is difficult to make full use of RNN representational power. Recurrent networks either tend to overestimate noisy data, or behave like finite-memory sources with shallow memory; they hardly beat classical fixed-order Markov models. To overcome data nonstationarity, we use a special technique that combines sophisticated models fitted on a larger data set, with a fixed set of simple-minded symbolic predictors using only recent inputs. Finally, we compare our predictors with the GARCH family of econometric models designed to capture time-dependent volatility structure in financial returns. GARCH models have been used to trade volatility. Experimental results show that while GARCH models cannot generate any significantly positive profit, by careful use of recurrent networks or Markov models, the market makers can generate a statistically significant excess profit, but then there is no reason to prefer RNN over much more simple and straightforward Markov models. We argue that any report containing RNN results on financial tasks should be accompanied by results achieved by simple finite-memory sources combined with simple techniques to fight nonstationarity in the data.     

Two-stage structural damage detection using fuzzy neural networks and data fusion techniques

It is proposed in this paper a novel two-stage structural damage detection approach using fuzzy neural networks (FNNs) and data fusion techniques. The method is used for structural health monitoring and damage detection, particularly for cases where the measurement data is enormous and with uncertainties. In the first stage of structural damage detection, structural modal parameters derived from structural vibration responses are fed into an FNN as the input. The output values from the FNN are defuzzified to produce a rough structural damage assessment. Later, in the second stage, the values output from three different FNN models are input directly to the data fusion center where fusion computation is performed. The final fusion decision is made by filtering the result with a threshold function, hence a refined structural damage assessment of superior reliability. The proposed approach has been applied to a 7-degree of freedom building model for structural damage detection, and proves to be feasible, efficient and satisfactory. Furthermore, the simulation result also shows that the identification accuracy can be boosted with the proposed approach instead of FNN models alone.     

Training recurrent neural networks using a hybrid algorithm

This paper proposes a new hybrid approach for recurrent neural networks (RNN). The basic idea of this approach is to train an input layer by unsupervised learning and an output layer by supervised learning. In this method, the Kohonen algorithm is used for unsupervised learning, and dynamic gradient descent method is used for supervised learning. The performances of the proposed algorithm are compared with backpropagation through time (BPTT) on three benchmark problems. Simulation results show that the performances of the new proposed algorithm exceed the standard backpropagation through time in the reduction of the total number of iterations and in the learning time required in the training process.     

Forecasting foreign exchange rates using recurrent neural networks

This article proposes the use of recurrent neural networks in order to forecast foreign exchange rates. Artificial neural networks have proven to be efficient and profitable in fore casting financial time series. In particular, recurrent networks in which activity patterns pass through the network more than once before they generate an output pattern can learn ex tremely complex temporal sequences. Three recurrent architectures are compared in terms of prediction accuracy of futures forecast for Deutsche mark currency. A trading strategy is then devised and optimized. The profitability of the trading strategy taking into account trans action costs is shown for the different architectures. The methods described here which have obtained promising results in real time trading are applicable to other markets.     

Discontinuities in recurrent neural networks

This article studies the computational power of various discontinuous real computational models that are based on the classical analog recurrent neural network (ARNN). This ARNN consists of finite number of neurons; each neuron computes a polynomial net function and a sigmoid-like continuous activation function. We introduce arithmetic networks as ARNN augmented with a few simple discontinuous (e.g., threshold or zero test) neurons. We argue that even with weights restricted to polynomial time computable reals, arithmetic networks are able to compute arbitrarily complex recursive functions. We identify many types of neural networks that are at least as powerful as arithmetic nets, some of which are not in fact discontinuous, but they boost other arithmetic operations in the net function (e.g., neurons that can use divisions and polynomial net functions inside sigmoid-like continuous activation functions). These arithmetic networks are equivalent to the Blum-Shub-Smale model, when the latter is restricted to a bounded number of registers. With respect to implementation on digital computers, we show that arithmetic networks with rational weights can be simulated with exponential precision, but even with polynomial-time computable real weights, arithmetic networks are not subject to any fixed precision bounds. This is in contrast with the ARNN that are known to demand precision that is linear in the computation time. When nontrivial periodic functions (e.g., fractional part, sine, tangent) are added to arithmetic networks, the resulting networks are computationally equivalent to a massively parallel machine. Thus, these highly discontinuous networks can solve the presumably intractable class of PSPACE-complete problems in polynomial time.     

Intelligent data structures selection using neural networks

It is well known that abstract data types represent the core for any software application, and a proper use of them is an essential requirement for developing a robust and efficient system. Data structures are essential in obtaining efficient algorithms, having a major importance in the software development process. Selecting and creating the appropriate data structure for implementing an abstract data type can greatly impact the performance and the efficiency of the software systems. It is not a trivial problem for a software developer, as it is hard to anticipate all the use scenarios of the deployed application, and a static selection before the system’s execution is, generally, not accurate. In this paper, we are focusing on the problem of dynamic selection of efficient data structures for abstract data types implementation using a supervised learning approach. In order to dynamically select the most suitable representation for an aggregate according to the software system’s current execution context, a neural network will be used. We experimentally evaluate the proposed technique on a case study, emphasizing the advantages of the proposed model in comparison with existing similar approaches.     

Object detection using pulse coupled neural networks

Describes an object detection system based on pulse coupled neural networks. The system is designed and implemented to illustrate the power, flexibility and potential the pulse coupled neural networks have in real-time image processing. In the preprocessing stage, a pulse coupled neural network suppresses noise by smoothing the input image. In the segmentation stage, a second pulse coupled neural-network iteratively segments the input image. During each iteration, with the help of a control module, the segmentation network deletes regions that do not satisfy the retention criteria from further processing and produces an improved segmentation of the retained image. In the final stage each group of connected regions that satisfies the detection criteria is identified as an instance of the object of interest.     

Spoken keyword detection using autoassociative neural networks

Spoken keywords detection is essential to organize efficiently lots of hours of audio contents such as meetings, radio news, etc. These systems are developed with the purpose of indexing large audio databases or of detecting keywords in continuous speech streams. This paper addresses a new approach to spoken keyword detection using Autoassociative Neural Networks (AANN). The proposed work concerns the use of the distribution capturing ability of the Autoassociative neural network (AANN) for spoken keyword detection. It involves sliding a frame-based keyword template along the speech signal and using confidence score obtained from the normalized squared error of AANN to efficiently search for a match. This work formulates a new spoken keyword detection algorithm. The experimental results show that the proposed approach competes with the keyword detection methods reported in the literature and it is an alternative method to the existing key word detection methods.     

Situation recognition using image moments and recurrent neural networks

Surveillance is an important need for a secured and supervised environment. Manual supervision for the purpose of surveillance proves to be expensive and prone to slipups. Many researchers have worked to provide an automated solution to this problem. In this article, we present a solution to this problem using image moments and recurrent neural networks. For this purpose, frames are first extracted from a live video and the foreground of the frame is sieved out while the background is discarded. Feature vectors are obtained for each frame after computing raw, central, scale-invariant and rotation-invariant moments of the images. These vectors are used to train and ultimately simulate a recurrent neural network. The results generated from this model exhibit an accuracy of 96.4 % in identification of events within consecutive frames.     

Real-time torque control using discrete-time recurrent high-order neural networks

This paper presents a discrete-time direct current (DC) motor torque tracking controller, based on a recurrent high-order neural network to identify the plant model. In order to train the neural identifier, the extended Kalman filter (EKF) based training algorithm is used. The neural identifier is in series-parallel configuration that constitutes a well approximation method of the real plant by the neural identifier. Using the neural identifier structure that is in the nonlinear controllable form, the block control (BC) combined with sliding modes (SM) control techniques in discrete-time are applied. The BC technique is used to design a nonlinear sliding manifold such that the resulting sliding mode dynamics are described by a desired linear system. For the SM control technique, the equivalent control law is used in order to the plant output tracks a reference signal. For reducing the effect of unknown terms, it is proposed a specific desired dynamics for the sliding variables. The control problem is solved by the indirect approach, where an appropriate neural network (NN) identification model is selected; the NN parameters (synaptic weights) are adjusted according to a specific adaptive law (EKF), such that the response of the NN identifier approximates the response of the real plant for the same input. Then, based on the designed NN identifier a stabilizing or reference tracking controller is proposed (BC combined with SM). The proposed neural identifier and control applicability are illustrated by torque trajectory tracking for a DC motor with separate winding excitation via real-time implementation.     

Prediction and identification using wavelet-based recurrent fuzzy neural networks

This paper presents a wavelet-based recurrent fuzzy neural network (WRFNN) for prediction and identification of nonlinear dynamic systems. The proposed WRFNN model combines the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and the wavelet neural networks (WNN). This paper adopts the nonorthogonal and compactly supported functions as wavelet neural network bases. Temporal relations embedded in the network are caused by adding some feedback connections representing the memory units into the second layer of the feedforward wavelet-based fuzzy neural networks (WFNN). An online learning algorithm, which consists of structure learning and parameter learning, is also presented. The structure learning depends on the degree measure to obtain the number of fuzzy rules and wavelet functions. Meanwhile, the parameter learning is based on the gradient descent method for adjusting the shape of the membership function and the connection weights of WNN. Finally, computer simulations have demonstrated that the proposed WRFNN model requires fewer adjustable parameters and obtains a smaller rms error than other methods.     

Classification of fermentation process models using recurrent neural networks

In this article we present the classification of batch fermentation models. A recurrent neural network uses temporal information on the state variables together with the time values. It can select from several possible models of the process the model that best describes the dynamics of the process. A pre-treatment of the data, denoted by 'selfnormalization' is also proposed. It is shown by a parameter sensitivity study that the 'self-normalization' assigns to a family of models (same structure of model, with different parameters) an approximately unique representation. This representation is used for training the recurrent neural network. The dimension of the learning set is considerably reduced. The trained neural network is used for the classification of real lactic fermentation data. The model which best suits the experimental data is determined and, from this, the main phenomena governing the process. The response of the neural classifier represents only a comparative measure of belonging to each of the considered models. The results show the good capacity of the network to recognise the 'best' model. This technique can be used as an assisting tool to modelling of batch biotechnological processes.     

Soil-moisture estimation from TerraSAR-X data using neural networks

Early prediction of natural disasters like floods and landslides is essential for reasons of public safety. This can be attained by processing Synthetic-Aperture Radar (SAR) images and retrieving soil-moisture parameters. In this article, TerraSAR-X product images are investigated in combination with a water-cloud model based on the Shi semi-empirical model to determine the accuracy of soil-moisture parameter retrieval. SAR images were captured between January 2008 and September 2010 in the vicinity of the city Maribor, Slovenia, at different incidence angles. The water-cloud model provides acceptable estimated soil-moisture parameters at bare or scarcely vegetated soil areas. However, this model is too sensitive to speckle noise; therefore, a pre-processing step for speckle-noise reduction is carried out. Afterwards, self-organizing neural networks (SOM) are used to segment the areas at which the performance of this model is poor, and at the same time neural networks are also used for a more accurate approximation of model parameters’ values. Ground-truth is measured using the Pico64 sensor located on the field, simultaneously with capturing SAR images, in order to enable the comparison and validation of the obtained results. Experimental results show that the proposed method outperforms the water-cloud model accuracy over all incidence angles.     

LADAR target detection using morphological shared-weight neural networks

Morphological shared-weight neural networks (MSNN) combine the feature extraction capability of mathematical morphology with the function-mapping capability of neural networks in a single trainable architecture. The MSNN method has been previously demonstrated using a variety of imaging sensors, including TV, forward-looking infrared (FLIR) and synthetic aperture radar (SAR). In this paper, we provide experimental results with laser radar (LADAR). We present three sets of experiments. In the first set of experiments, we use the MSNN to detect different types of targets simultaneously. In the second set, we use the MSNN to detect only a particular type of target. In the third set, we test a novel scenario, referred to as the Sims scenario: we train the MSNN to recognize a particular type of target using very few examples. A detection rate of 86% with a reasonable number of false alarms was achieved in the first set of experiments and a detection rate of close to 100% with very few false alarms was achieved in the second and third sets of experiments. In all the experiments, a novel pre-processing method is used to create a pseudo-intensity images from the original LADAR range images.     

Epileptic EEG detection using neural networks and post-classification

Electroencephalogram (EEG) has established itself as an important means of identifying and analyzing epileptic seizure activity in humans. In most cases, identification of the epileptic EEG signal is done manually by skilled professionals, who are small in number. In this paper, we try to automate the detection process. We use wavelet transform for feature extraction and obtain statistical parameters from the decomposed wavelet coefficients. A feed-forward backpropagating artificial neural network (ANN) is used for the classification. We use genetic algorithm for choosing the training set and also implement a post-classification stage using harmonic weights to increase the accuracy. Average specificity of 99.19%, sensitivity of 91.29% and selectivity of 91.14% are obtained.     

递归神经网络的结构研究

从非线性动态系统的角度出发,对递归动态网络结构及其功能进行详尽的综述。将递归动态网络分为三大类：全局反馈递归网络、前向递归网络和混合型网络。每一类网络又可分为若干种网络。给出了每种网络描述网络特性的结构图,同时还对多种网络进行了功能对比,分析了各种网络的异同。     

Arabic sentiment analysis using recurrent neural networks: a review

Artificial Intelligence Review - Over the last decade, the amount of Arabic content created on websites and social media has grown significantly. Opinions are shared openly and freely on social...