Application of neural networks in forecasting engine systems reliability

This paper presents a comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems. Traditionally, failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions, which are at times difficult to validate. On the other hand, the time series modeling technique using neural networks provides a promising alternative. Neural network modeling via feed-forward multilayer perceptron (MLP) suffers from local minima problems and long computation time. The radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time. Illustrative examples using reliability testing and field data showed that the proposed model results in comparable or better predictive performance than traditional MLP model and the linear benchmark based on Box–Jenkins autoregressive-integrated-moving average (ARIMA) models. The effects of input window size and hidden layer nodes are further investigated. Appropriate design topologies can be determined via sensitivity analysis.     

Combined projection and kernel basis functions for classification in evolutionary neural networks

This paper proposes a hybrid neural network model using a possible combination of different transfer projection functions (sigmoidal unit, SU, product unit, PU) and kernel functions (radial basis function, RBF) in the hidden layer of a feed-forward neural network. An evolutionary algorithm is adapted to this model and applied for learning the architecture, weights and node typology. Three different combined basis function models are proposed with all the different pairs that can be obtained with SU, PU and RBF nodes: product–sigmoidal unit (PSU) neural networks, product–radial basis function (PRBF) neural networks, and sigmoidal–radial basis function (SRBF) neural networks; and these are compared to the corresponding pure models: product unit neural network (PUNN), multilayer perceptron (MLP) and the RBF neural network. The proposals are tested using ten benchmark classification problems from well known machine learning problems. Combined functions using projection and kernel functions are found to be better than pure basis functions for the task of classification in several datasets.     

A hybrid linear/nonlinear training algorithm for feedforward neuralnetworks

This paper presents a new hybrid optimization strategy for training feedforward neural networks. The algorithm combines gradient-based optimization of nonlinear weights with singular value decomposition (SVD) computation of linear weights in one integrated routine. It is described for the multilayer perceptron (MLP) and radial basis function (RBF) networks and then extended to the local model network (LMN), a new feedforward structure in which a global nonlinear model is constructed from a set of locally valid submodels. Simulation results are presented demonstrating the superiority of the new hybrid training scheme compared to second-order gradient methods. It is particularly effective for the LMN architecture where the linear to nonlinear parameter ratio is large.     

Geometrical interpretation and architecture selection of MLP

A geometrical interpretation of the multilayer perceptron (MLP) is suggested in this paper. Some general guidelines for selecting the architecture of the MLP, i.e., the number of the hidden neurons and the hidden layers, are proposed based upon this interpretation and the controversial issue of whether four-layered MLP is superior to the three-layered MLP is also carefully examined.     

Hybrid multilayered perceptron networks

This paper introduces a modified multilayered perception network (MLP) called the Hybrid Multilayered Perceptron (HMLP) network to improve the performance of a MLP network. The convergence rate of the proposed network is further improved by proposing a modified version of the recursive prediction error algorithm as the training algorithm. The capability of the proposed network architecture trained using the modified recursive prediction error algorithm was demonstrated using simulated and real data sets. The results indicated that the proposed network provides a significant improvement over a standard MLP network. These additional linear input connections do not significantly increase the complexity of the MLP network since the connections are linear. In fact, by using the linear input connections, the number of hidden nodes required by the standard MLP network model can be reduced, which will also reduce computational load. The performance of the HMLP network was also compared with Radial Basis Function (RBF) and Hybrid Radial Basis Function (HRBF) networks. It was found that the proposed HMLP network was much more efficient than both RBF and HRBF networks.     

Conventional modeling of the multilayer perceptron using polynomialbasis functions

A technique for modeling the multilayer perceptron (MLP) neural network, in which input and hidden units are represented by polynomial basis functions (PBFs), is presented. The MLP output is expressed as a linear combination of the PBFs and can therefore be expressed as a polynomial function of its inputs. Thus, the MLP is isomorphic to conventional polynomial discriminant classifiers or Volterra filters. The modeling technique was successfully applied to several trained MLP networks.     

Thalassaemia classification by neural networks and genetic programming

This paper presents the use of a neural network and a decision tree, which is evolved by genetic programming (GP), in thalassaemia classification. The aim is to differentiate between thalassaemic patients, persons with thalassaemia trait and normal subjects by inspecting characteristics of red blood cells, reticulocytes and platelets. A structured representation on genetic algorithms for non-linear function fitting or STROGANOFF is the chosen architecture for genetic programming implementation. For comparison, multilayer perceptrons are explored in classification via a neural network. The classification results indicate that the performance of the GP-based decision tree is approximately equal to that of the multilayer perceptron with one hidden layer. But the multilayer perceptron with two hidden layers, which is proven to have the most suitable architecture among networks with different number of hidden layers, outperforms the GP-based decision tree. Nonetheless, the structure of the decision tree reveals that some input features have no effects on the classification performance. The results confirm that the classification accuracy of the multilayer perceptron with two hidden layers can still be maintained after the removal of the redundant input features. Detailed analysis of the classification errors of the multilayer perceptron with two hidden layers, in which a reduced feature set is used as the network input, is also included. The analysis reveals that the classification ambiguity and misclassification among persons with minor thalassaemia trait and normal subjects is the main cause of classification errors. These results suggest that a combination of a multilayer perceptron with a blood cell analysis may give rise to a guideline/hint for further investigation of thalassaemia classification.     

A review of genetic algorithms applied to training radial basis function networks

The problems associated with training feedforward artificial neural networks (ANNs) such as the multilayer perceptron (MLP) network and radial basis function (RBF) network have been well documented. The solutions to these problems have inspired a considerable amount of research, one particular area being the application of evolutionary search algorithms such as the genetic algorithm (GA). To date, the vast majority of GA solutions have been aimed at the MLP network. This paper begins with a brief overview of feedforward ANNs and GAs followed by a review of the current state of research in applying evolutionary techniques to training RBF networks.     

A Novel Structure for Radial Basis Function Networks—WRBF

A novel structure for radial basis function networks is proposed. In this structure, unlike traditional RBF, we set some weights between input and hidden layer. These weights, which take values around unity, are multiplication factors for input vector and perform a linear mapping. Doing this, we increase free parameters of the network, but since these weights are trainable, the overall performance of the network is improved significantly. According to the new weight vector, we called this structure Weighted RBF or WRBF. Weight adjustment formula is provided by applying the gradient descent algorithm. Two classification problems used to evaluate performance of the new RBF network: letter classification using UCI dataset with 16 features, a difficult problem, and digit recognition using HODA dataset with 64 features, an easy problem. WRBF is compared with classic RBF and MLP network, and our experiments show that WRBF outperforms both significantly. For example, in the case of 200 hidden neurons, WRBF achieved recognition rate of 92.78% on UCI dataset while RBF and MLP achieved 83.13 and 89.25% respectively. On HODA dataset, WRBF reached 97.94% recognition rate whereas RBF achieved 97.14%, and MLP accomplished 97.63%.     

Fourier and wavelet descriptors for shape recognition using neural networks—a comparative study

This paper presents the application of three different types of neural networks to the 2-D pattern recognition on the basis of its shape. They include the multilayer perceptron (MLP), Kohonen self-organizing network and hybrid structure composed of the self-organizing layer and the MLP subnetwork connected in cascade. The recognition is based on the features extracted from the Fourier and wavelet transformations of the data, describing the shape of the pattern. Application of different neural network structures associated with different preprocessing of the data results in different accuracy of recognition and classification. The numerical experiments performed for the recognition of simulated shapes of the airplanes have shown the superiority of the wavelet preprocessing associated with the self-organizing neural network structure. The integration of the individual classifiers based on the weighted summation of the signals from the neural networks has been proposed and checked in numerical experiments.     

基于系统调用的混合HMM/MLP异常检测模型

首先描述了基于隐马尔可夫模型(HMM)的异常检测方法并指出其缺点．然后提出了一种将多层感知机(MLP)用作HMM的概率估计器的方法,以克服HMM方法的不足．最后建立了一个基于系统调用的混合HMM/MLP异常检测模型,并给出了该模型的训练和检测算法．实验结果表明,该混合系统的漏报率和误报率都低于HMM方法．     

State of charge neural computational models for high energy density batteries in electric vehicles

Environmental concerns, increasing gasoline demand together with unpopularity of alternative energy sources to propel vehicles, have pushed on hybrid electric vehicles (HEVs) solutions. The main problem in battery management of HEVs is how to determine the battery state of charge (SOC). Estimation of SOC is an active area of research, and several approaches have been presented in the literature to monitor the SOC of a cell. At the first step, we use an optimized structure multi-layer perceptron (MLP) and an RBF neural network to determine the SOC changes of a high energy density battery in this paper. Then, a hybrid optimized structure neural model is used for SOC prediction considering the aging effect through the state of health (SOH) and discharge efficiency (DE) parameters. In this way, particle swarm optimization (PSO) algorithm is used for determining the optimum number of nodes in hidden layer(s) of MLPs. Experimental results show that the SOC estimation error by the proposed hybrid optimized structure neural model is 1.9% when compared with the real SOC obtained from a discharge test. In addition, monolithic optimized structure MLP and RBF neural models offer a good estimation of differentiated SOC.     

APPLICATION OF ARTIFICIAL NEURAL NETWORKS IN CONTROLLED DRUG DELIVERY SYSTEMS

Estimation of release profiles of drugs normally requires time-consuming trial-and-error experiments. Feed-forward neural networks including multilayer perceptron (MLP), radial basis function network (RBFN), and generalized regression neural network (GRNN) are used to predict the release profile of betamethasone (BTM) and betamethasone acetate (BTMA) where in situ forming systems consist of poly (lactide-co-glycolide), N-methyl-1-2-pyrolidon, and ethyl heptanoat as a polymer, solvent, and additive, respectively. The input vectors of the artificial neural networks (ANNs) include drug concentration, gamma irradiation, additive substance, and type of drug. As the outputs of the ANNs, three features are extracted using the nonlinear principal component analysis technique. Leave-one-out cross-validation approach is used to train each ANN. We show that for estimation of BTM and BTMA release profiles, MLP outperforms GRNN and RBF networks in terms of reliability and efficiency.     

Interpretation and knowledge discovery from the multilayer perceptron network: Opening the black box

This paper interprets the outputs from the multilayer perceptron (MLP) network by finding the input data features at the input layer of the network which activate the hidden layer feature detectors. This leads directly to the deduction of the significant data inputs, the inputs that the network actually uses to perform the input/output mapping for a classification task, and the discovery of the most significant of these data inputs. The analysis presents a method for providing explanations for the network outputs and for representing the knowledge learned by the network in the form of significant input data relationships. During network development the explanation facilities and data relationships can be used for network validation and verification, and after development, for rule induction and data mining where this method provides a potential tool for knowledge discovery in databases (KDD).     

Knowledge-based fuzzy MLP for classification and rule generation

A new scheme of knowledge-based classification and rule generation using a fuzzy multilayer perceptron (MLP) is proposed. Knowledge collected from a data set is initially encoded among the connection weights in terms of class a priori probabilities. This encoding also includes incorporation of hidden nodes corresponding to both the pattern classes and their complementary regions. The network architecture, in terms of both links and nodes, is then refined during training. Node growing and link pruning are also resorted to. Rules are generated from the trained network using the input, output, and connection weights in order to justify any decision(s) reached. Negative rules corresponding to a pattern not belonging to a class can also be obtained. These are useful for inferencing in ambiguous cases. Results on real life and synthetic data demonstrate that the speed of learning and classification performance of the proposed scheme are better than that obtained with the fuzzy and conventional versions of the MLP (involving no initial knowledge encoding). Both convex and concave decision regions are considered in the process.     

Nonlinear channel equalization for QAM signal constellation usingartificial neural networks

Application of artificial neural networks (ANN's) to adaptive channel equalization in a digital communication system with 4-QAM signal constellation is reported in this paper. A novel computationally efficient single layer functional link ANN (FLANN) is proposed for this purpose. This network has a simple structure in which the nonlinearity is introduced by functional expansion of the input pattern by trigonometric polynomials. Because of input pattern enhancement, the FLANN is capable of forming arbitrarily nonlinear decision boundaries and can perform complex pattern classification tasks. Considering channel equalization as a nonlinear classification problem, the FLANN has been utilized for nonlinear channel equalization. The performance of the FLANN is compared with two other ANN structures [a multilayer perceptron (MLP) and a polynomial perceptron network (PPN)] along with a conventional linear LMS-based equalizer for different linear and nonlinear channel models. The effect of eigenvalue ratio (EVR) of input correlation matrix on the equalizer performance has been studied. The comparison of computational complexity involved for the three ANN structures is also provided.     

Identification and control using MLP,Elman, NARXSP and radial basis function networks: a comparative analysis

This paper describes four neural networks multilayer perceptron (MLP) network, Elman network, NARXSP network and radial basis function (RBF) network. Neural networks are applied for identification and control of DC servo motor and benchmark nonlinear system. Number of epochs required and time taken to train the controller are shown in the form of bar plots for four neural networks. Levenberg-Marquardt algorithm is used for training the controller using neural network toolbox in MATLAB. Each neural network controller is run ten times. Their performances are compared for each run in terms of number of epochs required and time taken to train each controller for tracking a reference trajectory.     

Development and performance evaluation of neural network classifiers for Indian internet shoppers

The rapid growth of usage of internet has paved the way towards the use of online shopping. Consumers’ behavior is one of the significant aspects that is considered by the service providers for the improvement of various services. Consumers are generally satisfied if their needs are fulfilled. In this paper an in depth investigation is made on the behavior of Indian consumers towards online shopping. Factor analysis is carried out to extract significant factors that affect online shopping of Indian consumers and these consumers are clustered based on their behavior, towards online shopping using hierarchical clustering. Employing the results of clustering in training of multilayer perceptron (MLP), functional link artificial neural network (FLANN) and radial basis function (RBF) networks efficient classifier models are developed. The performance of these classifiers are evaluated and compared with those obtained by conventional statistical based discriminant analysis. The simulation study demonstrates that the RBF network provides best classification performance of internet shoppers compared to those given by the FLANN, MLP and discriminant analysis based methods. The simulation study on the impact of different combination of inputs demonstrates that demographic input has least effect on classification performance. On the other hand the combination of psychological and cultural inputs play the most significant role in classification followed by psychological and then cultural inputs alone.     

An MLP-SVM combination architecture for offline handwritten digit recognition

This paper presents an original hybrid MLP-SVM method for unconstrained handwritten digits recognition. Specialized Support Vector Machines (SVMs) are introduced to improve significantly the multilayer perceptron (MLP) performance in local areas around the separating surfaces between each pair of digit classes, in the input pattern space. This hybrid architecture is based on the idea that the correct digit class almost systematically belongs to the two maximum MLP outputs and that some pairs of digit classes constitute the majority of MLP substitutions (errors). Specialized local SVMs are introduced to detect the correct class among these two classification hypotheses. The hybrid MLP-SVM recognizer achieves a recognition rate of 98.01%98.01\% , for real mail zipcode digits recognition task. By introducing a rejection mechanism based on the distances provided by the local SVMs, the error/reject trade-off performance of our recognition system is better than several classifiers reported in recent research.     

Spectrum occupancy prediction based on functional link artificial neural network (FLANN) in ISM band

Recent advancements in artificial neural networks (ANNs) motivated us to design a simple and faster spectrum prediction model termed the functional link artificial neural network (FLANN). The main objective of this paper is to gather realistic data to obtain utilization statistics for the industrial, scientific and medical band of 2.4–2.5 GHz. To present the occupancy statistics, we conducted measurement in indoors at the Swearingen Engineering Center, University of South Carolina. Further, we introduce different threshold-based spectrum prediction schemes to show the impact of threshold on occupancy, and propose a spectrum prediction algorithm based on FLANN to forecast a future spectrum usage profile from historical occupancy statistics. Spectrum occupancy is estimated and predicted by employing different ANN models including the Feed-forward multilayer perceptron (MLP), Recurrent MLP, Chebyshev FLANN and Trigonometric FLANN. It is observed that the absence of a hidden layer in FLANN makes it more efficient than the MLP model in predicting the occupancy faster and with less complexity. A set of illustrative results are presented to validate the performance of our proposed learning scheme.