Multilayer perceptron for reference evapotranspiration estimation in a semiarid region

Calculation of reference evapotranspiration (ET_o) is essential in hydrology and agriculture. ET_o plays an important role in planning and management of water resources and irrigation scheduling. The results of many studies strongly support the use of the Penman–Monteith FAO 56 (PMF-56) method as the standard method of estimating ET_o. The basic obstacle to using this method widely is the numerous meteorological variables required. Multilayer perceptron (MLP) networks optimized with different learning algorithms and activation functions were applied for estimating ET_o in a semiarid region in Iran. Four MLP models comprising various combinations of meteorological variables are developed. The MLP model which needs all of the meteorological parameters performed best for ET_o estimation amongst the other MLP models. It was also found that the ConjugateGradient, DeltaBarDelta, DeltaBarDelta and Levenberg–Marquardt were the best algorithms for training the MLP1, MLP2, MLP3 and MLP4 models, respectively.     

Hybrid continuous speech recognition systems by HMM,MLP and SVM: a comparative study

This paper presents a new hybrid method for continuous Arabic speech recognition based on triphones modelling. To do this, we apply Support Vectors Machine (SVM) as an estimator of posterior probabilities within the Hidden Markov Models (HMM) standards. In this work, we describe a new approach of categorising Arabic vowels to long and short vowels to be applied on the labeling phase of speech signals. Using this new labeling method, we deduce that SVM/HMM hybrid model is more efficient then HMMs standards and the hybrid system Multi-Layer Perceptron (MLP) with HMM. The obtained results for the Arabic speech recognition system based on triphones are 64.68 % with HMMs, 72.39 % with MLP/HMM and 74.01 % for SVM/HMM hybrid model. The WER obtained for the recognition of continuous speech by the three systems proves the performance of SVM/HMM by obtaining the lowest average for 4 tested speakers 11.42 %.     

Comparing three transition potential models: A case study of built-up transitions in North-East India

Change allocation is an important step in the Land Use Land Cover (LULC) change modelling. Many established LULC models use transition potential maps for the allocation of the estimated land demand. This study compares three commonly used techniques for transition potential modelling: (1) Multi-Layer Perceptron Neural Network (MLP), (2) Logistic Regression (LogReg), and (3) Similarity Weighted Instance-based Learning (SimWeight); and evaluates their applicability for built-up transitions. A case study has been taken from Guwahati city, in North-East India which experiences heterogeneous built-up growth in a limited area within the large topographic variations. With the same set of input and tested driving factors, all three models were simulated for the period 1989–2001 to produce the transition potential maps for 2011 and same amount of land demands, as in 2011 were allocated on the potential maps. The validation was done by (1) a multi-resolution validation method and (2) a region based method using the wards of the city. For this particular study, with the specific landscape environment and scale, MLP produced the most accurate change and predicted areas. The LogReg simulated the no change areas the most accurately, while the SimWeight could generate the edge extensions satisfactorily. We presented a detailed comparison of the change potentials and simulated maps and discuss the importance of evaluating the ability of the transition potential model used for LULC model. The results from this study can assist the LULC modelers to validate their transition potential models for generating accurate prediction maps. It can be also useful for planners and decision makers of Guwahati city and similar landscape, environment, scale in producing accurate transition potential zones for precise built-up growth modelling.     

Using function approximation to analyze the sensitivity of MLP withantisymmetric squashing activation function

Sensitivity analysis on a neural network is mainly investigated after the network has been designed and trained. Very few have considered this as a critical issue prior to network design. Piche's statistical method (1992, 1995) is useful for multilayer perceptron (MLP) design, but too severe limitations are imposed on both input and weight perturbations. This paper attempts to generalize Piche's method by deriving an universal expression of MLP sensitivity for antisymmetric squashing activation functions, without any restriction on input and output perturbations. Experimental results which are based on, a three-layer MLP with 30 nodes per layer agree closely with our theoretical investigations. The effects of the network design parameters such as the number of layers, the number of neurons per layer, and the chosen activation function are analyzed, and they provide useful information for network design decision-making. Based on the sensitivity analysis of MLP, we present a network design method for a given application to determine the network structure and estimate the permitted weight range for network training.     

Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks

This study deals with the neuro-fuzzy (NF) modelling of a real industrial winding process in which the acquired NF model can be exploited to improve control performance and achieve a robust fault-tolerant system. A new simulator model is proposed for a winding process using non-linear identification based on a recurrent local linear neuro-fuzzy (RLLNF) network trained by local linear model tree (LOLIMOT), which is an incremental tree-based learning algorithm. The proposed NF models are compared with other known intelligent identifiers, namely multilayer perceptron (MLP) and radial basis function (RBF). Comparison of our proposed non-linear models and associated models obtained through the least square error (LSE) technique (the optimal modelling method for linear systems) confirms that the winding process is a non-linear system. Experimental results show the effectiveness of our proposed NF modelling approach.     

Identification of non-linear processes using reciprocal multiquadric functions

In this paper radial basis function (RBF) networks are used to model general non-linear discrete-time systems. In particular, reciprocal multiquadric functions are used as activation functions for the RBF networks. A stepwise regression algorithm based on orthogonalization and a series of statistical tests is employed for designing and training of the network. The identification method yields non-linear models, which are stable and linear in the model parameters. The advantages of the proposed method compared to other radial basis function methods and backpropagation neural networks are described. Finally, the effectiveness of the identification method is demonstrated by the identification of two non-linear chemical processes, a simulated continuous stirred tank reactor and an experimental pH neutralization process.     

Achieving a More Robust Neural Network Model for Control of a MR Damper by Signal Sensitivity Analysis

Most neural network models can work accurately on their trained samples, but when encountering noise, there could be significant errors if the trained neural network is not robust enough to resist the noise. Sensitivity to perturbation in the control signal due to noise is very important for the prediction of an output signal. The goal of this paper is to provide a methodology of signal sensitivity analysis in order to enable the selection of an ideal Multi-Layer Perception (MLP) neural network model from a group of MLP models with different parameters, i.e. to get a highly accurate and robust model for control problems. This paper proposes a signal sensitivity which depends upon the variance of the output error due to noise in the input signals of a single output MLP with differentiable activation functions. On the assumption that noise arises from additive/multiplicative perturbations, the signal sensitivity of the MLP model can be easily calculated, and a method of lowering the sensitivity of the MLP model is proposed. A control system of a magnetorheological (MR) fluid damper, which is a relatively new type of device that shows the future promise for the control of vibration, is modelled by MLP. A large number of simulations on the MR damper’s MLP model show that a much better model is selected using the proposed method.     

Modelling of a fermentation process using Multi-Layer Perceptrons: Epochs vs Pattern learning,Sigmoid vs Linear transfer function

A number of differently configured Multi-Layer Perceptrons (MLP) were tested and compared for the system modelling of a highly non-linear fermentation process. The MLPs differed with respect to the inputs, the transfer functions, and the weight-updating schemes employed. In this paper it is shown that whereas all MLPs achieved the required error minimization their performances when used as models were greatly diverse. The modelling of the product concentration as a benchmark serves to highlight the benefits to be gained by adopting the configuration suggested by Lapedes and Farber (A. Lapedes and R. Farber 1987. Non-linear signal processing using neural networks: Prediction and system modeling. Los Alamos National Laboratory report, LA-UR-87-2662.) The configurations were tested with both training and previously unseen data. Further experiments with the selected configuration showed that a learning rate of 0·4 resulted in a model that was less sensitive towards the data.     

Faults diagnosis of a centrifugal pump using multilayer perceptron genetic algorithm back propagation and support vector machine with discrete wavelet transform‐based feature extraction

This paper presents a comparative study of two artificial intelligent systems, namely; Multilayer Perceptron (MLP) and support vector machine (SVM), to classify six fault conditions and the normal (nonfaulty) condition of a centrifugal pump. A hybrid training method for MLP is proposed for this work based on the combination of Back Propagation (BP) and Genetic Algorithm (GA). The two training algorithms are tested and compared separately as well. Features are extracted using Discrete Wavelet Transform (DWT), both approximations, details, and two mother wavelets were used to investigate their effectiveness on feature extraction. GA is also used to optimize the number of hidden layers and neurons of MLP. In this study, the feature extraction, GA‐based hidden layers, neurons selection, training algorithm, and classification performance, based on the strengths and weaknesses of each method, are discussed. From the results obtained, it is observed that the DWT with both MLP‐BP and SVM produces better classification rates and performances.     

Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks

This study deals with the neuro-fuzzy (NF) modelling of a real industrial winding process in which the acquired NF model can be exploited to improve control performance and achieve a robust fault-tolerant system. A new simulator model is proposed for a winding process using non-linear identification based on a recurrent local linear neuro-fuzzy (RLLNF) network trained by local linear model tree (LOLIMOT), which is an incremental tree-based learning algorithm. The proposed NF models are compared with other known intelligent identifiers, namely multilayer perceptron (MLP) and radial basis function (RBF). Comparison of our proposed non-linear models and associated models obtained through the least square error (LSE) technique (the optimal modelling method for linear systems) confirms that the winding process is a non-linear system. Experimental results show the effectiveness of our proposed NF modelling approach.     

CATSMLP: toward a robust and interpretable multilayer perceptron with sigmoid activation functions.

Enhancing the robustness and interpretability of a multilayer perceptron (MLP) with a sigmoid activation function is a challenging topic. As a particular MLP, additive TS-type MLP (ATSMLP) can be interpreted based on single-stage fuzzy IF-THEN rules, but its robustness will be degraded with the increase in the number of intermediate layers. This paper presents a new MLP model called cascaded ATSMLP (CATSMLP), where the ATSMLPs are organized in a cascaded way. The proposed CATSMLP is a universal approximator and is also proven to be functionally equivalent to a fuzzy inference system based on syllogistic fuzzy reasoning. Therefore, the CATSMLP may be interpreted based on syllogistic fuzzy reasoning in a theoretical sense. Meanwhile, due to the fact that syllogistic fuzzy reasoning has distinctive advantage over single-stage IF-THEN fuzzy reasoning in robustness, this paper proves in an indirect way that the CATSMLP is more robust than the ATSMLP in an upper-bound sense. Several experiments were conducted to confirm such a claim.     

Multilayer perceptron with different training algorithms for streamflow forecasting

Streamflow forecasting has always been a challenging task for water resources engineers and managers. This study applies Multilayer Perceptron (MLP) networks optimized with three training algorithms, including resilient back-propagation (MLP_RP), variable learning rate (MLP_GDX), and Levenberg–Marquardt (MLP_LM), to forecast streamflow in Aspas Watershed, located in Fars province in southwestern Iran. The algorithms were trained and tested using 3 years of data. Antecedent streamflow with 1 day time lag constituted the first input vector, and MLP with this vector, labeled as MLP1 was the first model. Inclusion of streamflow with two, three, and four time lags led to input vectors 2, 3, and 4 which when combined with MLP resulted in MLP2, MLP3, and MLP4, respectively. It was found that the Levenberg–Marquardt algorithm performed best among three types of training algorithms employed for training the MLP models. Generally, the MLP4_LM model yields the best result with a determination coefficient and a root mean square error of 0.93 and 2.6 (m³/s).     

基于系统调用的混合HMM/MLP异常检测模型

首先描述了基于隐马尔可夫模型(HMM)的异常检测方法并指出其缺点．然后提出了一种将多层感知机(MLP)用作HMM的概率估计器的方法,以克服HMM方法的不足．最后建立了一个基于系统调用的混合HMM/MLP异常检测模型,并给出了该模型的训练和检测算法．实验结果表明,该混合系统的漏报率和误报率都低于HMM方法．     

Decision support in machine vision system for monitoring of TFT-LCD glass substrates manufacturing

This study addresses classification methodology for the automatic inspection of a range of defects on the surface of glass substrates in thin film transistor liquid crystal display glass substrate manufacturing. The proposed methodology consisted of four stages: (1) feature extraction by calculating the wavelet co-occurrence signature from the substrate images, (2) handling of imbalanced dataset using the Synthetic Minority Over-sampling TEchnique (SMOTE), (3) reduction of the feature's dimension by principal component analysis, and (4) finally choosing the best classifier between three different methods: Classification And Regression Tree (CART), Multi-Layer Perceptron (MLP) and Support Vector Machine (SVM). In training the SVM and MLP classifiers, the simulated annealing algorithm was used to obtain the optimal tuning parameters for the classifiers. From the industrial case study, the proposed feature extraction algorithm could remove the defect-irrelevant image features and SMOTE increased the accuracy of all three methods. Furthermore, the optimized SVM and MLP models were more accurate than the CART model whereas a higher accuracy of 89.5% was observed for the proposed SVM model.     

Evolutionary training of hardware realizable multilayer perceptrons

The use of multilayer perceptrons (MLP) with threshold functions (binary step function activations) greatly reduces the complexity of the hardware implementation of neural networks, provides tolerance to noise and improves the interpretation of the internal representations. In certain case, such as in learning stationary tasks, it may be sufficient to find appropriate weights for an MLP with threshold activation functions by software simulation and, then, transfer the weight values to the hardware implementation. Efficient training of these networks is a subject of considerable ongoing research. Methods available in the literature mainly focus on two-state (threshold) nodes and try to train the networks by approximating the gradient of the error function and modifying appropriately the gradient descent, or by progressively altering the shape of the activation functions. In this paper, we propose an evolution-motivated approach, which is eminently suitable for networks with threshold functions and compare its performance with four other methods. The proposed evolutionary strategy does not need gradient related information, it is applicable to a situation where threshold activations are used from the beginning of the training, as in “on-chip” training, and is able to train networks with integer weights.     

强非线性Duffing系统分岔响应分析的MLP方法

同时考虑阻尼对响应频率和相位的影响,引入简单的变换,将有阻尼Duffing系统进行重写,得到的新系统在使用MLP方法的参数变换中,待定参数不受初始条件的影响,直接应用MLP方法有效的推导出受简谐激励作用下的含有阻尼的强非线性Duffing系统主共振和1／3亚谐共振的分岔响应方程．首次将MLP方法直接应用于含有阻尼的Duffing系统,极大的推广了MLP方法的应用范围,并对退化为无阻尼系统的结果与现有文献结果相比较,得到满意的结论．     

Ability to forecast unsteady aerodynamic forces of flapping airfoils by artificial neural network

The ability of artificial neural networks (ANN) to model the unsteady aerodynamic force coefficients of flapping motion kinematics has been studied. A neural networks model was developed based on multi-layer perception (MLP) networks and the Levenberg–Marquardt optimization algorithm. The flapping kinematics data were divided into two groups for the training and the prediction test of the ANN model. The training phase led to a very satisfactory calibration of the ANN model. The attempt to predict aerodynamic forces both the lift coefficient and drag coefficient showed that the ANN model is able to simulate the unsteady flapping motion kinematics and its corresponding aerodynamic forces. The shape of the simulated force coefficients was found to be similar to that of the numerical results. These encouraging results make it possible to consider interesting and new prospects for the modelling of flapping motion systems, which are highly non-linear systems.     

Evaluation of unsupervised feature extraction neural networks for face recognition

In this paper, new appearances based on neural networks (NN) algorithms are presented for face recognition. Face recognition is subdivided into two main stages: feature extraction and classifier. The suggested NN algorithms are the unsupervised Sanger principal component neural network (Sanger PCNN) and the self-organizing feature map (SOFM), which will be applied for features extraction of the frontal view of a face image. It is of interest to compare the unsupervised network with the traditional Eigenfaces technique. This paper presents an experimental comparison of the statistical Eigenfaces method for feature extraction and the unsupervised neural networks in order to evaluate the classification accuracies as comparison criteria. The classifier is done by the multilayer perceptron (MLP) neural network. Overcoming of the problem of the finite number of training samples per person is discussed. Experimental results are implemented on the Olivetti Research Laboratory database that contains variability in expression, pose, and facial details. The results show that the proposed method SOFM/MLP neural network is more efficient and robust than the Sanger PCNN/MLP and the Eigenfaces/MLP, when used a few number of training samples per person. As a result, it would be more applicable to utilize the SOFM/MLP NN in order to accomplish a higher level of accuracy within a recognition system.     

Enhanced Neural Network Modelling for a Real Multi-variable Chemical Process

Application of several Neural Network (NN) modelling techniques to model a Multi-Input Multi-Output (MIMO) nonlinear chemical process is investigated. The process is a laboratory scale chemical reactor with three inputs and three outputs. It typically represents industrial processes due to its nonlinearity, coupling effects and lack of a mathematical model. Different techniques have been used in collecting training data from the reactor. A novel method was used to select the model order and time-delay to determine the NN model input. A Radial Basis Function Network (RBFN) model was then developed. A Recursive Orthogonal Least Squares (ROLS) algorithm was applied as a numerically robust method to update the RBFN weight matrix. In this way, degradation of the modelling error due to ill-conditioning in the training data is avoided. Real data experiments show that the RBFN model developed has high accuracy.     

自由表面上球形浮体的拖曳运动仿真与试验

浮体在自由表面上的拖曳是一种常见的拖曳形式,由于自由表面上物体的运动情况相当复杂,目前对这种拖曳系统的设计和分析一般采用试验的方法.通过理论分析,对球形浮体在自由表面上的匀速直线拖曳运动建立了数学模型,其中对缆绳的分析采用Ablow-Schechter提出的有限差分模型,对球形浮体的分析采用水动力系数法,整个耦合系统用有限差分方法求解.之后,在拖曳水池进行了试验,试验结果表明,这种方法在拖曳速度比较低的情况下能够模拟真实情况,可以用来指导拖曳系统设计.