Sensitivity analysis of multilayer perceptron to input and weightperturbations

An important issue in the design and implementation of a neural network is the sensitivity of its output to input and weight perturbations. In this paper, we discuss the sensitivity of the most popular and general feedforward neural networks-multilayer perceptron (MLP). The sensitivity is defined as the mathematical expectation of the output errors of the MLP due to input and weight perturbations with respect to all input and weight values in a given continuous interval. The sensitivity for a single neuron is discussed first and an analytical expression that is a function of the absolute values of input and weight perturbations is approximately derived. Then an algorithm is given to compute the sensitivity for the entire MLP. As intuitively expected, the sensitivity increases with input and weight perturbations, but the increase has an upper bound that is determined by the structural configuration of the MLP, namely the number of neurons per layer and the number of layers. There exists an optimal value for the number of neurons in a layer, which yields the highest sensitivity value. The effect caused by the number of layers is quite unexpected. The sensitivity of a neural network may decrease at first and then almost keeps constant while the number increases.     

Spiking神经元输入脉冲扰动敏感性研究

脉冲神经网络是一种基于生物的网络模型,它的输入输出为具有时间特性的脉冲序列,其运行机制相比其他传统人工神经网络更加接近于生物神经网络。神经元之间通过脉冲序列传递信息,这些信息通过脉冲的激发时间编码能够更有效地发挥网络的学习性能。脉冲神经元的时间特性导致了其工作机制较为复杂,而spiking神经元的敏感性反映了当神经元输入发生扰动时输出的spike的变化情况,可以作为研究神经元内部工作机制的工具。不同于传统的神经网络,spiking神经元敏感性定义为输出脉冲的变化时刻个数与运行时间长度的比值,能直接反映出输入扰动对输出的影响程度。通过对不同形式的输入扰动敏感性的分析,可以看出spiking神经元的敏感性较为复杂,当全体突触发生扰动时,神经元为定值,而当部分突触发生扰动时,不同突触的扰动会导致不同大小的神经元敏感性。     

Single imputation with multilayer perceptron and multiple imputation combining multilayer perceptron and k-nearest neighbours for monotone patterns

The knowledge discovery process is supported by data files information gathered from collected data sets, which often contain errors in the form of missing values. Data imputation is the activity aimed at estimating values for missing data items. This study focuses on the development of automated data imputation models, based on artificial neural networks for monotone patterns of missing values. The present work proposes a single imputation approach relying on a multilayer perceptron whose training is conducted with different learning rules, and a multiple imputation approach based on the combination of multilayer perceptron and k-nearest neighbours. Eighteen real and simulated databases were exposed to a perturbation experiment with random generation of monotone missing data pattern. An empirical test was accomplished on these data sets, including both approaches (single and multiple imputations), and three classical single imputation procedures – mean/mode imputation, regression and hot-deck – were also considered. Therefore, the experiments involved five imputation methods. The results, considering different performance measures, demonstrated that, in comparison with traditional tools, both proposals improve the automation level and data quality offering a satisfactory performance.     

An optimum multilayer perceptron neural receiver for signaldetection

The M-input optimum likelihood-ratio receiver is generalized by considering the case of different signal amplitudes on the receiver primary input lines. Using the more general likelihood-ratio receiver as a reference, an equivalent optimum multilayer perceptron neural network (or neural receiver) is identified for detecting the presence of an M-dimensional target signal corrupted by bandlimited white Gaussian noise. Analytical results are supported by Monte Carlo simulation runs which indicate that the detection capability of the proposed neural receiver is not sensitive to the level of training or number of patterns in the training set.     

A multilayer perceptron solution to the match phase problem inrule-based artificial intelligence systems

In rule-based artificial intelligence (AI) planning, expert, and learning systems, it is often the case that the left-hand-sides of the rules must be repeatedly compared to the contents of some working memory. Normally, the intent is to determine which rules are relevant to the current situation (i.e., to find the conflict set). A technique using a multilayer perceptron to solve the match phase problem for rule-based AI systems is presented. A syntax for premise formulas (i.e., the left-hand-sides of the rules) is defined, and working memory is specified. From this, it is shown how to construct a multilayer perceptron that finds all of the rules which can be executed for the current situation in working memory. The complexity of the constructed multilayer perceptron is derived in terms of the maximum number of nodes and the required number of layers. A method for reducing the number of layers to at most three is presented     

The multilayer perceptron as an approximation to a Bayes optimaldiscriminant function

The multilayer perceptron, when trained as a classifier using backpropagation, is shown to approximate the Bayes optimal discriminant function. The result is demonstrated for both the two-class problem and multiple classes. It is shown that the outputs of the multilayer perceptron approximate the a posteriori probability functions of the classes being trained. The proof applies to any number of layers and any type of unit activation function, linear or nonlinear.     

An on-line hybrid learning algorithm for multilayer perceptron in identification problems

A hybrid learning algorithm for multilayered perceptrons (MLPs) and pattern-by-pattern training, based on optimized instantaneous learning rates and the recursive least squares method, is proposed. This hybrid solution is developed for on-line identification of process models based on the use of MLPs, and can speed up the learning process of the MLPs substantially, while simultaneously preserving the stability of the learning process. For illustration and test purposes the proposed algorithm is applied to the identification of a non-linear dynamic system.     

A multilayer perceptron neural network-based approach for the identification of responsiveness to interferon therapy in multiple sclerosis patients

Multiple sclerosis is an idiopathic inflammatory disease characterized by multiple focal lesions in the white matter of the central nervous system. Multiple sclerosis patients are usually treated with interferon-β, but disease activity decrease in only 30-40% of patients. In the attempt to differentiate between responders and non-responders, we screened the main genes involved in the interferon signaling pathway for 38 single nucleotide polymorphisms (SNPs) in a multiple sclerosis Caucasian population from South Italy. We then analyzed the data using a multilayer perceptron neural network-based approach, in which we evaluated the global weight of a set of SNPs localized in different genes and their association with response to interferon therapy through a feature selection procedure (a combination of automatic relevance determination and backward elimination). The neural approach appears to be a useful tool in identifying gene polymorphisms involved in the response of patients to interferon therapy: 2 out of 5 genes were identified as containing 4 out of 38 significant single nucleotide polymorphisms, with a global accuracy of 70% in predicting responder and non-responder patients.     

具有多层感知器力矩补偿的机器人自抗扰控制

本文针对机器人系统的控制特性,提出了一种基于自抗扰控制(ADRC)的关节控制算法,该算法可以克服传统控制算法中存在的如系统抗干扰能力弱,控制性能受限于建模精度,动态性能与稳态性能难以兼顾,控制律设计较为复杂等问题.针对受控系统特性给出了一套实际控制器的完整设计方法与参数整定方法,并根据控制性能指标设计优化函数完成了最优控制参数的优化,在系统参数辨识的基础上利用多层感知器(MLP)设计了对建模不确定性的补偿网络.数值仿真和实验结果均表明该算法能够实现机器人快速稳定的轨迹跟踪,具有良好的控制精度与很强的抗干扰能力,此外该算法不依赖于精确的系统模型,降低了实际设计和应用的难度,具有很好的工程应用价值.     

Genetic algorithm hybridized with multilayer perceptron to have an economical slope stability design

The present work aimed to evaluate and optimize the design of an artificial neural network (ANN) combined with an optimization algorithm of genetic algorithm (GA) for the calculation of slope stability safety factors (SF) in a pure cohesive slope. To make datasets of training and testing for the developed predictive models, 630 finite element limit equilibrium (FELE) analyses were performed. Similar to many artificial intelligence-based solutions, the database was involved in 189 testing datasets (e.g., 30% of the entire database) and 441 training datasets; for example, a range of 70% of the total database. Moreover, variables of multilayer perceptron (MLP) algorithm (for example, number of nodes in any hidden layer) and the algorithm of GA like population size was optimized by utilizing a series of trial and error process. The parameters in input, which were used in the analysis, consist of slope angle (β), setback distance ratio (b/B), applied stresses on the slope (F_y) and undrained shear strength of the cohesive soil (C_u) where the output was taken SF. The obtained network outputs for both datasets from MLP and GA-MLP models are evaluated according to many statistical indices. A total of 72 MLP trial and error (e.g., parameter study) the optimal architecture of 4 × 8 × 1 were determined for the MLP structure. Both proposed techniques result in a proper performance; however, according to the statistical indices, the GA–MLP model can somewhat accomplish the least mean square error (MSE) when compared to MLP. In an optimized GA–MLP network, coefficient of determination (R²) and root mean square error (RMSE) values of (0.975, and 0.097) and (0.969, and 0.107) were found, respectively, to both of the normalized training and testing datasets.

     

Enhanced training algorithms, and integrated training/architectureselection for multilayer perceptron networks

The standard backpropagation-based multilayer perceptron training algorithm suffers from a slow asymptotic convergence rate. Sophisticated nonlinear least-squares and quasi-Newton optimization techniques are used to construct enhanced multilayer perceptron training algorithms, which are then compared to the backpropagation algorithm in the context of several example problems. In addition, an integrated approach to training and architecture selection that uses the described enhanced algorithms is presented, and its effectiveness illustrated in the context of synthetic and actual pattern recognition problems.     

Singular perturbation for the dynamic interaction measure

The singular perturbation technique is used for the dynamic extension of the RGA (relative gain array) based on the state-space model. The dynamic interaction measure derived serves as a means of pairing input and output variables for the systems having substantially different time scales.     

Computation of Adalines' sensitivity to weight perturbation

In this paper, the sensitivity of Adalines to weight perturbation is discussed. According to the discrete feature of Adalines' input and output, the sensitivity is defined as the probability of an Adaline's erroneous outputs due to weight perturbation with respect to all possible inputs. By means of hypercube model and analytical geometry method, a heuristic algorithm is given to accurately compute the sensitivity. The accuracy of the algorithm is verified by computer simulations.     

A new approach to perceptron training

The training of perceptrons is discussed in the framework of nonsmooth optimization. An investigation of Rosenblatt's perceptron training rule shows that convergence or the failure to converge in certain situations can be easily understood in this framework. An algorithm based on results from nonsmooth optimization is proposed and its relation to the "constrained steepest descent" method is investigated. Numerical experiments verify that the "constrained steepest descent" algorithm may be further improved by the integration of methods from nonsmooth optimization.     

A sensitivity analysis for control systems having input time delays

The concept of singular sensitivity of a lumped linear system to small pure time delays in controls is introduced. From this, first order changes in a quadratic performance index can be calculated. A design method is proposed which calculates optimal linear feedback laws for the lumped system, but ensures that the effect of control delays is kept “small” in closed loop. Iterative computational algorithms are developed and simple examples presented. For stable plants, the computations always converge, while for unstable plants, the region of convergence limits the allowed delay to “non-dominant” values consistent with stabilisability under the given feedback structure.     

A multilayer perceptron model for the correlation between satellite data and soil vulnerability in the Ferlo,Senegal

ABSTRACT

Soil erosion processes which contribute to desertification and land degradation, constitute major environmental and social issues for the coming decades. This is particularly true in arid areas where rural populations mostly depend on soil ability to support crop production. Assessment of soil erosion across large and quite diverse areas is very difficult but crucial for planning and management of the natural resources. The purpose of this paper is to investigate a prediction model for soil vulnerability to erosion based on the use of the information contained in satellite images. Based on neural networks models, the used approach in this paper aims at checking a correlation between the digital content of satellite images and soil vulnerability factors: erosivity (R), the soil erodibility (K), and the slope length and steepness (LS); vulnerability (V) as described in the RUSLE model. Significant results have been obtained for R and K factors. This promising pilot study was conducted in South Ferlo, Senegal, a region with Sahelian environmental characteristics.     

A quantum backpropagation multilayer perceptron (QBMLP) for predicting iron adsorption capacity of calcareous soil from aqueous solution

Removal of miscible hazardous materials from aqueous solutions is an alarming problem for the environmental scientists. Several linear and nonlinear regression models like Langmuir, Freundlich, D–R, Tempkin isotherm models are in vogue for determining the adsorbing capacity of standard adsorbents used for this purpose.In this article, we propose a novel quantum inspired backpropagation multilayer perceptron (QBMLP) based on quantum gates (single qubit rotation gates and two qubit controlled-not gates) for the prediction of this adsorption behavior exhibited by calcareous soil oftentimes used in adsorbing miscible iron from aqueous solutions. The backpropagation learning formulae for the proposed QBMLP architecture has also been generalized for multiple number of layers in both field homogeneous and field heterogeneous configurations characterized by three standard activations, viz., sigmoid, tanh and tan 1.5h functions.Applications of the efficiency of the proposed QBMLP over the regression models are demonstrated with regards to the prediction behavior of the adsorption of iron by calcareous soil from an aqueous solution with effect to various characteristic adsorbent parameters. The adsorption process is considered to be a physical one since the activation energy (E_A) of ferrous ion adsorption is 9.469 kJ mol⁻¹ due to Arrhenius. Moreover, the thermodynamic parameters of Gibb's free energy (G₀), enthalpy (H₀) and entropy (S₀) values indicate it be spontaneous.Results indicate that QBMLP predicts the adsorption behavior of calcareous soil to a very closer extent thereby obviating the need for further regression/experimental analysis. Comparison with the performance of a similar classical multilayer perceptron (MLP) architecture also reveals the prediction and time efficiency of the proposed QBMLP architecture.     

An improved multilayer perceptron approach for detecting sugarcane yield production in IoT based smart agriculture

Internet of Things (IoT) as one of most powerful technologies can provides precision management and intelligent navigation for managers and manufacturing plants’ Smart agriculture to deal a good strategy for improving agricultural productions and maximizing farm efficiency. Sugar production is subsidiary to many diverse and various parameters. Due to a diverse variety of parameters and the lengthy process in precision agriculture, the analytical prediction is difficult and impossible. In such situations, using intelligent systems such as machine learning may be proposed as an alternative solution. This paper proposed an improved Multilayer Perceptron (MLP) approach to predict the amount of sugar yield production in IoT agriculture. Experimental results show that the proposed MLP algorithm has maximum accuracy of 99%, precision of 95%, recall of 96% and Minimum Mean Absolute Error (MAE) of 0.04% and Root mean square error (RMSE) of 0.006% for detecting sugarcane yield production in IoT Agriculture.     

Comparing evolutionary hybrid systems for design and optimization of multilayer perceptron structure along training parameters

In this paper we present a comparative study of several methods that combine evolutionary algorithms and local search to optimize multilayer perceptrons: A method that optimizes the architecture and initial weights of multilayer perceptrons; another that searches for training algorithm parameters, and finally, a co-evolutionary algorithm, introduced here, that handles the architecture, the network’s initial weights and the training algorithm parameters. Our aim is to determine how the co-evolutive method can obtain better results from the point of view of running time and classification ability. Experimental results show that the co-evolutionary method obtains similar or better results than the other approaches, requiring far less training epochs and thus, reducing running time.     

A combinatorial approach to understanding perceptron capabilities

This work investigates the classification capabilities of perceptrons which incorporate a single hidden layer of nodes from a theoretical viewpoint. In particular, the question of determining whether a given set can be realized as the decision region of such a network is considered. The main theoretic result demonstrates that the realizability of a set can be determined by restricting attention to any neighborhood of its boundary. This result is then used to identify general classes of realizable sets, and an example is given which shows that even though the realizability of a set might be readily discerned, the construction of an appropriate perceptron architecture may be complicated.