Sensitivity analysis of multilayer perceptron with differentiableactivation functions

In a neural network, many different sets of connection weights can approximately realize an input-output mapping. The sensitivity of the neural network varies depending on the set of weights. For the selection of weights with lower sensitivity or for estimating output perturbations in the implementation, it is important to measure the sensitivity for the weights. A sensitivity depending on the weight set in a single-output multilayer perceptron (MLP) with differentiable activation functions is proposed. Formulas are derived to compute the sensitivity arising from additive/multiplicative weight perturbations or input perturbations for a specific input pattern. The concept of sensitivity is extended so that it can be applied to any input patterns. A few sensitivity measures for the multiple output MLP are suggested. For the verification of the validity of the proposed sensitivities, computer simulations have been performed, resulting in good agreement between theoretical and simulation outcomes for small weight perturbations.     

Fingerprint classification using a fuzzy multilayer perceptron

A fuzzy multilayer perceptron is used for the classification of fingerprint patterns. The input vector consists of texturebased features along with some directional features. The output vector is defined in terms of membership values to the three classes, viz.Whorl, Left Loop and Right Loop. Perturbation is produced randomly at pixel locations to generate noisy patterns. This helps to demonstrate the ability of the model in handling distorted fingerprint images. A study is made on the effect of reducing the number of input features while increasing the size of the network on its recognition performance.     

Human face recognition using fuzzy multilayer perceptron

In this work a novel method for human face recognition that is based on fuzzy neural network has been presented. Here, Gabor wavelet transformation is used for extraction of features from face images as it deals with images in spatial as well as in frequency domain to capture different local orientations and scales efficiently. In face recognition problem multilayer perceptron (MLP) has already been adopted owing to its efficiency, but it does not capture overlapping and nonlinear manifolds of faces which exhibit different variations in illumination, expression, pose, etc. A fuzzy MLP on the other hand performs better than an MLP because fuzzy MLP can identify decision surfaces in case of nonlinear overlapping classes, whereas an MLP is restricted to crisp boundaries only. In the present work, a new approach for fuzzification of the feature sets obtained through Gabor wavelet transforms has been discussed. The feature vectors thus obtained are classified using a newly designed fuzzified MLP. The system has been tested on a composite database (DB-C) consisting of the ORL face database and another face database created for this purpose and a recognition rate of 97.875% with fuzzy MLP against a recognition rate of only 81.25% with MLP whose feature vectors were also obtained through same Gabor wavelet transforms has been obtained.     

On the classification of multispectral satellite images using the multilayer perceptron

For classifying multispectral satellite images, a multilayer perceptron (MLP) is trained using either (i) ground truth data or (ii) the output of a K-means clustering program or (iii) both, as applied to certain representative parts of the given data set. In the second case, different sets of clustered image outputs, which have been checked against actual ground truth data wherever available, are used for testing the MLP. The cover classes are, typically, different types of (a) vegetation (including forests and agriculture); (b) soil (including mountains, highways and rocky terrain); and (c) water bodies (including lakes). Since the extent of ground truth may not be sufficient for training neural networks, the proposed procedure (of using clustered output images) is believed to be novel and advantageous. Moreover, it is found that the MLP offers an accuracy of more than 99% when applied to the multispectral satellite images in our library. As importantly, comparison with some recent results shows that the proposed application of the MLP leads to a more accurate and faster classification of multispectral image data.     

Predicting the helpfulness of online reviews using multilayer perceptron neural networks

With the great development of e-commerce, users can create and publish a wealth of product information through electronic communities. It is difficult, however, for manufacturers to discover the best reviews and to determine the true underlying quality of a product due to the sheer volume of reviews available for a single product. The goal of this paper is to develop models for predicting the helpfulness of reviews, providing a tool that finds the most helpful reviews of a given product. This study intends to propose HPNN (a helpfulness prediction model using a neural network), which uses a back-propagation multilayer perceptron neural network (BPN) model to predict the level of review helpfulness using the determinants of product data, the review characteristics, and the textual characteristics of reviews. The prediction accuracy of HPNN was better than that of a linear regression analysis in terms of the mean-squared error. HPNN can suggest better determinants which have a greater effect on the degree of helpfulness. The results of this study will identify helpful online reviews and will effectively assist in the design of review sites.     

Nonlinear system identification using BBO-based multilayer perceptron network method

Microsystem Technologies - Recently, nonlinear system identification has received increasingly more attention due to its promising applications in engineering fields. It has become a challenging...     

High-order and multilayer perceptron initialization

Proper initialization is one of the most important prerequisites for fast convergence of feedforward neural networks like high-order and multilayer perceptrons. This publication aims at determining the optimal variance (or range) for the initial weights and biases, which is the principal parameter of random initialization methods for both types of neural networks. An overview of random weight initialization methods for multilayer perceptrons is presented. These methods are extensively tested using eight real-world benchmark data sets and a broad range of initial weight variances by means of more than 30000 simulations, in the aim to find the best weight initialization method for multilayer perceptrons. For high-order networks, a large number of experiments (more than 200000 simulations) was performed, using three weight distributions, three activation functions, several network orders, and the same eight data sets. The results of these experiments are compared to weight initialization techniques for multilayer perceptrons, which leads to the proposal of a suitable initialization method for high-order perceptrons. The conclusions on the initialization methods for both types of networks are justified by sufficiently small confidence intervals of the mean convergence times.     

Multimedia-oriented action recognition in Smart City-based IoT using multilayer perceptron

The Internet of Things (IoT) devices and technologies for smart city applications produces a vast amount of multimedia data (e.g., audio, video, image, text and sensorial data), such big data are difficult to handle with traditional techniques and algorithms. The emerging machine learning techniques have the potential to facilitate the development of a new class of applications that can deal with such multimedia big data. Recently, Activity Recognition systems suggest using of multimedia data to detect daily actions, since it provides more accurate patterns; prevent the arising complain on privacy issues (in case of using audio-base data) and able to work on a big data. In this paper, we propose a Deep Learning (DL) methodology for classifying audio data that is based on multilayer perceptron neural networks. The contributions of our work are to propose an efficient design of the network topology including hidden layers, neurons, and the fitness function. In addition, the proposed methodology contributed in producing high performance classifier in terms of accuracy and f-measure. The experiments have been conducted on four large audio-datasets that have been collected to represent different modalities in a smart city. The results indicated that the proposed methodology achieved high performance as compared to the state-of-the-art machine learning techniques.

     

On the closure of the set of functions that can be realized by agiven multilayer perceptron

Given a multilayer perceptron (MLP) with a fixed architecture, there are functions that can be approximated up to any degree of accuracy, without having to increase the number of the hidden nodes. Those functions belong to the closure F of the set F of the maps realizable by the MLP. In this paper, we give a list of maps with this property. In particular, it is proven that: 1) rational functions belongs to F for networks with inverse tangent activation function; and 2) products of polynomials and exponentials belongs to F for networks with sigmoid activation function. Moreover, for a restricted class of MLPs, we prove that the list is complete and give an analytic definition of F.     

Prediction of the response under impact of steel armours using a multilayer perceptron

This article puts forward the results obtained when using a neural network as an alternative to classical methods (simulation and experimental testing) in the prediction of the behaviour of steel armours against high-speed impacts. In a first phase, a number of impact cases are randomly generated, varying the values of the parameters which define the impact problem (radius, length and velocity of the projectile; thickness of the protection). After simulation of each case using a finite element code, the above-mentioned parameters and the results of the simulation (residual velocity and residual mass of the projectile) are used as input and output data to train and validate a neural network. In addition, the number of training cases needed to arrive at a given predictive error is studied. The results are satisfactory, this alternative providing a highly recommended option for armour design tasks, due to its simplicity of handling, low computational cost and efficiency.     

Visualization of learning in multilayer perceptron networks using principal component analysis

This paper is concerned with the use of scientific visualization methods for the analysis of feedforward neural networks (NNs). Inevitably, the kinds of data associated with the design and implementation of neural networks are of very high dimensionality, presenting a major challenge for visualization. A method is described using the well-known statistical technique of principal component analysis (PCA). This is found to be an effective and useful method of visualizing the learning trajectories of many learning algorithms such as backpropagation and can also be used to provide insight into the learning process and the nature of the error surface.     

Clustered defect detection of high quality chips using self-supervised multilayer perceptron

During electrical testing, each die on a wafer must be tested to determine whether it functions as originally designed. When defects, including scratches, stains or localized failed patterns, are clustered on the wafer, the tester may not detect all of the defective dies in the flawed area. A testing factory must assign a few workers to check the wafers and hand-mark the defective dies in the flawed region or close to the flawed region, to ensure that no defective die is present in the final assembly. This work presents an automatic wafer-scale defect cluster identifier that uses a multilayer perceptron to detect the defect cluster and mark all of the defective dies. The proposed identifier is compared with an existing tool used in industry. The experimental results confirm that the proposed algorithm is more effective at identifying defects and outperforms the present approach.     

Single imputation with multilayer perceptron and multiple imputation combining multilayer perceptron and k-nearest neighbours for monotone patterns

The knowledge discovery process is supported by data files information gathered from collected data sets, which often contain errors in the form of missing values. Data imputation is the activity aimed at estimating values for missing data items. This study focuses on the development of automated data imputation models, based on artificial neural networks for monotone patterns of missing values. The present work proposes a single imputation approach relying on a multilayer perceptron whose training is conducted with different learning rules, and a multiple imputation approach based on the combination of multilayer perceptron and k-nearest neighbours. Eighteen real and simulated databases were exposed to a perturbation experiment with random generation of monotone missing data pattern. An empirical test was accomplished on these data sets, including both approaches (single and multiple imputations), and three classical single imputation procedures – mean/mode imputation, regression and hot-deck – were also considered. Therefore, the experiments involved five imputation methods. The results, considering different performance measures, demonstrated that, in comparison with traditional tools, both proposals improve the automation level and data quality offering a satisfactory performance.     

CATSMLP: toward a robust and interpretable multilayer perceptron with sigmoid activation functions.

Enhancing the robustness and interpretability of a multilayer perceptron (MLP) with a sigmoid activation function is a challenging topic. As a particular MLP, additive TS-type MLP (ATSMLP) can be interpreted based on single-stage fuzzy IF-THEN rules, but its robustness will be degraded with the increase in the number of intermediate layers. This paper presents a new MLP model called cascaded ATSMLP (CATSMLP), where the ATSMLPs are organized in a cascaded way. The proposed CATSMLP is a universal approximator and is also proven to be functionally equivalent to a fuzzy inference system based on syllogistic fuzzy reasoning. Therefore, the CATSMLP may be interpreted based on syllogistic fuzzy reasoning in a theoretical sense. Meanwhile, due to the fact that syllogistic fuzzy reasoning has distinctive advantage over single-stage IF-THEN fuzzy reasoning in robustness, this paper proves in an indirect way that the CATSMLP is more robust than the ATSMLP in an upper-bound sense. Several experiments were conducted to confirm such a claim.     

Classification error of multilayer perceptron neural networks

In subject classification, artificial neural networks (ANNS) are efficient and objective classification methods. Thus, they have been successfully applied to the numerous classification fields. Sometimes, however, classifications do not match the real world, and are subjected to errors. These problems are caused by the nature of ANNS. We discuss these on multilayer perceptron neural networks. By studying of these problems, it helps us to have a better understanding on its classification.     

Acceleration of multilayer perceptron training with CUDA

CUDA acceleration of Broyden-Fletcher-Goldfarb-Shanno (BFGS) training algorithm is described. Speedup in comparison with reference single thread CPU realization is ～18.     

Empirical transfer function determination by the use of multilayer perceptron

Neural Networks are relevant statistical methods to extract information from data when physical phenomena are very complicated and cannot be described in terms of theoretical analysis. Scatterometers are active microwave radar which accurately measure the power of the backscatter signal versus incident signal in order to calculate the normalized radar cross section (σ₀) of the ocean surface. We use multilayer perceptrons in order to determine the Geophysical Model Function and to estimate the variability of the signal of ERS-1, ERS-2 and NSCAT scatterometers.     

Application of the recurrent multilayer perceptron in modelingcomplex process dynamics

A nonlinear dynamic model is developed for a process system, namely a heat exchanger, using the recurrent multilayer perceptron network as the underlying model structure. The perceptron is a dynamic neural network, which appears effective in the input-output modeling of complex process systems. Dynamic gradient descent learning is used to train the recurrent multilayer perceptron, resulting in an order of magnitude improvement in convergence speed over a static learning algorithm used to train the same network. In developing the empirical process model the effects of actuator, process, and sensor noise on the training and testing sets are investigated. Learning and prediction both appear very effective, despite the presence of training and testing set noise, respectively. The recurrent multilayer perceptron appears to learn the deterministic part of a stochastic training set, and it predicts approximately a moving average response of various testing sets. Extensive model validation studies with signals that are encountered in the operation of the process system modeled, that is steps and ramps, indicate that the empirical model can substantially generalize operational transients, including accurate prediction of instabilities not in the training set. However, the accuracy of the model beyond these operational transients has not been investigated. Furthermore, online learning is necessary during some transients and for tracking slowly varying process dynamics. Neural networks based empirical models in some cases appear to provide a serious alternative to first principles models.     

An optimum multilayer perceptron neural receiver for signaldetection

The M-input optimum likelihood-ratio receiver is generalized by considering the case of different signal amplitudes on the receiver primary input lines. Using the more general likelihood-ratio receiver as a reference, an equivalent optimum multilayer perceptron neural network (or neural receiver) is identified for detecting the presence of an M-dimensional target signal corrupted by bandlimited white Gaussian noise. Analytical results are supported by Monte Carlo simulation runs which indicate that the detection capability of the proposed neural receiver is not sensitive to the level of training or number of patterns in the training set.     

Evaluation of an optimal design method for a multilayer perceptron by using the design of experiments

We evaluated the performance of an optimal design method for a multilayer perceptron (MLP) by using the design of experiments (DOE). In our previous work, we proposed an optimal design method for MLPs in order to determine the optimal values of such parameters as the number of neurons in the hidden layers and the learning rates. In this article, we evaluate the performance of the proposed design method through a comparison with a genetic algorithm (GA)-based design method. We target an optimal design of MLPs with six layers. We also evaluate the proposed designed method in terms of calculating the amount of optimization. Through the above-mentioned evaluation and analysis, we aim at improving the proposed design method in order to obtain an optimal MLP with less effort.