Constructing and training feed-forward neural networks for pattern classification

A new approach of constructing and training neural networks for pattern classification is proposed. Data clusters are generated and trained sequentially based on distinct local subsets of the training data. Obtained clusters are then used to construct a feed-forward network, which is further trained using standard algorithms operating on the global training set. The network obtained using this approach effectively inherits the knowledge from the local training procedure before improving on its generalization ability through the subsequent global training. Various experiments demonstrate the superiority of this approach over competing methods.     

RBF neural networks for solving the inverse problem of backscattering spectra

This paper investigates a new method to solve the inverse problem of Rutherford backscattering (RBS) data. The inverse problem is to determine the sample structure information from measured spectra, which can be defined as a function approximation problem. We propose using radial basis function (RBF) neural networks to approximate an inverse function. Each RBS spectrum, which may contain up to 128 data points, is compressed by the principal component analysis, so that the dimensionality of input data and complexity of the network are reduced significantly. Our theoretical consideration is tested by numerical experiments with the example of the SiGe thin film sample and corresponding backscattering spectra. A comparison of the RBF method with multilayer perceptrons reveals that the former has better performance in extracting structural information from spectra. Furthermore, the proposed method can handle redundancies properly, which are caused by the constraint of output variables. This study is the first method based on RBF to deal with the inverse RBS data analysis problem.     

Combined projection and kernel basis functions for classification in evolutionary neural networks

This paper proposes a hybrid neural network model using a possible combination of different transfer projection functions (sigmoidal unit, SU, product unit, PU) and kernel functions (radial basis function, RBF) in the hidden layer of a feed-forward neural network. An evolutionary algorithm is adapted to this model and applied for learning the architecture, weights and node typology. Three different combined basis function models are proposed with all the different pairs that can be obtained with SU, PU and RBF nodes: product–sigmoidal unit (PSU) neural networks, product–radial basis function (PRBF) neural networks, and sigmoidal–radial basis function (SRBF) neural networks; and these are compared to the corresponding pure models: product unit neural network (PUNN), multilayer perceptron (MLP) and the RBF neural network. The proposals are tested using ten benchmark classification problems from well known machine learning problems. Combined functions using projection and kernel functions are found to be better than pure basis functions for the task of classification in several datasets.     

Classification of incomplete feature vectors by radial basis function networks

The paper describes the use of radial basis function neural networks with Gaussian basis functions to classify incomplete feature vectors. The method uses the fact that any marginal distribution of a Gaussian distribution can be determined from the mean vector and covariance matrix of the joint distribution.     

Fast evaluation of neural networks via confidence rating

Neural networks have become very useful tools for input–output knowledge discovery. However, some of the most powerful schemes require very complex machines and, thus, a large amount of calculation. This paper presents a general technique to reduce the computational burden associated with the operational phase of most neural networks that calculate their output as a weighted sum of terms, which comprises a wide variety of schemes, such as Multi-Net or Radial Basis Function networks. Basically, the idea consists on sequentially evaluating the sum terms, using a series of thresholds which are associated with the confidence that a partial output will coincide with the overall network classification criterion. Furthermore, we design some procedures for conveniently sorting out the network units, so that the most important ones are evaluated first. The possibilities of this strategy are illustrated with some experiments on a benchmark of binary classification problems, using RealAdaboost and RBF networks, which show that important computational savings can be achieved without significant degradation in terms of recognition accuracy.     

A hybrid radial basis function and data envelopment analysis neural network for classification

We propose a hybrid radial basis function network-data envelopment analysis (RBFN-DEA) neural network for classification problems. The procedure uses the radial basis function to map low dimensional input data from input space ℜ to a high dimensional ℜ⁺ feature space where DEA can be used to learn the classification function. Using simulated datasets for a non-linearly separable binary classification problem, we illustrate how the RBFN-DEA neural network can be used to solve it. We also show how asymmetric misclassification costs can be incorporated in the hybrid RBFN-DEA model. Our preliminary experiments comparing the RBFN-DEA with feed forward and probabilistic neural networks show that the RBFN-DEA fares very well.     

Improving the generalization performance of RBF neural networks using a linear regression technique

In this paper we present a method for improving the generalization performance of a radial basis function (RBF) neural network. The method uses a statistical linear regression technique which is based on the orthogonal least squares (OLS) algorithm. We first discuss a modified way to determine the center and width of the hidden layer neurons. Then, substituting a QR algorithm for the traditional Gram–Schmidt algorithm, we find the connected weight of the hidden layer neurons. Cross-validation is utilized to determine the stop training criterion. The generalization performance of the network is further improved using a bootstrap technique. Finally, the solution method is used to solve a simulation and a real problem. The results demonstrate the improved generalization performance of our algorithm over the existing methods.     

MIMLRBF: RBF neural networks for multi-instance multi-label learning

In multi-instance multi-label learning (MIML), each example is not only represented by multiple instances but also associated with multiple class labels. Several learning frameworks, such as the traditional supervised learning, can be regarded as degenerated versions of MIML. Therefore, an intuitive way to solve MIML problem is to identify its equivalence in its degenerated versions. However, this identification process would make useful information encoded in training examples get lost and thus impair the learning algorithm's performance. In this paper, RBF neural networks are adapted to learn from MIML examples. Connections between instances and labels are directly exploited in the process of first layer clustering and second layer optimization. The proposed method demonstrates superior performance on two real-world MIML tasks.     

A study on the use of statistical tests for experimentation with neural networks: Analysis of parametric test conditions and non-parametric tests

In this paper, we focus on the experimental analysis on the performance in artificial neural networks with the use of statistical tests on the classification task. Particularly, we have studied whether the sample of results from multiple trials obtained by conventional artificial neural networks and support vector machines checks the necessary conditions for being analyzed through parametrical tests. The study is conducted by considering three possibilities on classification experiments: random variation in the selection of test data, the selection of training data and internal randomness in the learning algorithm.The results obtained state that the fulfillment of these conditions are problem-dependent and indefinite, which justifies the need of using non-parametric statistics in the experimental analysis.