Robust radial basis function neural networks

Function approximation has been found in many applications. The radial basis function (RBF) network is one approach which has shown a great promise in this sort of problems because of its faster learning capacity. A traditional RBF network takes Gaussian functions as its basis functions and adopts the least-squares criterion as the objective function, However, it still suffers from two major problems. First, it is difficult to use Gaussian functions to approximate constant values. If a function has nearly constant values in some intervals, the RBF network will be found inefficient in approximating these values. Second, when the training patterns incur a large error, the network will interpolate these training patterns incorrectly. In order to cope with these problems, an RBF network is proposed in this paper which is based on sequences of sigmoidal functions and a robust objective function. The former replaces the Gaussian functions as the basis function of the network so that constant-valued functions can be approximated accurately by an RBF network, while the latter is used to restrain the influence of large errors. Compared with traditional RBF networks, the proposed network demonstrates the following advantages: (1) better capability of approximation to underlying functions; (2) faster learning speed; (3) better size of network; (4) high robustness to outliers.     

Generalised Gaussian radial basis function neural networks

The mixed use of different shapes of radial basis functions (RBFs) in radial basis functions neural networks (RBFNNs) is investigated in this paper. For this purpose, we propose the use of a generalised version of the standard RBFNN, based on the generalised Gaussian distribution. The generalised radial basis function (GRBF) proposed in this paper is able to reproduce other different radial basis functions (RBFs) by changing a real parameter τ. In the proposed methodology, a hybrid evolutionary algorithm (HEA) is employed to estimate the number of hidden neuron, the centres, type and width of each RBF associated with each radial unit. In order to test the performance of the proposed methodology, an experimental study is presented with 20 datasets from the UCI repository. The GRBF neural network (GRBFNN) was compared to RBFNNs with Gaussian, Cauchy and inverse multiquadratic RBFs in the hidden layer and to other classifiers, including different RBFNN design methods, support vector machines (SVMs), a sparse probabilistic classifier (sparse multinominal logistic regression, SMLR) and other non-sparse (but regularised) probabilistic classifiers (regularised multinominal logistic regression, RMLR). The GRBFNN models were found to be better than the alternative RBFNNs for almost all datasets, producing the highest mean accuracy rank.     

Kernel orthonormalization in radial basis function neural networks

This paper deals with optimization of the computations involved in training radial basis function (RBF) neural networks. The main contribution of the reported work is the method for network weights calculation, in which the key idea is to transform the RBF kernels into an orthonormal set of functions (using the standard Gram-Schmidt orthogonalization). This significantly reduces the computing time if the RBF training scheme, which relies on adding one kernel hidden node at a time to improve network performance, is adopted. Another property of the method is that, after the RBF network weights are computed, the original network structure can be restored back. An additional strength of the method is the possibility to decompose the proposed computing task into a number of parallel subtasks so gaining further savings on computing time. Also, the proposed weight calculation technique has low storage requirements. These features make the method very attractive for hardware implementation. The paper presents a detailed derivation of the proposed network weights calculation procedure and demonstrates its validity for RBF network training on a number of data classification and function approximation problems.     

On the efficiency of the orthogonal least squares training methodfor radial basis function networks

The efficiency of the orthogonal least squares (OLS) method for training approximation networks is examined using the criterion of energy compaction. We show that the selection of basis vectors produced by the procedure is not the most compact when the approximation is performed using a nonorthogonal basis. Hence, the algorithm does not produce the smallest possible networks for a given approximation error. Specific examples are given using the Gaussian radial basis functions type of approximation networks.     

Output value-based initialization for radial basis function neural networks

The use of Radial Basis Function Neural Networks (RBFNNs) to solve functional approximation problems has been addressed many times in the literature. When designing an RBFNN to approximate a function, the first step consists of the initialization of the centers of the RBFs. This initialization task is very important because the rest of the steps are based on the positions of the centers. Many clustering techniques have been applied for this purpose achieving good results although they were constrained to the clustering problem. The next step of the design of an RBFNN, which is also very important, is the initialization of the radii for each RBF. There are few heuristics that are used for this problem and none of them use the information provided by the output of the function, but only the centers or the input vectors positions are considered. In this paper, a new algorithm to initialize the centers and the radii of an RBFNN is proposed. This algorithm uses the perspective of activation grades for each neuron, placing the centers according to the output of the target function. The radii are initialized using the center’s positions and their activation grades so the calculation of the radii also uses the information provided by the output of the target function. As the experiments show, the performance of the new algorithm outperforms other algorithms previously used for this problem.     

Descartes' rule of signs for radial basis function neural networks

We establish versions of Descartes' rule of signs for radial basis function (RBF) neural networks. The RBF rules of signs provide tight bounds for the number of zeros of univariate networks with certain parameter restrictions. Moreover, they can be used to infer that the Vapnik-Chervonenkis (VC) dimension and pseudodimension of these networks are no more than linear. This contrasts with previous work showing that RBF neural networks with two or more input nodes have superlinear VC dimension. The rules also give rise to lower bounds for network sizes, thus demonstrating the relevance of network parameters for the complexity of computing with RBF neural networks.     

On the statistical physics of radial basis function networks

Techniques from statistical physics have been applied successfully in recent years to the analysis of the generalization performance of neural networks. However, most of the analysis to date has been for perceptron-like networks or simple generalizations thereof such as committee machines, and none of the networks studied are used to any significant extent in practice. This letter presents results obtained in applying techniques from statistical physics to a popular class of neural networks that has been used successfully in many practical applications: the Gaussian radial basis function networks. We obtain expressions for the learning curves exhibited by these networks in the high-temperature limit for both realizable and unrealizable target rules.     

A classification technique based on radial basis function neural networks

In this paper, a new classification method is proposed based on the radial basis function (RBF) neural network architecture. The method is particularly useful for manufacturing processes, in cases where on-line sensors for classifying the product quality are not available. More specifically, the fuzzy means algorithm is employed on a set of training data, where the input data refer to variables that are measured on-line and the output data correspond to quality variables that are classified by human experts. The produced neural network model acts as an artificial sensor that is able to classify the product quality in real time. The proposed method is illustrated through an application to real data collected from a paper machine. The method produces successful results and outperforms a number of classifiers, which are based on the feedforward neural network (FNN) architecture.     

High-speed face recognition using self-adaptive radial basis function neural networks

In this work, we have proposed a self-adaptive radial basis function neural network (RBFNN)-based method for high-speed recognition of human faces. It has been seen that the variations between the images of a person, under varying pose, facial expressions, illumination, etc., are quite high. Therefore, in face recognition problem to achieve high recognition rate, it is necessary to consider the structural information lying within these images in the classification process. In the present study, it has been realized by modeling each of the training images as a hidden layer neuron in the proposed RBFNN. Now, to classify a facial image, a confidence measure has been imposed on the outputs of the hidden layer neurons to reduce the influences of the images belonging to other classes. This process makes the RBFNN as self-adaptive for choosing a subset of the hidden layer neurons, which are in close neighborhood of the input image, to be considered for classifying the input image. The process reduces the computation time at the output layer of the RBFNN by neglecting the ineffective radial basis functions and makes the proposed method to recognize face images in high speed and also in interframe period of video. The performance of the proposed method has been evaluated on the basis of sensitivity and specificity on two popular face recognition databases, the ORL and the UMIST face databases. On the ORL database, the best average sensitivity (recognition) and specificity rates are found to be 97.30 and 99.94%, respectively using five samples per person in the training set. Whereas, on the UMIST database, the above quantities are found to be 96.36 and 99.81%, respectively using eight samples per person in the training set. The experimental results indicate that the proposed method outperforms some of the face recognition approaches.     

Reformulated radial basis function neural networks with adjustable weighted norms

This article presents a new family of reformulated radial basis function (RBF) neural networks that employ adjustable weighted norms to measure the distance between the training vectors and the centers of the radial basis functions. The reformulated RBF model introduced in this article incorporates norm weights that can be updated during learning to facilitate the implementation of the desired input‐output mapping. Experiments involving classification and function approximation tasks verify that the proposed RBF neural networks outperform conventional RBF neural networks and reformulated RBF neural networks employing fixed Euclidean norms. Reformulated RBF neural networks with adjustable weighted norms are also strong competitors to conventional feedforward neural networks in terms of performance, implementation simplicity, and training speed. © 2003 Wiley Periodicals, Inc.     

Object tracking by mean shift and radial basis function neural networks

In this paper, a tracker based on mean shift and radial basis function neural networks called MS-RBF is addressed. As its name implies, two independent trackers have been combined and linked together. The mean shift algorithm estimates the target’s location within only two iterations. The scale and orientation of target are computed by exploiting 2-D correlation coefficient between reference and target candidate histograms instead of using Bhattacharyya coefficient. A code optimization strategy, named multiply–add–accumulate (MAC), is proposed to remove useless memory occupation and programmatic operations. MAC implementation has reduced computational load and made overall tracking process faster. The second tracker “RBFNN” has an input feature vector that contains variables such as local contrast, color histogram, gradient, intensity, and spatial frequency. The neural network learns the color and texture features from the target and background. Then, this information is used to detect and track the object in other frames. The neural network employs Epanechnikov activation functions. The features extracted in any frame are clustered by Fuzzy C-Means clustering which produces the means and variances of the clusters. The experimental results show that the proposed tracker can resist to different types of occlusions, sudden movement, and shape deformations.     

Robust training of radial basis networks

Robust training of radial-basis networks under non-normally distributed noise is considered. The simulation results show that multistep projection training algorithms minimizing various forms of module criteria are rather efficient in this case.     

Genetic algorithm-trained radial basis function neural networks for modelling photovoltaic panels

Radial basis function neural networks (RBFNs) can be applied to model the I–V characteristics and maximum power points (MPPs) of photovoltaic (PV) panels. The key issue for training an RBFN lies in determining the number of radial basis functions (RBFs) in the hidden layer. This paper presents a genetic algorithms-based RBFN training scheme to search for the optimal number of RBFs using only the input samples of a PV panel. The performance of the trained RBFN is comparable with that of the conventional model and the training algorithm is computationally efficient. The trained RBFNs have been applied to predict MPPs of two different practical PV panels. The results obtained are accurate enough for applying the models to control the PV systems for tracking the optimal power points.     

Invariance in radial basis function neural networks in human face classification

This paper is concerned with the types of invariance exhibited by Radial Basis Function (RBF) neural networks when used for human face classification, and the generalisation abilities arising from this behaviour. Experiments using face images in ranges from face-on to profile show the RBF network's invariance to 2-D shift, scale and y-axis rotation. Finally, the suitability of RBF techniques for future, more automated face classification purposes is discussed.     

Face recognition with radial basis function (RBF) neural networks

A general and efficient design approach using a radial basis function (RBF) neural classifier to cope with small training sets of high dimension, which is a problem frequently encountered in face recognition, is presented. In order to avoid overfitting and reduce the computational burden, face features are first extracted by the principal component analysis (PCA) method. Then, the resulting features are further processed by the Fisher's linear discriminant (FLD) technique to acquire lower-dimensional discriminant patterns. A novel paradigm is proposed whereby data information is encapsulated in determining the structure and initial parameters of the RBF neural classifier before learning takes place. A hybrid learning algorithm is used to train the RBF neural networks so that the dimension of the search space is drastically reduced in the gradient paradigm. Simulation results conducted on the ORL database show that the system achieves excellent performance both in terms of error rates of classification and learning efficiency.     

Speech recognition using pulse-coupled neural networks with a radial basis function

We have developed a novel pulse-coupled neural network (PCNN) for speech recognition. One of the advantages of the PCNN is in its biologically based neural dynamic structure using feedback connections. To recall the memorized pattern, a radial basis function (RBF) is incorporated into the proposed PCNN. Simulation results show that the PCNN with a RBF can be useful for phoneme recognition. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

A review of genetic algorithms applied to training radial basis function networks

The problems associated with training feedforward artificial neural networks (ANNs) such as the multilayer perceptron (MLP) network and radial basis function (RBF) network have been well documented. The solutions to these problems have inspired a considerable amount of research, one particular area being the application of evolutionary search algorithms such as the genetic algorithm (GA). To date, the vast majority of GA solutions have been aimed at the MLP network. This paper begins with a brief overview of feedforward ANNs and GAs followed by a review of the current state of research in applying evolutionary techniques to training RBF networks.     

Median radial basis function neural network

Radial basis functions (RBFs) consist of a two-layer neural network, where each hidden unit implements a kernel function. Each kernel is associated with an activation region from the input space and its output is fed to an output unit. In order to find the parameters of a neural network which embeds this structure we take into consideration two different statistical approaches. The first approach uses classical estimation in the learning stage and it is based on the learning vector quantization algorithm and its second-order statistics extension. After the presentation of this approach, we introduce the median radial basis function (MRBF) algorithm based on robust estimation of the hidden unit parameters. The proposed algorithm employs the marginal median for kernel location estimation and the median of the absolute deviations for the scale parameter estimation. A histogram-based fast implementation is provided for the MRBF algorithm. The theoretical performance of the two training algorithms is comparatively evaluated when estimating the network weights. The network is applied in pattern classification problems and in optical flow segmentation.     

基于径向基函数神经网络的转子系统裂纹故障诊断

结合基于模型的转子系统诊断技术和径向基函数(RBF)神经网络在辨识非线性系统动态时的优势,本文提出了一种新的转子系统裂纹故障诊断方法.该方法采用RBF神经网络对裂纹转子系统的未知动态进行辨识,实现部分神经网络权值收敛到最优值以及神经网络对系统未知动态的局部准确逼近;诊断过程中利用已辨识的信息实现转子系统裂纹故障的快速检测与分离.所提方法尤其适用于微小裂纹的在线检测与定量识别.最后,以Jeffcott转子系统裂纹故障诊断为例进行仿真,仿真结果验证了所提方法的有效性.