A fast identification algorithm for box-cox transformation based radial basis function neural network

In this letter, a Box-Cox transformation-based radial basis function (RBF) neural network is introduced using the RBF neural network to represent the transformed system output. Initially a fixed and moderate sized RBF model base is derived based on a rank revealing orthogonal matrix triangularization (QR decomposition). Then a new fast identification algorithm is introduced using Gauss-Newton algorithm to derive the required Box-Cox transformation, based on a maximum likelihood estimator. The main contribution of this letter is to explore the special structure of the proposed RBF neural network for computational efficiency by utilizing the inverse of matrix block decomposition lemma. Finally, the Box-Cox transformation-based RBF neural network, with good generalization and sparsity, is identified based on the derived optimal Box-Cox transformation and a D-optimality-based orthogonal forward regression algorithm. The proposed algorithm and its efficacy are demonstrated with an illustrative example in comparison with support vector machine regression.     

Adaptive annealing learning algorithm-based robust wavelet neural networks for function approximation with outliers

In this study, a robust wavelet neural network (WNN) is proposed to approximate functions with outliers. In the proposed methodology, firstly, support vector machine with wavelet kernel function (WSVM) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs. Then, an adaptive annealing learning algorithm (AALA) is adopted to accommodate the translations, the dilations, and the weights of the WNNs. In the learning procedure, the AALA is proposed to overcome the problems of initialization and the cut-off points in the robust learning algorithm. Hence, when an initial structure of the WNNs is determined by a support vector regression (SVR) approach, the WNNs with AALA (AALA-WNNs) have fast convergence speed and can robust against outliers. Two examples are simulated to verify the feasibility and efficiency of the proposed algorithm.     

Learning identity with radial basis function networks

Radial basis function (RBF) networks are compared with other neural network techniques on a face recognition task for applications involving identification of individuals using low-resolution video information. The RBF networks are shown to exhibit useful shift, scale and pose (y-axis head rotation) invariance after training when the input representation is made to mimic the receptive field functions found in early stages of the human vision system. In particular, representations based on difference of Gaussian (DoG) filtering and Gabor wavelet analysis are compared. Extensions of the techniques to the case of image sequence analysis are described and a time delay (TD) RBF network is used for recognising simple movement-based gestures. Finally, we discuss how these techniques can be used in real-life applications that require recognition of faces and gestures using low-resolution video images.     

Robust radial basis function neural networks

Function approximation has been found in many applications. The radial basis function (RBF) network is one approach which has shown a great promise in this sort of problems because of its faster learning capacity. A traditional RBF network takes Gaussian functions as its basis functions and adopts the least-squares criterion as the objective function, However, it still suffers from two major problems. First, it is difficult to use Gaussian functions to approximate constant values. If a function has nearly constant values in some intervals, the RBF network will be found inefficient in approximating these values. Second, when the training patterns incur a large error, the network will interpolate these training patterns incorrectly. In order to cope with these problems, an RBF network is proposed in this paper which is based on sequences of sigmoidal functions and a robust objective function. The former replaces the Gaussian functions as the basis function of the network so that constant-valued functions can be approximated accurately by an RBF network, while the latter is used to restrain the influence of large errors. Compared with traditional RBF networks, the proposed network demonstrates the following advantages: (1) better capability of approximation to underlying functions; (2) faster learning speed; (3) better size of network; (4) high robustness to outliers.     

Numerical solution of PDEs via integrated radial basis function networks with adaptive training algorithm

This paper develops a mesh-free numerical method for solving PDEs, based on integrated radial basis function networks (IRBFNs) with adaptive residual subsampling training scheme. The multiquadratic function is chosen as the transfer function of the neurons. The nonlinear algebraic equation systems for weights training are solved by Levenberg–Marquardt algorithm. The performance of the proposed method is demonstrated in numerical examples by approximating several functions and solving nonlinear PDEs. The result of numerical experiments shows that the IRBFNs with the adaptive procedure requires less neurons to attain the desired accuracy than conventional radial basis function networks.     

A sequential multi-category classifier using radial basis function networks

This paper presents a new sequential multi-category classifier using radial basis function (SMC-RBF) network for real-world classification problems. The classification algorithm processes the training data one by one and builds the RBF network starting with zero hidden neuron. The growth criterion uses the misclassification error, the approximation error to the true decision boundary and a distance measure between the current sample and the nearest neuron belonging to the same class. SMC-RBF uses the hinge loss function (instead of the mean square loss function) for a more accurate estimate of the posterior probability. For network parameter updates, a decoupled extended Kalman filter is used to reduce the computational overhead. Performance of the proposed algorithm is evaluated using three benchmark problems, viz., image segmentation, vehicle and glass from the UCI machine learning repository. In addition, performance comparison has also been done on two real-world problems in the areas of remote sensing and bio-informatics. The performance of the proposed SMC-RBF classifier is also compared with the other RBF sequential learning algorithms like MRAN, GAP-RBFN, OS-ELM and the well-known batch classification algorithm SVM. The results indicate that SMC-RBF produces a higher classification accuracy with a more compact network. Also, the study indicates that using a function approximation algorithm for classification problems may not work well when the classes are not well separated and the training data is not uniformly distributed among the classes.     

Adaptive radial basis function networks with kernel shape parameters

Radial basis function network (RBFN), commonly used in the classification applications, has two parameters, kernel center and radius that can be determined by unsupervised or supervised learning. But it has a disadvantage that it considers that all the independent variables have the equal weights. In that case, the contour lines of the kernel function are circular, but in fact, the influence of each independent variable on the model is so different that it is more reasonable if the contour lines are oval. To overcome this disadvantage, this paper presents an adaptive radial basis function network (ARBFN) with kernel shape parameters and derives the learning rules from supervised learning. To verify that this architecture is superior to that of the traditional RBFN, we make a comparison between three artificial and fifteen real examples in this study. The results show that ARBFN is much more accurate than the traditional RBFN, illustrating that the shape parameters can actually improve the accuracy of RBFN.     

Generalised Gaussian radial basis function neural networks

The mixed use of different shapes of radial basis functions (RBFs) in radial basis functions neural networks (RBFNNs) is investigated in this paper. For this purpose, we propose the use of a generalised version of the standard RBFNN, based on the generalised Gaussian distribution. The generalised radial basis function (GRBF) proposed in this paper is able to reproduce other different radial basis functions (RBFs) by changing a real parameter τ. In the proposed methodology, a hybrid evolutionary algorithm (HEA) is employed to estimate the number of hidden neuron, the centres, type and width of each RBF associated with each radial unit. In order to test the performance of the proposed methodology, an experimental study is presented with 20 datasets from the UCI repository. The GRBF neural network (GRBFNN) was compared to RBFNNs with Gaussian, Cauchy and inverse multiquadratic RBFs in the hidden layer and to other classifiers, including different RBFNN design methods, support vector machines (SVMs), a sparse probabilistic classifier (sparse multinominal logistic regression, SMLR) and other non-sparse (but regularised) probabilistic classifiers (regularised multinominal logistic regression, RMLR). The GRBFNN models were found to be better than the alternative RBFNNs for almost all datasets, producing the highest mean accuracy rank.     

Speaker identification using multi-step clustering algorithm with transformation-based GMM

To improve the performance of speaker recognition, the embedded linear transformation is used to integrate both transformation and diagonal-covariance Caussian mixture into a unified framework. In the case, the mixture number of GMM must be fixed in model training. The cluster expectation-maximization (EM) algorithm is a well-known technique in which the mixture number is regarded as an estimated parameter. This paper presents a new model structure that integrates a multi-step cluster algorithm into the estimating process of GMM with the embedded transformation. In the approach, the transformation matrix, the mixture number and model parameters are simultaneously estimated according to a maximum likelihood criterion. The proposed method is demonstrated on a database of three data sessions for text independent speaker identification. The experiments show that this method outperforms the traditional GMM with cluster EM algorithm. This text was submitted by the authors in English.     

基于径向基网络与交互式遗传算法的服装设计

针对将交互式遗传算法应用到服装设计中产生的人的疲劳问题,提出利用神经网络来逼近适应度函数.给出了以GA操作产生的每代最佳个体初步作为神经网络径向基网络函数的中心值并结合相似距离值,利用K-Means求出径向基网络的各参数以逼近适应度函数.在服装设计系统应用中取得了良好的效果.     

A clustering algorithm for radial basis function neural network initialization

In this paper, we propose an Output-Constricted Clustering (OCC) algorithm for Radial Basis Function Neural Network (RBFNN) initialization. OCC first roughly partitions the output based on the required precision and then refinedly clusters data based on the input complexity within each output partition. The main contribution of the proposed clustering algorithm is that we introduce the concept of separability, which is a criterion to judge the suitability of the number of sub-clusters in each output partition. As a result, OCC is able to determine the proper number of sub-clusters with appropriate locations within each output partition by considering both input and output information. The resulting clusters from OCC are used to initialize RBFNN, with proper number and initial locations of for hidden neurons. As a result, RBFNN starting it's learning from a good point, is able to achieve better approximation performance than existing clustering methods for RBFNN initialization. This better performance is illustrated by a number of examples.     

Accurate and robust image registration based on radial basis neural networks

Neural network-based image registration using global image features is relatively a new research subject, and the schemes devised so far use a feedforward neural network to find the geometrical transformation parameters. In this work, we propose to use a radial basis function neural network instead of feedforward neural network to overcome lengthy pre-registration training stage. This modification has been tested on the neural network-based registration approach using discrete cosine transformation features in the presence of noise. The experimental registration work is conducted in two different levels: estimation of transformation parameters from a local range for fine registration and from a medium range for coarse registration. For both levels, the performances of the feedforward neural network-based and radial basis function neural network-based schemes have been obtained and compared to each other. The proposed scheme does not only speed up the training stage enormously but also increases the accuracy and gives robust results in the presence of additive Gaussian noise owing to the better generalization ability of the radial basis function neural networks.     

GPS orbit approximation using radial basis function networks

We present solutions for GPS orbit computation from broadcast and precise ephemerides using a group of artificial neural networks (ANNs), i.e. radial basis function networks (RBFNs). The problem of broadcast orbit correction, resulting from precise ephemerides, has already been solved using traditional polynomial and trigonometric interpolation. As an alternative approach RBFN broadcast orbit correction produces results within the accuracy range of the traditional methods. Our study shows RBFN broadcast orbit correction performs well also near the end of data intervals and for short data spans (～20 min). Regarding limitations of polynomial and trigonometric extrapolation, the most significant advantage of using RBFNs over the traditional methods for GPS broadcast orbit approximation arises from its short time prediction capability.     

Probabilistic self-organizing map and radial basis function networks

We propose in this paper a new learning algorithm probabilistic self-organizing map (PRSOM) using a probabilistic formalism for topological maps. This algorithm approximates the density distribution of the input set with a mixture of normal distributions. The unsupervised learning is based on the dynamic clusters principle and optimizes the likelihood function. A supervised version of this algorithm based on radial basis functions (RBF) is proposed. In order to validate the theoretical approach, we achieve regression tasks on simulated and real data using the PRSOM algorithm. Moreover, our results are compared with normalized Gaussian basis functions (NGBF) algorithm.     

Kernel orthonormalization in radial basis function neural networks

This paper deals with optimization of the computations involved in training radial basis function (RBF) neural networks. The main contribution of the reported work is the method for network weights calculation, in which the key idea is to transform the RBF kernels into an orthonormal set of functions (using the standard Gram-Schmidt orthogonalization). This significantly reduces the computing time if the RBF training scheme, which relies on adding one kernel hidden node at a time to improve network performance, is adopted. Another property of the method is that, after the RBF network weights are computed, the original network structure can be restored back. An additional strength of the method is the possibility to decompose the proposed computing task into a number of parallel subtasks so gaining further savings on computing time. Also, the proposed weight calculation technique has low storage requirements. These features make the method very attractive for hardware implementation. The paper presents a detailed derivation of the proposed network weights calculation procedure and demonstrates its validity for RBF network training on a number of data classification and function approximation problems.     

Training radial basis function networks using biogeography-based optimizer

Training artificial neural networks is considered as one of the most challenging machine learning problems. This is mainly due to the presence of a large number of solutions and changes in the search space for different datasets. Conventional training techniques mostly suffer from local optima stagnation and degraded convergence, which make them impractical for datasets with many features. The literature shows that stochastic population-based optimization techniques suit this problem better and are reliably alternative because of high local optima avoidance and flexibility. For the first time, this work proposes a new learning mechanism for radial basis function networks based on biogeography-based optimizer as one of the most well-regarded optimizers in the literature. To prove the efficacy of the proposed methodology, it is employed to solve 12 well-known datasets and compared to 11 current training algorithms including gradient-based and stochastic approaches. The paper considers changing the number of neurons and investigating the performance of algorithms on radial basis function networks with different number of parameters as well. A statistical test is also conducted to judge about the significance of the results. The results show that the biogeography-based optimizer trainer is able to substantially outperform the current training algorithms on all datasets in terms of classification accuracy, speed of convergence, and entrapment in local optima. In addition, the comparison of trainers on radial basis function networks with different neurons size reveal that the biogeography-based optimizer trainer is able to train radial basis function networks with different number of structural parameters effectively.

     

Combined genetic algorithm optimization and regularized orthogonalleast squares learning for radial basis function networks

Presents a two-level learning method for radial basis function (RBF) networks. A regularized orthogonal least squares (ROLS) algorithm is employed at the lower level to construct RBF networks while the two key learning parameters, the regularization parameter and the RBF width, are optimized using a genetic algorithm (GA) at the upper level. Nonlinear time series modeling and prediction is used as an example to demonstrate the effectiveness of this hierarchical learning approach.     

Identification of nonlinear systems with outliers using wavelet neural networks based on annealing dynamical learning algorithm

This paper presents an annealing dynamical learning algorithm (ADLA) to train wavelet neural networks (WNNs) for identifying nonlinear systems with outliers. In ADLA–WNNs, wavelet-based support vector regression (WSVR) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs due to the similarity between WSVR and WNNs. After initialization, ADLA with nonlinear time-varying learning rates is applied to train the WNNs. In the ADLA, the determination of the learning rates would be a key work for the trade-off between stability and speed of convergence. A computationally efficient optimization method, particle swarm optimization (PSO), is adopted to find the optimal learning rates to overcome the stagnation in the training procedure of WNNs. Due to the advantages of WSVR and ADLA (WSVR–ADLA), the WSVR-based ADLA–WNNs (WSVR–ADLA–WNNs) can robust against outliers and achieve the promising efficiency of system identifications. Three examples are simulated to confirm the performance of the proposed algorithm. From the simulated results, the feasibility and superiority of the proposed WSVR–ADLA–WNNs for identifying nonlinear systems with artificial outliers are verified.     

Speech recognition using pulse-coupled neural networks with a radial basis function

We have developed a novel pulse-coupled neural network (PCNN) for speech recognition. One of the advantages of the PCNN is in its biologically based neural dynamic structure using feedback connections. To recall the memorized pattern, a radial basis function (RBF) is incorporated into the proposed PCNN. Simulation results show that the PCNN with a RBF can be useful for phoneme recognition. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002