Universal approximation using incremental constructive feedforward networks with random hidden nodes

According to conventional neural network theories, single-hidden-layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable. However, as observed in most neural network implementations, tuning all the parameters of the networks may cause learning complicated and inefficient, and it may be difficult to train networks with nondifferential activation functions such as threshold networks. Unlike conventional neural network theories, this paper proves in an incremental constructive method that in order to let SLFNs work as universal approximators, one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer. In such SLFNs implementations, the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions g:R/spl rarr/R and the activation functions for RBF nodes can be any integrable piecewise continuous functions g:R/spl rarr/R and /spl int//sub R/g(x)dx/spl ne/0. The proposed incremental method is efficient not only for SFLNs with continuous (including nondifferentiable) activation functions but also for SLFNs with piecewise continuous (such as threshold) activation functions. Compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters.     

Approximation of algebraic and trigonometric polynomials by feedforward neural networks

This paper presents a function approximation to a general class of polynomials by using one-hidden-layer feedforward neural networks(FNNs). Both the approximations of algebraic polynomial and trigonometric polynomial functions are discussed in details. For algebraic polynomial functions, an one-hidden-layer FNN with chosen number of hidden-layer nodes and corresponding weights is established by a constructive method to approximate the polynomials to a remarkable high degree of accuracy. For trigonometric functions, an upper bound of approximation is therefore derived by the constructive FNNs. In addition, algorithmic examples are also included to confirm the accuracy performance of the constructive FNNs method. The results show that it improves efficiently the approximations of both algebraic polynomials and trigonometric polynomials. Consequently, the work is really of both theoretical and practical significance in constructing a one-hidden-layer FNNs for approximating the class of polynomials. The work also paves potentially the way for extending the neural networks to approximate a general class of complicated functions both in theory and practice.     

The essential order of approximation for neural networks

There have been various studies on approximation ability of feedforward neural networks (FNNs). Most of the existing studies are, however, only concerned with density or upper bound estimation on how a multivariate function can be approximated by an FNN, and consequently, the essential approximation ability of an FNN cannot be revealed. In this paper, by establishing both upper and lower bound estimations on approximation order, the essential approximation ability (namely, the essential approximation order) of a class of FNNs is clarified in terms of the modulus of smoothness of functions to be approximated. The involved FNNs can not only approximate any continuous or integrable functions defined on a compact set arbitrarily well, but also provide an explicit lower bound on the number of hidden units required. By making use of multivariate approximation tools, it is shown that when the functions to be approximated are Lipschitzian with order up to 2, the approximation speed of the FNNs is uniquely deter     

Implementation of Spectral Transformations in the Class of Fast Neural Networks

In the paper, the use of neural networks for the implementation of fast algorithms of spectral transformations is discussed. It is shown that the fast algorithms are particular cases of fast neural networks (FNNs). Methods for parametric tuning FNNs to a given system of basis functions are suggested. Neural network implementations of the fast Walsh and wavelet transformations and the fast Fourier, Vilenkin–Christiansen, and Haar transforms are constructed. The discussions are illustrated by examples.     

Heuristics for the selection of weights in sequential feed-forward neural networks: An experimental study

The selection of weights of the new hidden units for sequential feed-forward neural networks (FNNs) usually involves a non-linear optimization problem that cannot be solved analytically in the general case. A suboptimal solution is searched heuristically. Most models found in the literature choose the weights in the first layer that correspond to each hidden unit so that its associated output vector matches the previous residue as best as possible. The weights in the second layer can be either optimized (in a least-squares sense) or not. Several exceptions to the idea of matching the residue perform an (implicit or explicit) orthogonalization of the output vectors of the hidden units. In this case, the weights in the second layer are always optimized. An experimental study of the aforementioned approaches to select the weights for sequential FNNs is presented. Our results indicate that the orthogonalization of the output vectors of the hidden units outperforms the strategy of matching the residue, both for approximation and generalization purposes.     

Parameter Optimization of Interval Type-2 Fuzzy Neural Networks Based on PSO and BBBC Methods

Interval type-2 fuzzy neural networks (IT2FNNs) can be seen as the hybridization of interval type-2 fuzzy systems (IT2FSs) and neural networks (NNs). Thus, they naturally inherit the merits of both IT2FSs and NNs. Although IT2FNNs have more advantages in processing uncertain, incomplete, or imprecise information compared to their type-1 counterparts, a large number of parameters need to be tuned in the IT2FNNs, which increases the difficulties of their design. In this paper, big bang-big crunch (BBBC) optimization and particle swarm optimization (PSO) are applied in the parameter optimization for Takagi-Sugeno-Kang (TSK) type IT2FNNs. The employment of the BBBC and PSO strategies can eliminate the need of backpropagation computation. The computing problem is converted to a simple feed-forward IT2FNNs learning. The adoption of the BBBC or the PSO will not only simplify the design of the IT2FNNs, but will also increase identification accuracy when compared with present methods. The proposed optimization based strategies are tested with three types of interval type-2 fuzzy membership functions (IT2FMFs) and deployed on three typical identification models. Simulation results certify the effectiveness of the proposed parameter optimization methods for the IT2FNNs.      

The errors of simultaneous approximation of multivariate functions by neural networks

There have been many studies on the simultaneous approximation capability of feed-forward neural networks (FNNs). Most of these, however, are only concerned with the density or feasibility of performing simultaneous approximations. This paper considers the simultaneous approximation of algebraic polynomials, employing Taylor expansion and an algebraic constructive approach, to construct a class of FNNs which realize the simultaneous approximation of any smooth multivariate function and all of its derivatives. We also present an upper bound on the approximation accuracy of the FNNs, expressed in terms of the modulus of continuity of the functions to be approximated.     

Adaptive backstepping controller design for nonlinear uncertain systems using fuzzy neural systems

In this article, we propose an adaptive backstepping control scheme using fuzzy neural networks (FNNs), ABC_FNN, for a class of nonlinear non-affine systems in non-triangular form. The nonlinear non-affine system contains the uncertainty, external disturbance or parameters variations. Two kinds of FNN systems are used to estimate the unknown system functions. According to the FNN estimations, the adaptive backstepping control (ABC_FNN) signal can be generated by backstepping design procedure such that the system output follows the desired trajectory. To ensure robustness and performance, a proportional-integral-surface function and robust controller are designed to improve the control performance. Based on the Lyapunov stability theory, the stability of a closed-loop system is guaranteed and the adaptive laws of the FNN parameters are obtained. This approach is also valid for nonlinear affine system with uncertainty or disturbance. The uncertainty and disturbance terms are estimated by FNNs and treated by the ABC_FNN scheme. Finally, the effectiveness of the proposed ABC_FNN is demonstrated through the simulation of controlling a nonlinear non-affine system and the continuously stirred tank reactor plant to demonstrate the performances of our approach.     

Convergence analysis of BP neural networks via sparse response regularization

Backpropagation (BP) algorithm is the typical strategy to train the feedforward neural networks (FNNs). Gradient descent approach is the popular numerical optimal method which is employed to implement the BP algorithm. However, this technique frequently leads to poor generalization and slow convergence. Inspired by the sparse response character of human neuron system, several sparse-response BP algorithms were developed which effectively improve the generalization performance. The essential idea is to impose the responses of hidden layer as a specific L₁ penalty term on the standard error function of FNNs. In this paper, we mainly focus on the two remaining challenging tasks: one is to solve the non-differential problem of the L₁ penalty term by introducing smooth approximation functions. The other aspect is to provide a rigorous convergence analysis for this novel sparse response BP algorithm. In addition, an illustrative numerical simulation has been done to support the theoretical statement.     

一种结构自适应的径向基函数神经网络

提出了一种新的结构自适应的径向基函数(RBF)神经网络模型。在该模型中,自组织映射(SOM)神经网络作为聚类网络,采用无监督学习算法对输入样本进行自组织分类,并将分类中心及其对应的权值向量传递给RBF神经网络,分别作为径向基函数的中心和相应的权值向量;RBF神经网络作为基础网络,采用高斯函数实现输入层到隐层的非线性映射,输出层则采用有监督学习算法训练网络的权值,从而实现输入层到输出层的非线性映射。通过对字母数据集进行仿真,表明该网络具有较好的性能。     

A Method for a Simulation of Continuous Intracranial Pressure Curves

The study introduces a method to simulate continuously an intracranial pressure (ICP) wave form. In a system analysis approach the intracranial compartment was viewed as a black box with arterial blood pressure (ABP) as an input signal and ICP as an output. A weight function was used to transform the ABP curve into the ICP curve. The output ICP waveform was generated using a weight function derived from the transcranial Doppler blood flow velocity (FV) and ABP curves. In order to establish the relationship between TCD characteristics and weight functions simultaneous recordings of FV, ABP, and ICP curves of a defined group of patients were used. A linear function between the TCD characteristics and the weight functions was obtained by calculating a series of multiple regression analyses. Given examples demonstrate the procedure's capabilities in predicting the mean ICP, the pulse and respiratory waveform modulations, and the trends of ICP changes.     

Virtual field strategy for collaborative signal and information processing in wireless heterogeneous sensor networks

A novel collaborative signal and information processing (CSIP) method, which is based on virtual fields excited by sensor nodes, is proposed for wireless heterogeneous sensor networks. These virtual fields influence states and operations in sensor nodes located in their regions of influence (ROIs) and thus collaboration is implemented through interactions between surrounding virtual fields and sensor nodes. Described by a group of radial basis functions (RBFs), virtual fields have different magnitudes and ROIs due to different initial energy, communication ranges, sensing ranges and information processing capabilities in heterogeneous sensor nodes. Dynamic mobile agent itinerary decision and adaptive node active probability updating are studied with virtual field strategies in a heterogeneous sensor network using mobile-agent-based computing paradigm. Simulation results demonstrate that this approach can reduce energy consumption in sensor nodes. Information gain efficiency and network lifetime are also increased.     

Support-vector-based fuzzy neural network for pattern classification

Fuzzy neural networks (FNNs) for pattern classification usually use the backpropagation or C-cluster type learning algorithms to learn the parameters of the fuzzy rules and membership functions from the training data. However, such kinds of learning algorithms usually cannot minimize the empirical risk (training error) and expected risk (testing error) simultaneously, and thus cannot reach a good classification performance in the testing phase. To tackle this drawback, a support-vector-based fuzzy neural network (SVFNN) is proposed for pattern classification in this paper. The SVFNN combines the superior classification power of support vector machine (SVM) in high dimensional data spaces and the efficient human-like reasoning of FNN in handling uncertainty information. A learning algorithm consisting of three learning phases is developed to construct the SVFNN and train its parameters. In the first phase, the fuzzy rules and membership functions are automatically determined by the clustering principle. In the second phase, the parameters of FNN are calculated by the SVM with the proposed adaptive fuzzy kernel function. In the third phase, the relevant fuzzy rules are selected by the proposed reducing fuzzy rule method. To investigate the effectiveness of the proposed SVFNN classification, it is applied to the Iris, Vehicle, Dna, Satimage, Ijcnn1 datasets from the UCI Repository, Statlog collection and IJCNN challenge 2001, respectively. Experimental results show that the proposed SVFNN for pattern classification can achieve good classification performance with drastically reduced number of fuzzy kernel functions.     

Two-stage structural damage detection using fuzzy neural networks and data fusion techniques

It is proposed in this paper a novel two-stage structural damage detection approach using fuzzy neural networks (FNNs) and data fusion techniques. The method is used for structural health monitoring and damage detection, particularly for cases where the measurement data is enormous and with uncertainties. In the first stage of structural damage detection, structural modal parameters derived from structural vibration responses are fed into an FNN as the input. The output values from the FNN are defuzzified to produce a rough structural damage assessment. Later, in the second stage, the values output from three different FNN models are input directly to the data fusion center where fusion computation is performed. The final fusion decision is made by filtering the result with a threshold function, hence a refined structural damage assessment of superior reliability. The proposed approach has been applied to a 7-degree of freedom building model for structural damage detection, and proves to be feasible, efficient and satisfactory. Furthermore, the simulation result also shows that the identification accuracy can be boosted with the proposed approach instead of FNN models alone.     

一种结构自适应径向基函数神经网络

提出了一种新的结构自适应的径向基函数(RBF)神经网络模型。在该网络中,自组织映射(SOM)神经网络作为聚类网络,采用无监督学习算法对输入样本进行自组织分类,并将分类中心及其对应的权值向量传递给RBF神经网络,作为径向基函数的中心和相应的权值向量;RBF神经网络作为基础网络,采用高斯函数实现输入层到隐层的非线性映射,输出层则采用有监督学习算法训练网络的权值,从而实现输入层到输出层的非线性映射。通过对字母数据集进行仿真,表明该网络具有较好的性能。     

A testbed for coverage control using mixed wireless sensor networks

Wireless sensor networks (WSNs) is a relatively new technology that has been proposed for several applications including wide area monitoring. Such applications may include stationary or mobile sensor platforms or they may include several stationary and some mobile-robotic sensor nodes that can move in the area in order to achieve certain objectives, e.g., monitor areas that are not adequately covered or assist in the transfer of data to prevent the energy depletion of certain critical nodes. Such networks that consist of both stationary and mobile nodes are referred to as mixed WSNs. This paper presents the development of an experimental testbed for mixed WSNs consisting of stationary and mobile sensor nodes that collaborate to improve the sensing coverage and event detection of the network in a given deployment area. The paper describes the hardware and infrastructure of the testbed as well as a case study for coverage control that was investigated using the testbed. We point out that the developed testbed can be used for the evaluation and validation of different algorithms for coverage control that involve collaboration between stationary and mobile sensors to improve the WSN's monitoring capabilities. In addition, it can also be used to investigate other objectives as well as other concepts (e.g., network control).     

On the theory of flexible neural networks – Part I: a survey paper

Although flexible neural networks (FNNs) have been used more successfully than classical neural networks (CNNs) in many industrial applications, nothing is rigorously known about their properties. In fact they are not even well known to the systems and control community. In the first part of this paper, existing structures of and results on FNNs are surveyed. In the second part FNNs are examined in a theoretical framework. As a result, theoretical evidence is given for the superiority of FNNs over CNNs and further properties of the former are developed. More precisely, several fundamental properties of feedforward and recurrent FNNs are established. This includes the universal approximation capability, minimality, controllability, observability, and identifiability. In the broad sense, the results of this paper help that general use of FNNs in systems and control theory and applications be based on firm theoretical foundations. Theoretical analysis and synthesis of FNN-based systems thus become possible. The paper is concluded by a collection of topics for future work.     

On unique representations of certain dynamical systems produced by continuous-time recurrent neural networks

This article extends previous mathematical studies on elucidating the redundancy for describing functions by feedforward neural networks (FNNs) to the elucidation of redundancy for describing dynamical systems (DSs) by continuous-time recurrent neural networks (RNNs). In order to approximate a DS on R(n) using an RNN with n visible units, an n-dimensional affine neural dynamical system (A-NDS) can be used as the DS actually produced by the above RNN under an affine map from its visible state-space R(n) to its hidden state-space. Therefore, we consider the problem of clarifying the redundancy for describing A-NDSs by RNNs and affine maps. We clarify to what extent a pair of an RNN and an affine map is uniquely determined by its corresponding A-NDS and also give a nonredundant sufficient search set for the DS approximation problem based on A-NDS.     

Global and local modelling in RBF networks

Radial basis function networks are traditionally known as local approximation networks as they are composed by a number of elements which, individually, mainly take care of the approximation about a specific area of the input space. Then, the joint global output of the network is obtained as a linear combination of the individual elements' output. However, in the network optimization, the performance of the global model is normally the only objective to optimize. This might cause a deficient local modelling of the input space, thus partially losing the local character of this type of models. This work presents a modified radial basis function network that maintains the approximation capabilities of the local sub-models whereas the model is globally optimized. This property is obtained thanks to a special partitioning of the input space, that leads to a direct global-local optimization. A learning methodology adapted to the proposed model is used in the simulations, consisting of a clustering algorithm for the initialization of the centers and a local search technique. In the experiments, the proposed model shows satisfactory local and global modelling capabilities both in artificial and real applications.     

Design of information granule-oriented RBF neural networks and its application to power supply for high-field magnet

To realize effective modeling and secure accurate prediction abilities of models for power supply for high-field magnet (PSHFM), we develop a comprehensive design methodology of information granule-oriented radial basis function (RBF) neural networks. The proposed network comes with a collection of radial basis functions, which are structurally as well as parametrically optimized with the aid of information granulation and genetic algorithm. The structure of the information granule-oriented RBF neural networks invokes two types of clustering methods such as K-Means and fuzzy C-Means (FCM). The taxonomy of the resulting information granules relates to the format of the activation functions of the receptive fields used in RBF neural networks. The optimization of the network deals with a number of essential parameters as well as the underlying learning mechanisms (e.g., the width of the Gaussian function, the numbers of nodes in the hidden layer, and a fuzzification coefficient used in the FCM method). During the identification process, we are guided by a weighted objective function (performance index) in which a weight factor is introduced to achieve a sound balance between approximation and generalization capabilities of the resulting model. The proposed model is applied to modeling power supply for high-field magnet where the model is developed in the presence of a limited dataset (where the small size of the data is implied by high costs of acquiring data) as well as strong nonlinear characteristics of the underlying phenomenon. The obtained experimental results show that the proposed network exhibits high accuracy and generalization capabilities.