Environment-adaptation mobile radio propagation prediction usingradial basis function neural networks

This paper investigates the application of a radial basis function (RBF) neural network to the prediction of field strength based on topographical and morphographical data. The RBF neural network is a two-layer localized receptive field network whose output nodes from a combination of radial activation functions computed by the hidden layer nodes. Appropriate centers and connection weights in the RBF network lead to a network that is capable of forming the best approximation to any continuous nonlinear mapping up to an arbitrary resolution. Such an approximation introduces best nonlinear approximation capability into the prediction model in order to accurately predict propagation loss over an arbitrary environment based on adaptive learning from measurement data. The adaptive learning employs hybrid competitive and recursive least squares algorithms. The unsupervised competitive algorithm adjusts the centers while the recursive least squares (RLS) algorithm estimates the connection weights. Because these two learning rules are both linear, rapid convergence is guaranteed. This hybrid algorithm significantly enhances the real-time or adaptive capability of the RBF-based prediction model. The applications to Okumura's (1968) data are included to demonstrate the effectiveness of the RBF neural network approach     

多层前向网络的逼近机理与拓扑结构学习方法

对多层前向网络的最小二乘逼近机理进行了系统的分析，指出隐层节点函数特性的特定选择是构成网络有效逼近能力最关键的因素。分析了增加隐层数和增加隐节点数在改进网络逼近效果方面不同的作用机理，给出了前向网络拓扑结构学习的通用算法和其对应的神经生物学机制。     

Neuromorphic control: adaptation and learning

A structure for a neural network-based robotic motion controller is presented. Simulations of both position and force servos are carried out, and the approach is shown to be useful for a nonlinear system in an uncertain environment. The neural network comprises a four-layer network, including input/output layers and two hidden layers. Time delay elements are included in the first hidden layer, so that the neural network can learn dynamics of the system. The authors also implement a new learning method based on fuzzy logic, which is useful to accelerate learning and improve convergence     

利用EM算法估计隐含观测量的回馈神经网络学习新方法

研究回馈神经网络(RNN)参数估计的新方法.利用隐含观测量,将复杂RNN的训练分解为线性输出层和多个单隐元的参数估计.基于每个隐元激励函数的多点线性近似,RNN可利用统计混合专家网络模型(ME)描述,从而将RNN的参数估计转化为包含隐含观测量的线性系统的最大似然估计问题,最后利用期望最大化(EM)算法获得RNN的隐含观测量及其参数估计.     

Linear and nonlinear ARMA model parameter estimation using anartificial neural network

Addresses parametric system identification of linear and nonlinear dynamic systems by analysis of the input and output signals. Specifically, the authors investigate the relationship between estimation of the system using a feedforward neural network model and estimation of the system by use of linear and nonlinear autoregressive moving-average (ARMA) models. By utilizing a neural network model incorporating a polynomial activation function, the authors show the equivalence of the artificial neural network to the linear and nonlinear ARMA models. They compare the parameterization of the estimated system using the neural network and ARMA approaches by utilizing data generated by means of computer simulations. Specifically, the authors show that the parameters of a simulated ARMA system can be obtained from the neural network analysis of the simulated data or by conventional least squares ARMA analysis. The feasibility of applying neural networks with polynomial activation functions to the analysis of experimental data is explored by application to measurements of heart rate (HR) and instantaneous lung volume (ILV) fluctuations     

基于核函数的非线性口袋算法

应用满足Mercer条件的核函数设计非线性算法已经成为机器学习领域一项新的非线性技术.核感知器算法利用核思想非线性地推广了线性感知器算法,使其可以处理原始输入空间中的非线性分类问题和高维特征空间中的线性问题.线性口袋算法改进了线性感知器算法,能够直接处理线性不可分问题.为了进一步改进线性口袋算法和核感知器算法,本文提出基于核函数的非线性口袋算法,即核口袋算法,其目标是找到一个使错分样本数最小的非线性判别函数,并证明了其收敛性.核口袋算法的特点是用简单的迭代过程和核函数来实现非线性分类器的设计.基准数据集的实验结果证明核口袋算法的性能优于线性口袋算法和核感知器算法.     

多层前馈网络在模式识别中的理论和应用

本文从理论上证明了具有线性输出单元的多层前馈网络能用作最优特征提取器。同时还证明了多层前馈网络分类器的输出函数是最小均方误差意义下对Ｂａｙｅｓ决策函数的逼近，对于具有线性输出单元的三层前馈网络，当隐层单元数足够多时，这种逼近能达到任意精度。在此基础上，我们提出了一个综合了特征提取网络和分类器网络的组合神经网络模型，其性能好于单个的三层前馈网络。     

Flood forecasting using radial basis function neural networks

A radial basis function (RBF) neural network (NN) is proposed to develop a rainfall-runoff model for three-hour-ahead flood forecasting. For faster training speed, the RBF NN employs a hybrid two-stage learning scheme. During the first stage, unsupervised learning, fuzzy min-max clustering is introduced to determine the characteristics of the nonlinear RBFs. In the second stage, supervised learning, multivariate linear regression is used to determine the weights between the hidden and output layers. The rainfall-runoff relation can be considered as a linear combination of some nonlinear RBFs. Rainfall and runoff events of the Lanyoung River collected during typhoons are used to train, validate,and test the network. The results show that the RBF NN can be considered a suitable technique for predicting flood flow     

Subsethood based adaptive linguistic networks for pattern classification

This paper presents a fuzzy-neural network that admits both numeric as well as linguistic inputs. Numeric inputs are fuzzified by input nodes upon presentation to the network. Fuzzy rule-based knowledge is translated directly into a network architecture. Connections in the network are represented by fuzzy sets: Input to hidden connections represent rule antecedents; hidden to output connections represent rule consequents. The novelty of the model lies in the method of activation spread in the network which is based on a fuzzy mutual subsethood measure. Rule (hidden) node activations are computed as a fuzzy inner product. For a given numeric or fuzzy input, numeric outputs are computed using volume based defuzzification. A supervised learning procedure based on gradient descent is employed to train the network. The model has a natural capability for inference, function approximation, and classification and is versatile in that it can handle numeric and fuzzy inputs simultaneously. In this paper, we focus on the classification ability of the model and demonstrate its performance on three benchmark classification problems: the Iris data set, Ripley's synthetic two class problem, and Pal and Mitra's Telegu vowel data. Results show that the classifier performs at par or better than various other techniques.     

Analysis and pruning of nonlinear auto-association networks

In the paper, an analysis of a three-layer nonlinear auto-association network with linear output neurons and sigmoidal hidden neurons is carried out. Simulations have shown that the hidden layer neurons of this network operate mainly in their linear region. By studying the statistical relations governing the operation of such a network, the nearly linear behaviour of the sigmoidal hidden neurons was verified. Dealing with the network as being totally linear, a pruning algorithm is proposed to find out the minimum number of hidden neurons needed to reconstruct the input data within a certain error threshold. The performance of the pruning algorithm is illustrated with two examples.     

Real-time data-reusing adaptive learning of a radial basis function network for tracking evoked potentials

Tracking variations in both the latency and amplitude of evoked potential (EP) is important in quantifying properties of the nervous system. Adaptive filtering is a powerful tool for tracking such variations. In this paper, a data-reusing non-linear adaptive filtering method, based on a radial basis function network (RBFN), is implemented to estimate EP. The RBFN consists of an input layer of source nodes, a single hidden layer of non-linear processing units and an output layer of linear weights. It has built-in nonlinear activation functions that allow learning of function mappings. Moreover, it produces satisfactory estimates of signals against a background noise without a priori knowledge of the signal, provided that the signal and noise are independent. In clinical situations where EP responses change rapidly, the convergence rate of the algorithm becomes a critical factor. A carefully designed data-reusing RBFN can accelerate the convergence rate markedly and, thus, enhance its performance. Both theoretical analysis and simulation results support the improved performance of our new algorithm.     

利用隐含观测量估计将复杂神经网络分解为多个单隐元学习的新方法

本文从统计学的角度研究多层多隐元前神经网络（ＮＮ）的参数估计学习问题,利用ＮＮ激励函数的析线线性近似,提出一种求解多隐层多隐元ＮＮ每个隐元指导信号（隐含观测量）的新方法,利用每个隐元的指导信号估计可以半多隐多层多隐元ＮＮ的参数估计学习转化为多个相互独立的单隐元ＮＮ参数估计学习训练问题,从而将复杂系统参数估计问题转化为简单系统的参数估计问题而得以解决。     

The influence of parameter initialization on the training time and accuracy of a nonlinear regression model

In order to construct a nonlinear regression model we have to accurately (in some sense) initialize parameters of the model. In this work we performed comparison of several widely used methods and several newly developed approached for initialization of parameters of a regression model, represented as a decomposition in a linear dictionary of some parametric functions (sigmoids). We proposed a general deterministic approach for initialization, providing repeatability of results, reduction of a learning time and in some cases increase of a regression model accuracy; we developed two new algorithms (based on a piecewise-linear approximation and based on local properties of approximable dependency) in the framework of the proposed approach; we developed randomized initialization algorithm (spherical initialization) for effective approximation of high-dimensional dependencies; we improved the classical initialization method SCAWI (by locating centers of sigmoids in sample points), providing a regression model accuracy improvement on specific classes of dependencies (smooth functions and discontinuous functions with a number of local peculiarities in an input domain) when using RProp algorithm for learning; we performed comparison of classical and newly proposed initialization methods and highlighted the most efficient ones.     

采用混合核支持向量机的DOA估计

人工神经网络( ANN)进行建模时通常需要准备大量的数据样本,同时网络结构一般都比较复杂;而采用支持向量机( SVM)进行建模时,不同核函数有不同的效果,各有利弊,且选取SVM模型参数的理论支撑尚不完整。为了解决这些问题,提出了一种基于混合核函数的支持向量机来改善来波到达角( DOA)的估计性能,并结合二进制粒子群算法( PSO)来对混合核函数进行参数寻优。该混合核函数由全局核函数和局部核函数构成,提高了SVM的泛化能力和学习能力。首先通过拟合多项式函数,验证了该混合核SVM的有效性。将该方法用于DOA估计建模,在不同信噪比和快拍数下,通过与径向基函数( RBF)神经网络、基于各单一核函数的SVM和MUSIC算法预测结果对比,混合核SVM均方差有所降低,提高了DOA估计的精度且有更好的稳定性。     

对角型广义RBF神经网络与非线性时间序列预测

径向基函数（RBF）神经网络在非线性时间序列预测方面发挥着重要作用。本文提出了对角型广义RBF神经网络模型,并利用贝叶斯阴阳（BYY)谐和学习算法进行隐层单元个数的选择和参数初始值的设置,且建立了同步LMS算法进行参数学习。进一步,将对角型广义RBF神经网络应用于非线性时间序列预测,得到了预测准确率高和速度快的效果。      

Nonparametric identification of Hammerstein systems

A discrete-time nonlinear Hammerstein system is identified, and the correlation and frequency-domain methods for identification of its linear subsystem are presented. The main results concern the estimation of the nonlinear memoryless subsystem. No conditions concerning the functional form of the transform characteristic of the subsystem are made, and an algorithm for estimation of the characteristic is given. The algorithm is simply a nonparametric kernel estimate of the regression function calculated from dependent data. It is shown that the algorithm converges to the characteristic of the subsystem regardless of the probability distribution of the input variable. Pointwise as well as global consistencies are established. For Lipschitz characteristics the rate of the convergence in probability is O(n^-1/3 )     

Parallel,self-organizing,hierarchical neural networks with forward-backward training

A forward-backward training algorithm for parallel, self-organizing hierarchical neural networks (PSHNNs) is described. Using linear algebra, it is shown that the forward-backward training of ann-stage PSHNN until convergence is equivalent to the pseudo-inverse solution for a single, total network designed in the least-squares sense with the total input vector consisting of the actual input vector and its additional nonlinear transformations. These results are also valid when a single long input vector is partitioned into smaller length vectors. A number of advantages achieved are: small modules for easy and fast learning, parallel implementation of small modules during testing, faster convergence rate, better numerical error-reduction, and suitability for learning input nonlinear transformations by other neural networks. The backpropagation (BP) algorithm is proposed for learning input nonlinearitics. Better performance in terms of deeper minimum of the error function and faster convergence rate is achieved when a single BP network is replaced by a PSHNN of equal complexity in which each stage is a BP network of smaller complexity than the single BP network.     

MOS circuit for nonlinear Hebbian learning

An analogue MOS circuit that implements a nonlinear Hebbian learning rule is presented. The circuit has two differential inputs (voltages) and yields an output (current) which approximates the product of a cubic and hyperbolic tangent functions of the two input voltages. This has been successfully incorporated in an analogue integrated circuit implementation of the Herault-Jutten neuromorphic network.<>     

基于VW-IGRBF神经网络的出水BOD软测量

针对污水处理过程具有复杂非线性特性以及出水BOD难以精确测量的问题,文中提出一种基于变宽度的逆平方根和高斯函数线性组合的RBF神经网络软测量方法。神经网络的激活函数由逆平方根函数和高斯函数线性组合,弥补了单一激活函数在某些区间饱和的问题,提高了隐层激活函数的表达能力和自适应能力。由于激活函数的宽度对模型的泛化性能有较大的影响,因此引入基于核密度的变宽度策略可以有效提高网络泛化能力。文中采用改进LM算法实现了神经网络参数的在线学习。基于污水处理过程实际运行数据的仿真实验表明,所提方法对于出水BOD具有较高的预测精度和良好的自适应能力。     

On Nonparametric Identification of Wiener Systems

In this paper, a new method for the identification of the Wiener nonlinear system is proposed. The system, being a cascade connection of a linear dynamic subsystem and a nonlinear memoryless element, is identified by a two-step semiparametric approach. The impulse response function of the linear part is identified via the nonlinear least-squares approach with the system nonlinearity estimated by a pilot nonparametric kernel regression estimate. The obtained estimate of the linear part is then used to form a nonparametric kernel estimate of the nonlinear element of the Wiener system. The proposed method permits recovery of a wide class of nonlinearities which need not be invertible. As a result, the proposed algorithm is computationally very efficient since it does not require a numerical procedure to calculate the inverse of the estimate. Furthermore, our approach allows non-Gaussian input signals and the presence of additive measurement noise. However, only linear systems with a finite memory are admissible. The conditions for the convergence of the proposed estimates are given. Computer simulations are included to verify the basic theory