Training algorithms for Radial Basis Function Networks to tackle learning processes with imbalanced data-sets

Nowadays, many real applications comprise data-sets where the distribution of the classes is significantly different. These data-sets are commonly known as imbalanced data-sets. Traditional classifiers are not able to deal with these kinds of data-sets because they tend to classify only majority classes, obtaining poor results for minority classes. The approaches that have been proposed to address this problem can be categorized into three types: resampling methods, algorithmic adaptations and cost sensitive techniques.Radial Basis Function Networks (RBFNs), artificial neural networks composed of local models or RBFs, have demonstrated their efficiency in different machine learning areas. Centers, widths and output weights for the RBFs must be determined when designing RBFNs.Taking into account the locally tuned response of RBFs, the objective of this paper is to study the influence of global and local paradigms on the weights training phase, within the RBFNs design methodology, for imbalanced data-sets. Least Mean Square and the Singular Value Decomposition have been chosen as representatives of local and global weights training paradigms respectively. These learning algorithms are inserted into classical RBFN design methods that are run on imbalanced data-sets and also on these data-sets preprocessed with re-balance techniques. After applying statistical tests to the results obtained, some guidelines about the RBFN design methodology for imbalanced data-sets are provided.     

Genetically evolved radial basis function network based prediction of drill flank wear

The most important factor that governs the performance of a radial basis function network (RBFN) is the optimization of the network architecture, i.e. determining the exact number of radial basis functions (RBFs) in the hidden layer that can best minimize the error between the actual and network outputs. This work presents a genetic algorithm (GA) based evolution of optimal RBFN architecture and compares its performance with the conventional RBFN training procedure employing a two stage methodology, i.e. utilizing the k-means clustering algorithm for the unsupervised training in the first stage, and using linear supervised techniques for subsequent error minimization in the second stage. The validation of the proposed methodology is carried out for the prediction of flank wear in the drilling process following a series of experiments involving high speed steel (HSS) drills for drilling holes on mild-steel workpieces. The genetically grown RBFN not only provides an improved network performance, it is also computationally efficient as it eliminates the need for the error minimization routine in the second stage training of RBFN.     

Evolutional RBFNs prediction systems generation in the applications of financial time series data

An artificial neural prediction system is automatically developed with the combinations of step wise regression analysis (SRA), dynamic learning and recursive-based particle swarm optimization (RPSO) learning algorithms. In the first stage, the SRA can be considered like a data filtering machine to choose two primary factors from 20 channel technical indexes as input variables of the RBFNs system. Then, an efficient dynamic learning algorithm is applied to sequentially generate RBFs functions from training data set, where it can efficiently determine the proper number of RBFs’ centers and their associated positions. It can be exploited to forecast appropriate behaviors of the wanted identified financial time series data. While characteristics of training data set are automatically mined and generated by the proposed dynamic learning algorithm, architecture of the RBFNs prediction system is initially represented with collected information. Moreover, the RPSO learning scheme with the hybrid particle swarm optimization (PSO) and recursive least-squares (RLS) learning methods are applied to extract those appropriate parameters of the RBFNs prediction system.The RBFNs prediction systems are implemented in data analysis, module generation and price trend of the financial time series data. It not only automatically determines proper RBFs number but also fast approach the desired target in actual trading of Taiwan stock index (TAIEX). Computer simulations in training and testing phases of historic TAIEX are compared with other learning methods, which illustrate our great performance not only increases the accuracy of the stock price prediction but also improves the win rate in the trend of TAIEX.     

一种新的径向基概率神经网络模型(Ⅰ):基本理论

文中在径向基函数网络（ＲＢＦＮ）和概率神经网络（ＰＮＮ）的基础上，提出了一种径向基概率神经网络（ＲＢＰＮＮ）模型，这种网络保留了前两种网络模型的优点，既可以减少网络连接权值的训练时间，又能减少网络隐单元的数目，同时，网络用于测试的时间也较ＲＢＦＮ明显地下降．     

AGENT-BASED APPROACH TO THE DESIGN OF RBF NETWORKS

This article proposes a novel approach to the radial basis function network (RBFN) design. Its main idea is to apply the agent-based population learning algorithm to the task of initialization and training RBFNs. The approach allows for an effective network initialization and estimation of its output weights. The initialization involves two stages, where in the first one initial clusters are produced using the similarity-based procedure and next, in the second stage, prototypes (centroids) from the thus-obtained clusters are selected. The agent-based population learning algorithm is used to select prototypes. In the proposed implementation of the algorithm, both tasks—RBFN initialization and RBFN training—are carried out by a team of agents executing various local search procedures and cooperating with a view to determine the solution to the RBFN design problem at hand. The performance of the RBFN constructed using the proposed agent-based approach is analyzed and evaluated. The proposed approach is also compared with different RBFN initialization and training procedures in the literature.     

CO2RBFN: an evolutionary cooperative–competitive RBFN design algorithm for classification problems

This paper presents a new evolutionary cooperative–competitive algorithm for the design of Radial Basis Function Networks (RBFNs) for classification problems. The algorithm, CO²RBFN, promotes a cooperative–competitive environment where each individual represents a radial basis function (RBF) and the entire population is responsible for the final solution. The proposal considers, in order to measure the credit assignment of an individual, three factors: contribution to the output of the complete RBFN, local error and overlapping. In addition, to decide the operators’ application probability over an RBF, the algorithm uses a Fuzzy Rule Based System. It must be highlighted that the evolutionary algorithm considers a distance measure which deals, without loss of information, with differences between nominal features which are very usual in classification problems. The precision and complexity of the network obtained by the algorithm are compared with those obtained by different soft computing methods through statistical tests. This study shows that CO²RBFN obtains RBFNs with an appropriate balance between accuracy and simplicity, outperforming the other methods considered.     

Short-term photovoltaic power forecasting using hybrid contrastive learning and temporal convolutional network under future meteorological information absence

Photovoltaic (PV) power generation is widely utilized to satisfy the increasing energy demand due to its cleanness and inexhaustibility. Accurate PV power forecasting can improve the penetration of PV power in the grid. However, it is pretty challenging to predict PV power in short-term under precious future meteorological information absence conditions. To address this problem, this study proposes the hybrid Contrastive Learning and Temporal Convolutional Network (CL-TCN), and this forecasting approach consists of two parts, including model training and adaptive processes of forecasting models. In the model training stage, this forecasting method firstly trains 18 TCN models for 18 time points from 9:00 a.m. to 17:30 p.m. These TCN models are trained by only using historical PV power data samples, and each model is used to predict the next half-hour power output. The adaptive process of models means that, in a practical forecasting stage, PV power samples from historical data are firstly evaluated and scored by a CL based data scoring mechanism to search for the most similar data samples to current measured samples. Then these similar samples are further applied to training a single above-mentioned well-trained TCN model to improve its performance in forecasting the next half-hour PV power. The experimental results tested at the time resolution of 30 min demonstrate that the proposed approach has superior performance in forecasting accuracy not only in smooth PV power samples but also in fluctuating PV power samples. Moreover, the proposed CL based data scoring mechanism can filter useless data samples effectively accelerating the forecasting process.     

A new hybrid methodology for cooperative-coevolutionary optimization of radial basis function networks

This paper presents a new multiobjective cooperative–coevolutive hybrid algorithm for the design of a Radial Basis Function Network (RBFN). This approach codifies a population of Radial Basis Functions (RBFs) (hidden neurons), which evolve by means of cooperation and competition to obtain a compact and accurate RBFN. To evaluate the significance of a given RBF in the whole network, three factors have been proposed: the basis function’s contribution to the network’s output, the error produced in the basis function radius, and the overlapping among RBFs. To achieve an RBFN composed of RBFs with proper values for these quality factors our algorithm follows a multiobjective approach in the selection process. In the design process, a Fuzzy Rule Based System (FRBS) is used to determine the possibility of applying operators to a certain RBF. As the time required by our evolutionary algorithm to converge is relatively small, it is possible to get a further improvement of the solution found by using a local minimization algorithm (for example, the Levenberg–Marquardt method). In this paper the results of applying our methodology to function approximation and time series prediction problems are also presented and compared with other alternatives proposed in the bibliography.     

GPS orbit approximation using radial basis function networks

We present solutions for GPS orbit computation from broadcast and precise ephemerides using a group of artificial neural networks (ANNs), i.e. radial basis function networks (RBFNs). The problem of broadcast orbit correction, resulting from precise ephemerides, has already been solved using traditional polynomial and trigonometric interpolation. As an alternative approach RBFN broadcast orbit correction produces results within the accuracy range of the traditional methods. Our study shows RBFN broadcast orbit correction performs well also near the end of data intervals and for short data spans (～20 min). Regarding limitations of polynomial and trigonometric extrapolation, the most significant advantage of using RBFNs over the traditional methods for GPS broadcast orbit approximation arises from its short time prediction capability.     

Plane-Gaussian artificial neural network

Multilayer perceptrons (MLPs) and radial basis functions networks (RBFNs) have been widely concerned in recent years. In this paper, based on k-plane clustering (kPC) algorithm, we propose a novel artificial network model termed as Plane-Gaussian network to enlarge the arsenal of the neural networks. This network adopts a so-called Plane-Gaussian activation function (PGF) in hidden neurons. Replacing traditional central point of Gaussian radial basis function (RBF) with central hyperplane, PGF forms a band-shaped rather than spheral-shaped receptive field in RBF, which makes PGF able to express its peculiar geometrical characteristics: locality and globality. Importantly, it is also proved that PGF network (PGFN) having one hidden layer is capable of universal approximation. As a universal approximator, PGFN gives an informal way of bridging the gap between MLP and RBFN. The experiments report comparison between training time and classification accuracies on some artificial and UCI datasets and conclude that (1) PGFN runs significantly faster than MLP and (2) PGFN has comparable or better classification performance than MLP and RBFN, especially in subspace-distributed datasets.     

积灰条件下光伏阵列最优倾角问题研究

在光伏发电系统设计中,安装倾角的选择对光伏发电效率具有重要影响.针对太阳能光伏阵列常见的表层积灰现象,改变传统的只考虑最大辐射量的倾角确定方法,提出了综合考虑表层积灰情况下的最优发电倾角计算方法,使倾角的确定更加合理与完善.建立了积灰辐射量统一发电模型,并基于Matlab对模型进行了仿真验证.以福建某光伏电站为例,搭建实验平台,通过相关数据的检验与预测,得到了光伏电池板的综合最优倾角.实验结果证明该模型所确定的倾角比传统模型可以得到更大的发电量,提高了光伏利用效率.     

基于RBFN-PCR方法的制浆蒸煮终点预测模型

实现制浆蒸煮终点的精确预测是稳定纸浆质量的关键，也是制浆行业一直未能很好解决的难题。利用径向基函数网络(RBFN)的最佳逼近性能和主成分回归(PCR)的空间变换技术，提出了一种径向基函数网络-主成分回归(RBFN-PCR)的建模方法。RBFN-PCR方法在确定隐含层结构和参数时，将隐单元数取为训练样本数，径向基函数中心矢量取相应样本值，宽度参数采用尝试方法选取，隐含层到输出层的网络权系数运用PCR求解。RBFN—PCR方法既保留了径向基函数网络的结构，又用数学方法直接求解，免去了冗长的训练过程和其它诸多欠缺。将RBFN-PCR方法用于间歇制浆蒸煮终点预测的建模，并通过实际生产数据对模型进行验证，结果显示模型的平均预测相对误差为6．2l％，与文献上的10．62％相比，预测精度提高了许多。     

Application of soft computing models to hourly weather analysis in southern Saskatchewan,Canada

Accurate weather forecasts are necessary for planning our day-to-day activities. However, dynamic behavior of weather makes the forecasting a formidable challenge. This study presents a soft computing model based on a radial basis function network (RBFN) for 24-h weather forecasting of southern Saskatchewan, Canada. The model is trained and tested using hourly weather data of temperature, wind speed and relative humidity in 2001. The performance of the RBFN is compared with those of multi-layered perceptron (MLP) network, Elman recurrent neural network (ERNN) and Hopfield model (HFM) to examine their applicability for weather analysis. Reliabilities of the models are then evaluated by a number of statistical measures. The results indicate that the RBFN produces the most accurate forecasts compared to the MLP, ERNN and HFM.     

Control of chaotic dynamical systems using radial basis function network approximators

This paper presents a general control method based on radial basis function networks (RBFNs) for chaotic dynamical systems. For many chaotic systems that can be decomposed into a sum of a linear and a nonlinear part, under some mild conditions the RBFN can be used to well approximate the nonlinear part of the system dynamics. The resulting system is then dominated by the linear part, with some small or weak residual nonlinearities due to the RBFN approximation errors. Thus, a simple linear state-feedback controller can be devised, to drive the system response to a desirable set-point. In addition to some theoretical analysis, computer simulations on two representative continuous-time chaotic systems (the Duffing and the Lorenz systems) are presented to demonstrate the effectiveness of the proposed method.     

Neural network model for the efficient calculation of Green's functions in layered media

In this article, neural networks are employed for fast and efficient calculation of Green's functions in a layered medium. Radial basis function networks (RBFNs) are effectively trained to estimate the coefficients and the exponents that represent a Green's function in the discrete complex image method (DCIM). Results show very good agreement with the DCIM, and the trained RBFNs are very fast compared with the corresponding DCIM. © 2003 Wiley Periodicals, Inc. Int J RF and Microwave CAE 13: 128–135, 2003.     

Almost optimal estimates for approximation and learning by radial basis function networks

This paper quantifies the approximation capability of radial basis function networks (RBFNs) and their applications in machine learning theory. The target is to deduce almost optimal rates of approximation and learning by RBFNs. For approximation, we show that for large classes of functions, the convergence rate of approximation by RBFNs is not slower than that of multivariate algebraic polynomials. For learning, we prove that, using the classical empirical risk minimization, the RBFNs estimator can theoretically realize the almost optimal learning rate. The obtained results underlie the successful application of RBFNs in various machine learning problems.     

酸性磷铵料浆浓缩过程中氟含量的预测模型

在磷铵生产过程中,料浆的氟含量预测对生产具有重要意义。本文将径向基函数网络(RBFN)与循环子空间回归(CSR)相结合,设计了RBFN—CSR建模方法。RBFN—CSR方法在确定隐含层结构和参数时,将隐单元数取为训练样本数,径向基函数中心矢量取相应样本值,宽度参数根据样本分布情况采用尝试方法选取,隐含层到输出层的网络权系数运用CSR求解。CSR求解过程包容了最小二乘回归(LSR)、主成分回归(PCR)、偏最小二乘回归(PLSR)以及很多中间的回归方法,它可在非常广泛的范围内根据某一准则选择最优的网络结构参数。运用RBFN—CSR方法建立了酸性磷铵料浆浓缩过程中氟含量的预测模型,交叉验证表明,该模型具有较高的预测精度和良好的稳定性能,有一定的实际应用价值。     

Immunological Algorithm-based Neural Network Learning for Sales Forecasting

This study proposes a variation immunological system (VIS) algorithm with radial basis function neural network (RBFN) learning for function approximation and the exercise of industrial computer (IC) sales forecasting. The proposed VIS algorithm was applied to the RBFN to execute the learning process for adjusting the network parameters involved. To compare the performance of relevant algorithms, three benchmark problems were used to justify the results of the experiment. With better accuracy in forecasting, the trained RBFN can be practically utilized in the IC sales forecasting exercise to make predictions and could enhance business profit.     

Adaptive GA-based reconfiguration of photovoltaic array combating partial shading conditions

This paper concerns the study and simulation of a PV array self-organizing configuration. It introduces a new method to reconfigure the PV array using a genetic algorithm in order to maximize the output power as well as reducing the number of switching. The proposed method involves the simulation of a PV array composed of 16 panels 4 strings with 4 panels in series and associated parallel, as well as an algorithm that controls the improvement of the overall performance under different shading conditions. The obtained results using MATLAB/Simulink simulation show improvement rating varying between 106.49 and 171.03%, which is huge compared to a static configuration operating below the total available power. Another important point is the number of iterations needed to find the optimal configuration (between 6 and 132 for a population of 50 configurations tested at each generation); this means that in the worst case (132 iterations), the proposed algorithm performed 132 × 50 = 6600 configurations instead of 16¹⁶ = 1.84 × 10¹⁹ necessary in case of exhaustive search to test all possible configurations. This last point is very important in the implementation of the proposed system in auto-tuning of the system in real-time condition. Besides using genetic algorithm to track the optimal configuration, our main contribution consists of improving the output power while reducing the number of switching by keeping PV modules, if possible, in same position (0 switching) or on the same line/column (1 switching) in few iteration needing only two sensors one for the voltage and another for the current of the PV array.

     

On robustness of radial basis function network with input perturbation

In this article, we have proposed a methodology for making a radial basis function network (RBFN) robust with respect to additive and multiplicative input noises. This is achieved by properly selecting the centers and widths for the radial basis function (RBF) units of the hidden layer. For this purpose, firstly, a set of self-organizing map (SOM) networks are trained for center selection. For training a SOM network, random Gaussian noise is injected in the samples of each class of the data set. The number of SOM networks is same as the number of classes present in the data set, and each of the SOM networks is trained separately by the samples belonging to a particular class. The weight vector associated with a unit in the output layer of a particular SOM network corresponding to a class is used as the center of a RBF unit for that class. To determine the widths of the RBF units, p-nearest neighbor algorithm is used class-wise. Proper selection of centers and widths makes the RBFN robust with respect to input perturbation and outliers present in the data set. The weights between the hidden and output layers of RBFN are obtained by pseudo inverse method. To test the robustness of the proposed method in additive and multiplicative noise scenarios, ten standard data sets have been used for classification. Proposed method has been compared with three existing methods, where the centers have been generated in three ways: randomly, using k-means algorithm, and based on SOM network. Simulation results show the superiority of the proposed method compared to those methods. Wilcoxon signed-rank test also shows that the proposed method is statistically better than those methods.