Prototype selection algorithms for distributed learning

Distributed learning from data is one of the typical tasks solved by distributed data-mining techniques and is seen as a fundamental computational problem. One of the approaches suitable for distributed learning is to select, by data reduction, relevant local patterns, called also prototypes, from geographically distributed databases. Next, locally selected prototypes can be moved to other sites and merged into the global knowledge model. The paper presents three agent-based population learning algorithms for distributed learning. The proposed algorithms are based on agent collaborations in distributed prototype selection processes and on agent collaborations when the learning global model is created. The basic property of the presented algorithms is that the prototypes are selected by agent-based population learning algorithm from data clusters induced at distributed sites. The main goal of the paper is to empirically compare how the way of inducing such clusters can influence the distributed learning performance. The paper investigates the agent-based population learning algorithms used to solve distributed data reduction and gives a brief discussion of the procedures for clusters initialization. Finally, computational experiment results are shown.     

Designing RBF Networks Using the Agent-Based Population Learning Algorithm

Radial Basis Function Neural Networks (RBFNs) are nowadays quite popular due to their ability to discover and approximate complex nonlinear dependencies within the data under analysis. Performance of the RBF network depends on numerous factors related to its initialization and training. The paper proposes an approach to the radial basis function networks design, where initial parameters of the network, output weights and parameters of the transfer function are set using the proposed agent-based population learning algorithm (PLA). The algorithm is validated experimentally. Advantages and main features of the PLA-based RBF designs are discussed basing on results of the computational experiment.     

Integration of support vector regression and annealing dynamical learning algorithm for MIMO system identification

This paper presents a robust approach to identify multi-input multi-output (MIMO) systems. Integrating support vector regression (SVR) and annealing dynamical learning algorithm (ADLA), the proposed method is adopted to optimize a radial basis function network (RBFN) for identification of MIMO systems. In the system identification, first, SVR is adopted to determine the number of hidden layer nodes, the initial structure of the RBFN. After initialization, ADLA with nonlinear time-varying learning rate is then applied to train the RBFN. In the ADLA, the determination of the learning rate would be an important work for the trade-off between stability and speed of convergence. A computationally efficient optimization method, particle swarm optimization (PSO) method, is adopted to simultaneously find optimal learning rates. Due to the advantages of SVR and ADLA (SVR-ADLA), the proposed RBFN (SVR-ADLA-RBFN) has good performance for MIMO system identification. Two examples are illustrated to show the feasibility and superiority of the proposed SVR-ADLA-RBFNs for identification of MIMO systems. Simulation results are provided to demonstrate the effectiveness of the proposed algorithm.     

一种RBF网络结构调整的稳健增量学习方法

以实现RBF网络的增量学习能力和提高其增量学习的稳健性为目的,给出了一种RBF网络增量学习算法.算法首先对初始数据集进行聚类得到初始的RBF网络结构,然后采用GAP-RBF算法中的隐层节点调整策略动态调整网络结构实现RBF网络增量学习.RBF网络的初始化降低了初始数据集样本训练顺序对RBF网络性能的影响,增强了其增量学习的稳健性.IRIS数据集和雷达实测数据集仿真实验表明,算法具有较好的增量学习能力.     

Fuzzy neural network with general parameter adaptation for modelingof nonlinear time-series

By taking advantage of fuzzy systems and neural networks, a fuzzy-neural network with a general parameter (GP) learning algorithm and heuristic model structure determination is proposed in this paper. Our network model is based on the Gaussian radial basis function network (RBFN). We use the flexible GP approach both for initializing the off-line training algorithm and fine-tuning the nonlinear model efficiently in online operation. A modification of the robust unbiasedness criterion using distorter (UCD) is utilized for selecting the structural parameters of this adaptive model. The UCD approach provides the desired modeling accuracy and avoids the risk of over-fitting. In order to illustrate the operation of the proposed modeling scheme, it is experimentally applied to a fault detection application.     

基于降噪自编码神经网络的事件相关电位脑电信号分析方法

提出一种基于降噪自编码神经网络事件相关电位分析方法,首先建立3层神经网络结构,利用降噪自编码对神经网络进行初始化,实现了降噪自编码深度学习模型的无监督学习.从无标签数据中自动学习数据特征,通过优化模型训练得到的权值作为神经网络初始化参数.其次,经过有标签的样本进行网络参数的微调即可完成对神经网络的训练,该方法有效解决了神经网络训练中因随机选择初始化参数,而导致网络易陷入局部极小的缺陷.最后,利用上述神经网络对第3届脑机接口竞赛数据集Data set Ⅱ(事件相关电位脑电信号)进行分类分析.实验结果表明:利用降噪自编码迭代2500次训练神经网络模型,在受试者A和受试者B样本数据叠加5次、10次、15次3种情况下获得的分类准确率分别为73.4%, 87.4%和97.2%.该最高准确率优于其他分类方法,比竞赛第1名联合支持向量机(SVM)分类器(ESVM)提高了0.7%,为事件相关电位脑电信号提供了一种深度学习分析方法.     

Reviewing and designing pre-processing units for RBF networks: initial structure identification and coarse-tuning of free parameters

This paper reviews some frequently used methods to initialize an radial basis function (RBF) network and presents systematic design procedures for pre-processing unit(s) to initialize RBF network from available input–output data sets. The pre-processing units are computationally hybrid two-step training algorithms that can be named as (1) construction of initial structure and (2) coarse-tuning of free parameters. The first step, the number, and the locations of the initial centers of RBF network can be determined. Thus, an orthogonal least squares algorithm and a modified counter propagation network can be employed for this purpose. In the second step, a coarse-tuning of free parameters is achieved by using clustering procedures. Thus, the Gustafson–Kessel and the fuzzy C-means clustering methods are evaluated for the coarse-tuning. The first two-step behaves like a pre-processing unit for the last stage (or fine-tuning stage—a gradient descent algorithm). The initialization ability of the proposed four pre-processing units (modular combination of the existing methods) is compared with three non-linear benchmarks in terms of root mean square errors. Finally, the proposed hybrid pre-processing units may initialize a fairly accurate, IF–THEN-wise readable initial model automatically and efficiently with a minimum user inference.     

A new hybrid methodology for cooperative-coevolutionary optimization of radial basis function networks

This paper presents a new multiobjective cooperative–coevolutive hybrid algorithm for the design of a Radial Basis Function Network (RBFN). This approach codifies a population of Radial Basis Functions (RBFs) (hidden neurons), which evolve by means of cooperation and competition to obtain a compact and accurate RBFN. To evaluate the significance of a given RBF in the whole network, three factors have been proposed: the basis function’s contribution to the network’s output, the error produced in the basis function radius, and the overlapping among RBFs. To achieve an RBFN composed of RBFs with proper values for these quality factors our algorithm follows a multiobjective approach in the selection process. In the design process, a Fuzzy Rule Based System (FRBS) is used to determine the possibility of applying operators to a certain RBF. As the time required by our evolutionary algorithm to converge is relatively small, it is possible to get a further improvement of the solution found by using a local minimization algorithm (for example, the Levenberg–Marquardt method). In this paper the results of applying our methodology to function approximation and time series prediction problems are also presented and compared with other alternatives proposed in the bibliography.     

修正的广义学习向量量化算法

讨论了Pal等的广义学习量化算法(GLVQ)和Karayiannis等的模糊学习量化算法(FGLVQ)的优缺点，提出了修正广义学习量化(RGLVQ)算法。该算法的迭代系数有很好的上下界，解决了GLVQ的“Scale”问题，又不像FGLVQ算法对初始学习率敏感。用IRIS数据集对算法进行了测试，并应用所给算法进行了用于图像压缩的量化码书设计。该文算法与FGLVQ类算法性能相当，但少了大量浮点除法，实验过程表明节约训练时间约l0％。     

GA-based learning for rule identification in fuzzy neural networks

Employing an effective learning process is a critical topic in designing a fuzzy neural network, especially when expert knowledge is not available. This paper presents a genetic algorithm (GA) based learning approach for a specific type of fuzzy neural network. The proposed learning approach consists of three stages. In the first stage the membership functions of both input and output variables are initialized by determining their centers and widths using a self-organizing algorithm. The second stage employs the proposed GA based learning algorithm to identify the fuzzy rules while the final stage tunes the derived structure and parameters using a back-propagation learning algorithm. The capabilities of the proposed GA-based learning approach are evaluated using a well-examined benchmark example and its effectiveness is analyzed by means of a comparative study with other approaches. The usefulness of the proposed GA-based learning approach is also illustrated in a practical case study where it is used to predict the performance of road traffic control actions. Results from the benchmarking exercise and case study effectively demonstrate the ability of the proposed three stages learning approach to identify relevant fuzzy rules from a training data set with a higher prediction accuracy than alternative approaches.     

一种面向数据分析的新型人工免疫模型研究

提出了一种新型的人工免疫网络模型TSIN。通过应用包括克隆选择、基于合作的变异以及抗体抑制在内的免疫算子,抗体种群从单一的个体逐步分化繁殖成为有效的聚类。这些聚类既能够准确地表示原始数据集在形态空间中的分布特性,又能够较好地拟合局部分布形态,这些都为高维数据的分析提供了良好的基础。描述了TSIN学习算法的总体框架,详细分析了其中的关键环节。仿真实验表明,TSIN具有良好的数据分析能力,且较传统的自组织神经网络方法更能体现数据中蕴含的拓扑关系和分布特性。     

多模态粒子群集成神经网络

提出一种基于多模态粒子群算法的神经网络集成方法,在网络训练每个迭代周期内利用改进的快速聚类算法在权值搜索空间上动态地把搜索粒子分为若干类,求得每一类的最优粒子,然后计算最优个体两两之间的输出空间相异度,合并相异度过低的两类粒子,最终形成不但权值空间相异、而且输出空间也相异的若干类粒子,每类粒子负责一个成员网络权值的搜索,其中最优粒子对应于一个成员网络,所有类的最优粒子组成神经网络集成,成员网络的个数是由算法自动确定的.算法控制网络多样性的方法更直接、更有效.与负相关神经网络集成、bagging和boosting方法比较,实验结果表明,此算法较好地提高了神经网络集成的泛化能力.     

Wavelet fuzzy LVQ based speaker verification system

This work evaluates the performance of speaker verification system based on Wavelet based Fuzzy Learning Vector Quantization (WLVQ) algorithm. The parameters of Gaussian mixture model (GMM) are designed using this proposed algorithm. Mel Frequency Cepstral Coefficients (MFCC) are extracted from the speech data and vector quantized through Wavelet based FLVQ algorithm. This algorithm develops a multi resolution codebook by updating both winning and nonwinning prototypes through an unsupervised learning process. This codebook is used as mean vector of GMM. The other two parameters, weight and covariance are determined from the clusters formed by the WLVQ algorithm. The multi resolution property of wavelet transform and ability of FLVQ in regulating the competition between prototypes during learning are combined in this algorithm to develop an efficient codebook for GMM. Because of iterative nature of Expectation Maximization (EM) algorithm, the applicability of alternative training algorithms is worth investigation. In this work, the performance of speaker verification system using GMM trained by LVQ, FLVQ and WLVQ algorithms are evaluated and compared with EM algorithm. FLVQ and WLVQ based training algorithms for modeling speakers using GMM yields better performance than EM based GMM.     

Self-splitting competitive learning: a new on-line clusteringparadigm

Clustering in the neural-network literature is generally based on the competitive learning paradigm. The paper addresses two major issues associated with conventional competitive learning, namely, sensitivity to initialization and difficulty in determining the number of prototypes. In general, selecting the appropriate number of prototypes is a difficult task, as we do not usually know the number of clusters in the input data a priori. It is therefore desirable to develop an algorithm that has no dependency on the initial prototype locations and is able to adaptively generate prototypes to fit the input data patterns. We present a new, more powerful competitive learning algorithm, self-splitting competitive learning (SSCL), that is able to find the natural number of clusters based on the one-prototype-take-one-cluster (OPTOC) paradigm and a self-splitting validity measure. It starts with a single prototype randomly initialized in the feature space and splits adaptively during the learning process until all clusters are found; each cluster is associated with a prototype at its center. We have conducted extensive experiments to demonstrate the effectiveness of the SSCL algorithm. The results show that SSCL has the desired ability for a variety of applications, including unsupervised classification, curve detection, and image segmentation.     

采用数据挖掘和代理技术的入侵检测系统研究

入侵检测系统是一种检测网络入侵行为并能够主动保护自己免受攻击的一种网络安全技术,是网络防火墙的合理补充.介绍了应用几种数据挖掘方法进行入侵检测的过程,并在此基础上提出了一个采用数据挖掘技术的基于代理的网络入侵检测系统模型.该模型由一定数量的代理组成,训练和检测过程完全不同与其它系统.由于代理的自学习能力,该系统具有自适应性和可扩展性.     

Detection of anomalous episodes in urban Ozone maps

In addition to classification and regression, outlier detection has emerged as a relevant activity in deep learning. In comparison with previous approaches where the original features of the examples were used for separating the examples with high dissimilarity from the rest of the examples, deep learning can automatically extract useful features from raw data, thus removing the need for most of the feature engineering efforts usually required with classical machine learning approaches. This requires training the deep learning algorithm with labels identifying the examples or with numerical values. Although outlier detection in deep learning has been usually undertaken by training the algorithm with categorical labels—classifier—, it can also be performed by using the algorithm as regressor. Nowadays numerous urban areas have deployed a network of sensors for monitoring multiple variables about air quality. The measurements of these sensors can be treated individually—as time series—or collectively. Collectively, a variable monitored by a network of sensors can be transformed into a map. Maps can be used as images in machine learning algorithms—including computer vision algorithms—for outlier detection. The identification of anomalous episodes in air quality monitoring networks allows later processing this time period with finer‐grained scientific packages involving fluid dynamic and chemical evolution software, or the identification of malfunction stations. In this work, a Convolutional Neural Network is trained—as a regressor—using as input Ozone‐urban images generated from the Air Quality Monitoring Network of Madrid (Spain). The learned features are processed by Density‐based Spatial Clustering of Applications with Noise (DBSCAN) algorithm for identifying anomalous maps. Comparisons with other deep learning architectures are undertaken, for instance, autoencoders—undercomplete and denoizing—for learning salient features of the maps and later to use as input of DBSCAN. The proposed approach is able efficiently find maps with local anomalies compared to other approaches based on raw images or latent features extracted with autoencoders architectures with DBSCAN.     

Genetically evolved radial basis function network based prediction of drill flank wear

The most important factor that governs the performance of a radial basis function network (RBFN) is the optimization of the network architecture, i.e. determining the exact number of radial basis functions (RBFs) in the hidden layer that can best minimize the error between the actual and network outputs. This work presents a genetic algorithm (GA) based evolution of optimal RBFN architecture and compares its performance with the conventional RBFN training procedure employing a two stage methodology, i.e. utilizing the k-means clustering algorithm for the unsupervised training in the first stage, and using linear supervised techniques for subsequent error minimization in the second stage. The validation of the proposed methodology is carried out for the prediction of flank wear in the drilling process following a series of experiments involving high speed steel (HSS) drills for drilling holes on mild-steel workpieces. The genetically grown RBFN not only provides an improved network performance, it is also computationally efficient as it eliminates the need for the error minimization routine in the second stage training of RBFN.     

Evolutionary learning based sustainable strain sensing model for structural health monitoring of high-rise buildings

Strain sensor network-based structural health monitoring systems have been used to assess the safety of high-rise buildings. In consideration of life cycle of high-rise buildings, long-term measurement by sensors should be required. However, because of unpredictable problems such as the lack of durability of sensors and data loggers, disruption in communication, and loss of data, long-term strain measurement of major structural members is currently infeasible. For sustainable safety assessment of high-rise buildings, this paper presents a sustainable strain-sensing model that employs an artificial neural network (ANN) to estimate the strain responses of columns depending on the wind-induced behavior of high-rise buildings. The ANN model used in the paper is based on evolutionary learning consists of training in radial basis function neural network (RBFN) and evolving in genetic algorithm. In this evolutionary RBFN (ERBFN). Weights between layers are trained and variables of Gaussian function in the RBFN are evolved to estimate strain responses of the column of the high-rise building structure. A wind tunnel test was performed to produce wind data and strains in column members in a high-rise building model. In the wind tunnel test, a specimen consisting of a core, perimeter columns, and outriggers is used to simulate the conditions of typical high-rise buildings with a slenderness ratio of 5.0. The proposed model is trained and verified by using the wind data such as wind speeds and directions and the corresponding strains measured with fiber optic grating sensors. In addition to estimation of the maximum and minimum values of strains in vertical members in a high-rise building, it is found that the proposed model can build a relationship between the wind data and strain of vertical members.     

Data compression and local metrics for nearest neighborclassification

A local distance measure for the nearest neighbor classification rule is shown to achieve high compression rates and high accuracy on real data sets. In the approach proposed here, first, a set of prototypes is extracted during training and, then, a feedback learning algorithm is used to optimize the metric. Even if the prototypes are randomly selected, the proposed metric outperforms, both in compression rate and accuracy, common editing procedures like ICA, RNN, and PNN. Finally, when accuracy is the major concern, we show how compression can be traded for accuracy by exploiting voting techniques. That indicates how voting can be successfully integrated with instance-based approaches, overcoming previous negative results     

Optimal design of radial basis function neural networks for fuzzy-rule extraction in high dimensional data

The design of an optimal radial basis function neural network (RBFNF) is not a straightforward procedure. In this paper we take advantage of the functional equivalence between RBFN and fuzzy inference systems to propose a novel efficient approach to RBFN design for fuzzy rule extraction. The method is based on advanced fuzzy clustering techniques. Solutions to practical problems are proposed. By combining these different solutions, a general methodology is derived. The efficiency of our method is demonstrated on challenging synthetic and real world data sets.