自构造RBF神经网络及其参数优化

径向基函数神经网络的构造需要确定每个RBF的中心、宽度和数目。该文利用改进的聚类算法自动构造RBFN,考虑样本的类别属性,根据样本分布自动计算RBF的中心和宽度,并确定RBF的数目。所有的网络参数采用非线性优化算法来优化。通过IRIS分类问题和混沌时间序列预测评价自构建RBFN的性能,验证参数优化效果。结果表明,自构造RBFN不但能够自动确定网络结构,而且具有良好的模式分类和函数逼近能力。通过对网络参数的非线性优化,该算法明显改善了网络性能。     

Neural Learning from Unbalanced Data

This paper describes the result of our study on neural learning to solve the classification problems in which data is unbalanced and noisy. We conducted the study on three different neural network architectures, multi-layered Back Propagation, Radial Basis Function, and Fuzzy ARTMAP using three different training methods, duplicating minority class examples, Snowball technique and multidimensional Gaussian modeling of data noise. Three major issues are addressed: neural learning from unbalanced data examples, neural learning from noisy data, and making intentional biased decisions. We argue that by properly generated extra training data examples around the noise densities, we can train a neural network that has a stronger capability of generalization and better control of the classification error of the trained neural network. In particular, we focus on problems that require a neural network to make favorable classification to a particular class such as classifying normal(pass)/abnormal(fail) vehicles in an assembly plant. In addition, we present three methods that quantitatively measure the noise level of a given data set. All experiments were conducted using data examples downloaded directly from test sites of an automobile assembly plant. The experimental results showed that the proposed multidimensional Gaussian noise modeling algorithm was very effective in generating extra data examples that can be used to train a neural network to make favorable decisions for the minority class and to have increased generalization capability.     

Rule-Based Learning Systems for Support Vector Machines

In this article, we propose some methods for deriving symbolic interpretation of data in the form of rule based learning systems by using Support Vector Machines (SVM). First, Radial Basis Function Neural Networks (RBFNN) learning techniques are explored, as is usual in the literature, since the local nature of this paradigm makes it a suitable platform for performing rule extraction. By using support vectors from a learned SVM it is possible in our approach to use any standard Radial Basis Function (RBF) learning technique for the rule extraction, whilst avoiding the overlapping between classes problem. We will show that merging node centers and support vectors explanation rules can be obtained in the form of ellipsoids and hyper-rectangles. Next, in a dual form, following the framework developed for RBFNN, we construct an algorithm for SVM. Taking SVM as the main paradigm, geometry in the input space is defined from a combination of support vectors and prototype vectors obtained from any clustering algorithm. Finally, randomness associated with clustering algorithms or RBF learning is avoided by using only a learned SVM to define the geometry of the studied region. The results obtained from a certain number of experiments on benchmarks in different domains are also given, leading to a conclusion on the viability of our proposal.     

Bagging组合的不平衡数据分类方法

提出一种基于Bagging组合的不平衡数据分类方法CombineBagging,采用少数类过抽样算法SMOTE进行数据预处理,在此基础上利用C-SVM、径向基函数神经网络、Random Forests 3种不同的基分类器学习算法,分别对采样后的数据样本进行Bagging集成学习,通过投票规则集成学习结果。实验结果表明,该方法能够提高少数类的分类准确率,有效处理不平衡数据分类问题。     

小数据集条件下基于数据再利用的BN参数学习

着重研究了小数据集条件下结合凸约束的离散贝叶斯网络(Bayesian network, BN)参数学习问题, 主要任务是用先验知识弥补数据的不足以提高参数学习精度. 已有成果认为数据和先验知识是独立的, 在参数学习算法中仅将二者机械结合. 经过理论研究后, 本文认为数据和先验知识并不独立, 原有算法浪费了这部分有用信息. 本文立足于数据信息分类, 深入挖掘数据和先验知识之间的约束信息来提高参数学习精度, 提出了新的BN 参数学习算法 --凸约束条件下基于数据再利用的贝叶斯估计. 通过仿真实验展示了所提算法在精度和其他性能上的优势, 进一步证明数据和先验知识不独立思想的合理性.     

A learning algorithm for Recurrent Radial Basis Function Networks

This paper proposes a Recurrent Radial Basis Function network (RRBFN) that can be applied to temporal pattern classifications and predictions. Based on the architecture of the conventional Radial Basis Function networks, the RRBFNs use Gaussian nonlinearity and have feedback paths between every hidden node. These feedback paths enable the networks to learn temporal patterns without an input buffer to hold the recent elements of an input sequence. A gradient descent learning algorithm for the RRBFNs is derived. Two RRBFNs with different number of hidden nodes were tested using a temporal sequence generated by an infinite impulse response filter. The results show that the RRBFNs can approximate the filter more accurate than the Continually Running Fully Recurrent networks trained by the Real-Time Recurrent Learning algorithm.     

Designing RBF Networks Using the Agent-Based Population Learning Algorithm

Radial Basis Function Neural Networks (RBFNs) are nowadays quite popular due to their ability to discover and approximate complex nonlinear dependencies within the data under analysis. Performance of the RBF network depends on numerous factors related to its initialization and training. The paper proposes an approach to the radial basis function networks design, where initial parameters of the network, output weights and parameters of the transfer function are set using the proposed agent-based population learning algorithm (PLA). The algorithm is validated experimentally. Advantages and main features of the PLA-based RBF designs are discussed basing on results of the computational experiment.     

A hybrid MPSO-BP structure adaptive algorithm for RBFNs

This paper introduces a novel hybrid algorithm to determine the parameters of radial basis function neural networks (number of neurons, centers, width and weights) automatically. The hybrid algorithm combines the mix encoding particle swarm optimization algorithm with the back propagation (BP) algorithm to form a hybrid learning algorithm (MPSO-BP) for training Radial Basis Function Networks (RBFNs), which adapts to the network structure and updates its weights by choosing a special fitness function. The proposed method is used to deal with three nonlinear problems, and the results obtained are compared with existent bibliography, showing an improvement over the published methods.     

Classification of gene-expression data: The manifold-based metric learning way

Classification of microarray gene-expression data can potentially help medical diagnosis, and becomes an important topic in bioinformatics. However, microarray data sets are usually of small sample size relative to an overwhelming number of genes. This makes the classification problem fairly challenging. Instance-based learning (IBL) algorithms, such as nearest neighbor (k-NN), are usually the baseline algorithm due to their simplicity. However, practices show that k-NN performs not very well in this field. This paper introduces manifold-based metric learning to improve the performance of IBL methods. A novel metric learning algorithm is proposed by utilizing both local manifold structural information and local discriminant information. In addition, a random subspace extension is also presented. We apply the proposed algorithm to the gene-classification problem in three ways: one in the original feature space, another in the reduced feature space, and the third via the random subspace extension. Statistical evaluation shows that the proposed algorithm can achieve promising results, and gain significant performance improvement over traditional IBL algorithms.     

超平面中心的RBF神经网络及其新方法

在传统的径向基神经网络框架的基础上,通过引入中心超平面的概念,提出了超平面中心的径向基函数神经网络。在此网络中以点到中心超平面的距离代替传统的径向基神经网络中点到点的距离,其优势在于中心超平面作为数据中心包含了更多原始数据之间的信息。以函数逼近和数据分类的实验为例,证明了超平面中心的径向基神经网络相对于传统的网络有一定的优势。     

Robust and adaptive techniques in self-organizing neural networks

Robust and adaptive training algorithms aiming at enhancing the capabilities of self-organizing and Radial Basis Function (RBF) neural networks are reviewed in this paper. The following robust variants of Learning Vector Quantizer (LVQ) are described: the order statistics LVQ, the L ₂ LVQ and the split-merge LVQ. Successful application of the marginal median LVQ that belongs to the class of order statistics LVQs in the self-organized selection of the centers in RBF neural networks is reported. Moreover, the use of the median absolute deviation in the estimation of the covariance matrix of the observations assigned to each hidden unit in RBF neural networks is proposed. Applications that prove the superiority of the proposed variants of LVQ and RBF neural networks in noisy color image segmentation, color-based image recognition, segmentation of ultrasonic images, motion-field smoothing and moving object segmentation are outlined.     

混杂数据的多核几何平均度量学习

在机器学习和模式识别任务中,选择一种合适的距离度量方法是至关重要的.度量学习主要利用判别性信息学习一个马氏距离或相似性度量.然而,大多数现有的度量学习方法都是针对数值型数据的,对于一些有结构的数据（比如符号型数据）,用传统的距离度量来度量两个对象之间的相似性是不合理的;其次,大多数度量学习方法会受到维度的困扰,高维度使得训练时间长,模型的可扩展性差.提出了一种基于几何平均的混杂数据度量学习方法.采用不同的核函数将数值型数据和符号型数据分别映射到可再生核希尔伯特空间,从而避免了特征的高维度带来的负面影响.同时,提出了一个基于几何平均的多核度量学习模型,将混杂数据的度量学习问题转化为求黎曼流形上两个点的中心点问题.在UCI数据集上的实验结果表明,针对混杂数据的多核度量学习方法与现有的度量学习方法相比,在准确性方面展现出更优异的性能.     

The Supervised Network Self-Organizing Map for Classification of Large Data Sets

Complex application domains involve difficult pattern classification problems. The state space of these problems consists of regions that lie near class separation boundaries and require the construction of complex discriminants while for the rest regions the classification task is significantly simpler. The motivation for developing the Supervised Network Self-Organizing Map (SNet-SOM) model is to exploit this fact for designing computationally effective solutions. Specifically, the SNet-SOM utilizes unsupervised learning for classifying at the simple regions and supervised learning for the difficult ones in a two stage learning process. The unsupervised learning approach is based on the Self-Organizing Map (SOM) of Kohonen. The basic SOM is modified with a dynamic node insertion/deletion process controlled with an entropy based criterion that allows an adaptive extension of the SOM. This extension proceeds until the total number of training patterns that are mapped to neurons with high entropy (and therefore with ambiguous classification) reduces to a size manageable numerically with a capable supervised model. The second learning phase (the supervised training) has the objective of constructing better decision boundaries at the ambiguous regions. At this phase, a special supervised network is trained for the computationally reduced task of performing the classification at the ambiguous regions only. The performance of the SNet-SOM has been evaluated on both synthetic data and on an ischemia detection application with data extracted from the European ST-T database. In all cases, the utilization of SNet-SOM with supervised learning based on both Radial Basis Functions and Support Vector Machines has improved the results significantly related to those obtained with the unsupervised SOM and has enhanced the scalability of the supervised learning schemes. The highly disciplined design of the generalization performance of the Support Vector Machine allows to design the proper model for the particular training set.     

Representation of Finite State Automata in Recurrent Radial Basis Function Networks

In this paper, we propose some techniques for injecting finite state automata into Recurrent Radial Basis Function networks (R2BF). When providing proper hints and constraining the weight space properly, we show that these networks behave as automata. A technique is suggested for forcing the learning process to develop automata representations that is based on adding a proper penalty function to the ordinary cost. Successful experimental results are shown for inductive inference of regular grammars.     

径向函数网络对国民经济生产总值预测研究

为了建立国民经济生产总值（GDP）神经网络预测模型,构造了双层网络结构的基于径向基网络,通过学习训练,确定径向基数神经网络参数和结构。仿真结果表明,生成的径向基函数模型应用于国民经济预测比BP神经网络模型具有更高的预测精度和良好的泛化能力。     

基于信息论方法的分类数据相似性度量

两实例的距离或相似性度量在数据挖掘和机器学习中扮演着重要的角色。常用的距离度量方法主要适用于数值数据,针对分类数据,本文提出一种数据驱动的相似性度量方法。该方法利用属性值与类标签的信息,将属性值的类条件概率结合信息论来度量分类数据的相似性。为了与已提出的相似性度量方法作比较,把各度量方法与k最近邻算法结合,对多个分类数据集进行分类,通过十折交叉验证比较结果的错误率。实验表明该度量结合k最近邻方法使分类具有较低的错误率。     

基于神经网络的分类决策树构造

目前基于符号处理的方法是解决分类规则提取问题的主要方法,而基于神经网络的连接主义方法则用的不多,其主要原因在于虽然神经网络的分类精度高,但难于提取其所隐含的分类规则与知识.针对这个问题,结合神经网络的具体特点,该文提出了一种基于神经网络的构造分类决策树的新方法.该方法通过神经网络训练建立各属性与分类结果之间的关系,进而通过提取各属性与分类结果之间的导数关系来建立分类决策树.给出了具体的决策树构造算法.同时为了提高神经网络所隐含关系的提取效果,提出了关系强化约束的概念并建立了具体的模型.实际应用结果证明了算法的有效性.     

融入Universum学习的度量学习算法

度量学习通过更真实的刻画样本之间的距离,来提高分类和聚类的精度。GMML（Geometric Mean Metric Learning）在学习度量矩阵[A]时,使得在该度量下同类点之间的距离尽可能小,不同类点之间的距离尽可能大。GMML用来学习的训练样本均为目标类数据,而对于现实存在的为数众多的同领域非目标类数据,即Universum数据并未加以利用,不免造成信息的浪费,针对此,提出一种新的度量学习算法--融入Universum学习的GMML（U-GMML）。U-GMML期望得到一个新的度量矩阵[A],使得同类点之间的距离尽可能小,不同类点之间的距离尽可能大,且Universum数据与目标类数据的距离尽可能大,从而使得所学习的度量矩阵[A]更有利于分类。真实数据集上的实验结果验证了该算法的有效性。     

一种基于卡尔曼滤波和模糊控制的RBF神经网络新型学习算法

提出了基于Kalman滤波最优估计和模糊控制的径向基函数（Radical Basis Function, RBF）神经网络学习算法,用实例进行了仿真实验。结果表明,与传统的RBF网络学习算法比较,该算法具有明显快速的学习效率和较高的识别精度.     

基于数据挖掘的符号序列聚类相似度量模型

为了从消费者偏好序列中发现市场细分结构,采用数据挖掘领域中的符号序列聚类方法,提出一种符号序列聚类的研究方法和框架,给出RSM相似性度量模型。调整RSM模型参数,使得RSM可以变为与编辑距离、海明距离等价的相似性度量。通过RSM与其他序列相似性度量的比较,表明RSM具有更强的表达相似性概念的能力。由于RSM能够表达不同的相似性概念,从而使之能适用于不同的应用环境,并在其基础上提出自组织特征映射退火符号聚类模型,使得从消费者偏好进行市场细分结构研究的研究途径在实际应用中得以实现。