General fuzzy min-max neural network for clustering andclassification

This paper describes a general fuzzy min-max (GFMM) neural network which is a generalization and extension of the fuzzy min-max clustering and classification algorithms of Simpson (1992, 1993). The GFMM method combines supervised and unsupervised learning in a single training algorithm. The fusion of clustering and classification resulted in an algorithm that can be used as pure clustering, pure classification, or hybrid clustering classification. It exhibits a property of finding decision boundaries between classes while clustering patterns that cannot be said to belong to any of existing classes. Similarly to the original algorithms, the hyperbox fuzzy sets are used as a representation of clusters and classes. Learning is usually completed in a few passes and consists of placing and adjusting the hyperboxes in the pattern space; this is an expansion-contraction process. The classification results can be crisp or fuzzy. New data can be included without the need for retraining. While retaining all the interesting features of the original algorithms, a number of modifications to their definition have been made in order to accommodate fuzzy input patterns in the form of lower and upper bounds, combine the supervised and unsupervised learning, and improve the effectiveness of operations. A detailed account of the GFMM neural network, its comparison with the Simpson's fuzzy min-max neural networks, a set of examples, and an application to the leakage detection and identification in water distribution systems are given     

一种SOM和GRNN结合的模式全自动分类新方法

非监督学习算法的分类精度通常很难令人满意,而监督的学习算法需要人工选取训练样本,这有时很难得到,并且其分类精度直接依赖于所选取的学习样本。针对这些缺陷,提出一种非监督自组织神经网络(SOMNN)和监督的广义回归网络(GRNN)结合的全自动模式分类新方法。新方法首先通过SOMNN将原始数据进行自动聚类,再用所得的聚类中心以及中心邻近数据点训练GRNN,然后根据GRNN的分类结果重新计算聚类中心,再根据新的聚类中心和中心邻近点训练GRNN,如此反复,直至得到稳定的中心为止。Iris数据,Wine数据的实验结果都验证了新方法的可行性。     

Comments on “A fuzzy neural network and its application topattern recognition”

This note analyzes the unsupervised fuzzy neural network (FNNU) of the original paper by Kwan and Cai (ibid., vol.2, p.185-93, 1994) and finds the following: the FNNU is a clustering net, not a classifier net, and the number of clusters the network settles to may be less or more than the actual number of pattern classes (sometimes it could even be equal to the number of training data points); the huge number of connections in the FNNU can be drastically reduced without degrading its performance; and the algorithm does not have any learning capability for its parameters. Computational experience shows that usually the performance of a multilayer perceptron (MLP) is comparable to that of even a supervised version of FNN (trained by gradient descent algorithm) in terms of recognition scores, but an MLP has a much faster convergence than the supervised version of FNN     

Supervised extended ART: A fast neural network classifier trained by combining supervised and unsupervised learning

A neural network classifier, called supervised extended ART (SEART), that incorporates a supervised mechanism into the extended unsupervised ART is presented here. It uses a learning theory called Nested Generalized Exemplar (NGE) theory. In any time, the training instances may or may not have desired outputs, that is, this model can handle supervised learning and unsupervised learning simultaneously. The unsupervised component finds the cluster relations of instances, and the supervised component learns the desired associations between clusters and classes. In addition, this model has the ability of incremental learning. It works equally well when instances in a cluster belong to different classes. Also, multi-category and nonconvex classifications can be dealt with. Besides, the experimental results are very encouraging.     

基于Kmeans与SVM结合的遥感图像全自动分类方法*

遥感图像分类方法通常采用监督的学习算法,它需要人工选取训练样本,比较繁琐,而且有时很难得到;而非监督学习算法的分类精度通常很难令人满意.针对这些缺陷,提出一种基于K-means与支持向量机(SVM)结合的遥感图像全自动分类方法.首先使用K-means聚类算法对样本进行初始聚类,根据每类中样本数及其稀疏程度选取一些点作为标记的学习样本训练SVM分类器,然后用SVM对原始数据重新分类.Iris数据和遥感数据的实验结果均验证了新方法的有效性.     

Noise reduction and feature extraction based on low-rank representation and pairwise constraint preserving for hyperspectral images

ABSTRACT

Feature extraction (FE) methods play a central role in the classification of hyperspectral images (HSIs). However, all traditional FE methods work in original feature space (OFS), OFS may suffer from noise, outliers and poorly discriminative features. This paper presents a feature space enriching technique to address the problems of noise, outliers and poorly discriminative features which may exist in OFS. The proposed method is based on low-rank representation (LRR) with the capability of pairwise constraint preserving (PCP) termed LRR-PCP. LRR-PCP does not change the dimension of OFS and only can be used as an appropriate preprocessing procedure for any classification algorithm or DR methods. The proposed LRR-PCP aims to enrich the OFS and obtain extracted feature space (EFS) which results in features richer than OFS. The problems of noise and outliers can be decreased using LRR. But, LRR cannot preserve the intrinsic local structure of the original data and only capture the global structure of data. Therefore, two additional penalty terms are added into the objective function of LRR to keep the local discriminative ability and also preserve the data diversity. LRR-PCP method not only can be used in supervised learning but also in unsupervised and semi-supervised learning frameworks. The effectiveness of LRR-PCP is investigated on three HSI data sets using some existing DR methods and as a denoising procedure before the classification task. All experimental results and quantitative analysis demonstrate that applying LRR-PCP on OFS improves the performance of the classification and DR methods in supervised, unsupervised, and semi-supervised conditions.     

半监督中心最大化模糊C均值算法

在模式识别领域内,对于数据的分析方法一般分为:有监督的学习方法及无监督的学习方法。而这两类方法均与实际应用不符,一般生产所获之数据既不可能毫无信息可知又不可能全部信息已知。此外,由于实际生产的干扰因素过多导致所获之数据样本信息通常包含一些干扰信息,这些数据对传统的分析方法影响较大,其中尤以聚类方法最为敏感。针对以上两大问题,以经典的无监督聚类算法FCM算法为基础,通过引入半监督性质的隶属度补偿项以及减弱干扰点影响的中心最大化项构造出了新的聚类算法称之为半监督中心最大化模糊C均值算法,简称SCM-FCM。通过在UCI数据集上的仿真实验结果表明该算法较之于传统的无监督聚类分析方法有着更好的应用价值。     

Data Clustering with Partial Supervision

Clustering with partial supervision finds its application in situations where data is neither entirely nor accurately labeled. This paper discusses a semi-supervised clustering algorithm based on a modified version of the fuzzy C-Means (FCM) algorithm. The objective function of the proposed algorithm consists of two components. The first concerns traditional unsupervised clustering while the second tracks the relationship between classes (available labels) and the clusters generated by the first component. The balance between the two components is tuned by a scaling factor. Comprehensive experimental studies are presented. First, the discrimination of the proposed algorithm is discussed before its reformulation as a classifier is addressed. The induced classifier is evaluated on completely labeled data and validated by comparison against some fully supervised classifiers, namely support vector machines and neural networks. This classifier is then evaluated and compared against three semi-supervised algorithms in the context of learning from partly labeled data. In addition, the behavior of the algorithm is discussed and the relation between classes and clusters is investigated using a linear regression model. Finally, the complexity of the algorithm is briefly discussed.     

非一致性引导的无监督特征选择

由于无监督环境下特征选择缺少类别信息的依赖,所以利用模糊粗糙集理论提出一种非一致性度量方法DAM(disagreement measure),用于度量任意两个特征集合或特征间引起的模糊等价类含义的差异程度.在此基础上实现DAMUFS无监督特征选择算法,其在无监督条件下可以选择出包含更多信息量的特征子集,同时还保证特征子集中属性冗余度尽可能小.实验将DAMUFS算法与一些无监督以及有监督特征选择算法在多个数据集上进行分类性能比较,结果证明了DAMUFS的有效性.     

弱监督分层深度学习的车辆识别算法

针对已有分类器在结构形式和训练方法的不足,构建了一个以二维深度置信网络（2D deep belief networks,2D DBN）为架构的弱监督分层深度学习车辆识别算法。首先,将传统一维的深度置信网络(Deep belief networks,DBN)扩展成2D-DBN,并构建相应分类器结构,从而能够直接以二维图像像素矩阵作为输入; 其次,在传统无监督训练的目标函数中,引入了一个具有适当权重的判别度正则化项,将原有无监督训练转化为带有较弱监督性的弱监督训练方式,从而使提取的特征较传统无监督特征更具判别性。多组对比实验表明,本文所提算法在识别率等指标上要优于已有深度学习算法。     

An evaluation of dimension reduction techniques for one-class classification

Dimension reduction (DR) is important in the processing of data in domains such as multimedia or bioinformatics because such data can be of very high dimension. Dimension reduction in a supervised learning context is a well posed problem in that there is a clear objective of discovering a reduced representation of the data where the classes are well separated. By contrast DR in an unsupervised context is ill posed in that the overall objective is less clear. Nevertheless successful unsupervised DR techniques such as principal component analysis (PCA) exist—PCA has the pragmatic objective of transforming the data into a reduced number of dimensions that still captures most of the variation in the data. While one-class classification falls somewhere between the supervised and unsupervised learning categories, supervised DR techniques appear not to be applicable at all for one-class classification because of the absence of a second class label in the training data. In this paper we evaluate the use of a number of up-to-date unsupervised DR techniques for one-class classification and we show that techniques based on cluster coherence and locality preservation are effective.     

融合无监督和监督学习策略生成的多分类决策树

提出了一种融合无监督和监督两种学习策略生成多分类决策树的方法．它首先利用无监督聚类方法能够发现待分类样本之问的内在联系和规律的特点，确定出最为符合多类样本分布特征的决策树的树型，继而利用监督学习支持向量机的方法对样本进行准确的分类，通过采用核函数和不对称的Lagrangian系数限制条件，支持向量机很好的解决了样本特征空间上的线性不可分性和决策树型确定过程中出现的训练样本不对称性的影响、该方法具有较高的计算效率和准确性，在实验申取得了比较好的结果．     

基于邻域迭代最优化算法的网络故障分析*

迭代最优化算法是模式识别中重要的无指导学习方法。算法因随机确定k个聚类中心进行初始划分的原因,存在初始聚类中心选择的盲目性、容易陷入局部极值、忽略样本的聚类趋势等缺点。经过对迭代最优化算法的研究与分析,根据样本的聚类趋势,结合邻域思想,改进了聚类中心的选择方法,设计了基于样本邻域概念的迭代最优化算法,算法总的时间代价为O(n)。该算法已应用于基于SNMP协议的网络故障管理中的故障分析,分析结果与实际故障类型基本一致,并为计算机网络故障分析提供了一种可行的分析方法。     

A new hybrid feature selection approach using feature association map for supervised and unsupervised classification

Feature selection, both for supervised as well as for unsupervised classification is a relevant problem pursued by researchers for decades. There are multiple benchmark algorithms based on filter, wrapper and hybrid methods. These algorithms adopt different techniques which vary from traditional search-based techniques to more advanced nature inspired algorithm based techniques. In this paper, a hybrid feature selection algorithm using graph-based technique has been proposed. The proposed algorithm has used the concept of Feature Association Map (FAM) as an underlying foundation. It has used graph-theoretic principles of minimal vertex cover and maximal independent set to derive feature subset. This algorithm applies to both supervised and unsupervised classification. The performance of the proposed algorithm has been compared with several benchmark supervised and unsupervised feature selection algorithms and found to be better than them. Also, the proposed algorithm is less computationally expensive and hence has taken less execution time for the publicly available datasets used in the experiments, which include high-dimensional datasets.     

Scale selection for supervised image segmentation

Finding the right scales for feature extraction is crucial for supervised image segmentation based on pixel classification. There are many scale selection methods in the literature; among them the one proposed by Lindeberg is widely used for image structures such as blobs, edges and ridges. Those schemes are usually unsupervised, as they do not take into account the actual segmentation problem at hand. In this paper, we consider the problem of selecting scales, which aims at an optimal discrimination between user-defined classes in the segmentation. We show the deficiency of the classical unsupervised scale selection paradigms and present a supervised alternative. In particular, the so-called max rule is proposed, which selects a scale for each pixel to have the largest confidence in the classification across the scales. In interpreting the classifier as a complex image filter, we can relate our approach back to Lindeberg's original proposal. In the experiments, the max rule is applied to artificial and real-world image segmentation tasks, which is shown to choose the right scales for different problems and lead to better segmentation results.     

Unsupervised probabilistic feature selection using ant colony optimization

Feature selection (FS) is one of the most important fields in pattern recognition, which aims to pick a subset of relevant and informative features from an original feature set. There are two kinds of FS algorithms depending on the presence of information about dataset class labels: supervised and unsupervised algorithms. Supervised approaches utilize class labels of dataset in the process of feature selection. On the other hand, unsupervised algorithms act in the absence of class labels, which makes their process more difficult. In this paper, we propose unsupervised probabilistic feature selection using ant colony optimization (UPFS). The algorithm looks for the optimal feature subset in an iterative process. In this algorithm, we utilize inter-feature information which shows the similarity between the features that leads the algorithm to decreased redundancy in the final set. In each step of the ACO algorithm, to select the next potential feature, we calculate the amount of redundancy between current feature and all those which have been selected thus far. In addition, we utilize a matrix to hold ant related pheromone which shows the rate of the co-presence of every pair of features in solutions. Afterwards, features are ranked based on a probability function extracted from the matrix; then, their m-top is returned as the final solution. We compare the performance of UPFS with 15 well-known supervised and unsupervised feature selection methods using different classifiers (support vector machine, naive Bayes, and k-nearest neighbor) on 10 well-known datasets. The experimental results show the efficiency of the proposed method compared to the previous related methods.     

Unsupervised query-based learning of neural networks usingselective-attention and self-regulation

Query-based learning (QBL) has been introduced for training a supervised network model with additional queried samples. Experiments demonstrated that the classification accuracy is further increased. Although QBL has been successfully applied to supervised neural networks, it is not suitable for unsupervised learning models without external supervisors. In this paper, an unsupervised QBL (UQBL) algorithm using selective-attention and self-regulation is proposed. Applying the selective-attention, we can ask the network to respond to its goal-directed behavior with self-focus. Since there is no supervisor to verify the self-focus, a compromise is then made to environment-focus with self-regulation. In this paper, we introduce UQBL1 and UQBL2 as two versions of UQBL; both of them can provide fast convergence. Our experiments indicate that the proposed methods are more insensitive to network initialization. They have better generalization performance and can be a significant reduction in their training size.     

一种半监督判别邻域嵌入算法

邻域保持嵌入（Neighborhood Preserving Embedding,NPE）,作为局部线性嵌入（Locally Linear Embedding,LLE）的线性化版本,由于在映射前后保持了数据的局部几何结构并得到了原始数据的子空间描述,在模式识别领域具有较强的应用价值。但作为非监督处理算法,在具体的模式分类中有一定局限性,提出一种NPE的改进算法——半监督判别邻域嵌入（SSDNE）算法,引入标记后样本点的类别信息,并在正则项中引入样本的流形结构,最大化标记样本点的类间信息和类内信息。既增加了算法的辨别能力又减少了监督算法中对样本点进行全标记的工作量。在ORL和YaleB人脸库上的实验结果表明,改进的算法较PCA、LDA、LPP以及原保持近邻判别嵌入算法的识别性能有了较明显的改善。