SVM在多源遥感图像分类中的应用研究

在利用遥感图像进行土地利用/覆盖分类过程中,可采用以下两种途径来提高分类精度:一是通过增加有利于分类的数据源,引入地理辅助数据和归一化植被指数(NDVI)来进行多源信息融合;二是选择更好的分类方法,例如支持向量机(SVM)学习方法,由于该方法克服了最大似然法和神经网络的弱点,非常适合高维、复杂的小样本多源数据的分类。为了提高多源遥感图像分类的精度,还研究了支持向量机在遥感图像分类中模型的选择,包括多类模型和核函数的选择。分类结果表明,支持向量机比传统的分类方法具有更高的精度,尤其是基于径向基核函数和一对一多类方法的支持向量机模型更适合多源遥感图像分类,因此,基于支持向量机的多源土地利用/覆盖分类能大大提高分类精度。     

A hybrid model through the fusion of type-2 fuzzy logic systems and extreme learning machines for modelling permeability prediction

Extreme learning machines (ELM), as a learning tool, have gained popularity due to its unique characteristics and performance. However, the generalisation capability of ELM often depends on the nature of the dataset, particularly on whether uncertainty is present in the dataset or not. In order to reduce the effects of uncertainties in ELM prediction and improve its generalisation ability, this paper proposes a hybrid system through a combination of type-2 fuzzy logic systems (type-2 FLS) and ELM; thereafter the hybrid system was applied to model permeability of carbonate reservoir. Type-2 FLS has been chosen to be a precursor to ELM in order to better handle uncertainties existing in datasets beyond the capability of type-1 fuzzy logic systems. The type-2 FLS is used to first handle uncertainties in reservoir data so that its final output is then passed to the ELM for training and then final prediction is done using the unseen testing dataset. Comparative studies have been carried out to compare the performance of the proposed T2-ELM hybrid system with each of the constituent type-2 FLS and ELM, and also artificial neural network (ANN) and support Vector machines (SVM) using five different industrial reservoir data. Empirical results show that the proposed T2-ELM hybrid system outperformed each of type-2 FLS and ELM, as the two constituent models, in all cases, with the improvement made to the ELM performance far higher against that of type-2 FLS that had a closer performance to the hybrid since it is already noted for being able to model uncertainties. The proposed hybrid also outperformed ANN and SVM models considered.     

MPEG VBR video traffic modeling and classification using fuzzytechnique

We present an approach for MPEG variable bit rate (VBR) video modeling and classification using fuzzy techniques. We demonstrate that a type-2 fuzzy membership function, i.e., a Gaussian MF with uncertain variance, is most appropriate to model the log-value of I/P/B frame sizes in MPEG VBR video. The fuzzy c-means (FCM) method is used to obtain the mean and standard deviation (std) of T/P/B frame sizes when the frame category is unknown. We propose to use type-2 fuzzy logic classifiers (FLCs) to classify video traffic using compressed data. Five fuzzy classifiers and a Bayesian classifier are designed for video traffic classification, and the fuzzy classifiers are compared against the Bayesian classifier. Simulation results show that a type-2 fuzzy classifier in which the input is modeled as a type-2 fuzzy set and antecedent membership functions are modeled as type-2 fuzzy sets performs the best of the five classifiers when the testing video product is not included in the training products and a steepest descent algorithm is used to tune its parameters     

Support vector learning for fuzzy rule-based classification systems

To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, the support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet a certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.     

Polynomial regression interval-valued fuzzy systems

In recent years, the type-2 fuzzy sets theory has been used to model and minimize the effects of uncertainties in rule-base fuzzy logic system (FLS). In order to make the type-2 FLS reasonable and reliable, a new simple and novel statistical method to decide interval-valued fuzzy membership functions and probability type reduce reasoning method for the interval-valued FLS are developed. We have implemented the proposed non-linear (polynomial regression) statistical interval-valued type-2 FLS to perform smart washing machine control. The results show that our quadratic statistical method is more robust to design a reliable type-2 FLS and also can be extend to polynomial model.     

基于半监督模糊C均值算法的遥感影像分类

遥感影像数据因其固有的不确定性与复杂性,导致传统的无监督分类算法难以对其准确建模。基于模糊集理论的模式识别方法可以有效地表达数据的模糊性,其中二型模糊集能更好地刻画类间多重不确定性,而半监督法可以利用少量先验知识来解决算法对数据的泛化性问题,因此提出一种基于半监督的自适应区间二型模糊C均值遥感影像分类方法（SS-AIT2FCM）。首先,结合半监督和进化论思想,提出一种新的模糊权重指数选取方法,以提升自适应区间二型模糊C均值聚类算法的鲁棒性与泛化性,使算法更适用于光谱混叠严重、覆盖面积大、地物丰富的遥感数据分类;然后,通过对少量标记样本的软约束监督,对区间二型模糊算法迭代过程进行优化指导,来挖掘数据的最优表达。实验选用了北京颐和园区域的SPOT5多光谱遥感影像数据和广东横琴岛区域的Landsat TM多光谱遥感影像数据,对现有流行的模糊分类算法和SS-AIT2FCM的分类结果进行了比较。结果表明,SS-AIT2FCM获得了更高的分类精度与更清晰的类别边界,且有较好数据泛化能力。     

On classification of signals represented with data-dependent overcomplete dictionaries

This paper focuses on the problem of how data representation influences the generalization error of kernel-based learning machines like support vector machines (SVMs). We analyse the effects of sparse and dense data representations on the generalization error of SVM. We show that using sparse representations the performances of classifiers belonging to hypothesis spaces induced by polynomial or Gaussian kernel functions reduce to the performances of linear classifiers. Sparse representations reduce the generalization error as long as the representation is not too sparse as with very large dictionaries. Dense data representations reduce the generalization error also using very large dictionaries. We use two schemes for representing data in data-independent overcomplete Haar and Gabor dictionaries, and measure the generalization error of SVMs on benchmark datasets. We study sparse and dense representations in the case of data-dependent overcomplete dictionaries and we show how this leads to principal component analysis.     

Constructing support vector machine ensemble

Even the support vector machine (SVM) has been proposed to provide a good generalization performance, the classification result of the practically implemented SVM is often far from the theoretically expected level because their implementations are based on the approximated algorithms due to the high complexity of time and space. To improve the limited classification performance of the real SVM, we propose to use the SVM ensemble with bagging (bootstrap aggregating) or boosting. In bagging, each individual SVM is trained independently using the randomly chosen training samples via a bootstrap technique. In boosting, each individual SVM is trained using the training samples chosen according to the sample's probability distribution that is updated in proportional to the errorness of the sample. In both bagging and boosting, the trained individual SVMs are aggregated to make a collective decision in several ways such as the majority voting, least-squares estimation-based weighting, and the double-layer hierarchical combining. Various simulation results for the IRIS data classification and the hand-written digit recognition, and the fraud detection show that the proposed SVM ensemble with bagging or boosting outperforms a single SVM in terms of classification accuracy greatly.     

A Dempster Shafer-based fuzzy multisensor fusion system using airborne lidar and hyperspectral imagery

ABSTRACT

Nowadays, accurate spectral reflectance information is provided by hyperspectral (HS) data while light detection and ranging (lidar) data provides precise information about the height and geometrical properties of the surfaces. In the most research papers, data fusion of disparate sensors significantly improves object classification performance compared to that of just an individual sensor. Previous researches on fusion of these two sensors had problems such as crisp classifiers or simple fuzzy decision-making systems. This article tries to overcome these weaknesses by accurate support vector machine (SVM) and Fuzzy SVM as classifiers in crisp and fuzzy decision fusion system and fusion of two sensors by two different methods based on precise theories of Bayesian and Shafer. Also, the proposed method tries to compare the results of fusion of both data using decision fusion system with stacked features strategy. This study focuses on HS and lidar fusion through three main phases. The first phase is based on the using of Noise Weighted Harsanyi-Farrand-Chang method and principal component analysis to overcome the high dimensionality problem of HS data. The second phase is based on the feature extraction and selection strategy on lidar data. Finally, fuzzy SVM and Dempster Shafer methods are applied as fuzzy classification and fuzzy decision fusion strategies on the feature spaces. A co-registered HS and lidar data set from Houston of U.S.A. by 15 classes was available to examine the effectiveness of the proposed method. The results of this study highlight that the combination of HS and lidar data enable reliable mapping of land cover.     

基于类内超平面的模糊支持向量机

分析基于样本与类中心距离设计模糊支持向量机隶属度函数的缺点,使用类内超平面代替类中心,提出基于样本到超平面距离的隶属度函数设计方法.该方法降低隶属度函数对样本集几何形状的依赖,提高模糊支持向量机的泛化能力.最后数值实验表明,与传统的支持向量机和现有的3种不同隶属度函数的模糊支持向量机相比,新隶属度函数可达到最好的分类效果而且速度快.     

Wireless Sensor Network Lifetime Analysis Using Interval Type-2 Fuzzy Logic Systems

Extending the lifetime of the energy constrained wireless sensor networks is a crucial challenge in sensor network research. In this paper, we present a novel approach based on fuzzy logic systems to analyze the lifetime of a wireless sensor network. We demonstrate that a type-2 fuzzy membership function (MF), i.e., a Gaussian MF with uncertain standard deviation (std) is most appropriate to model a single node lifetime in wireless sensor networks. In our research, we study two basic sensor placement schemes: square-grid and hex-grid. Two fuzzy logic systems (FLSs): a singleton type-1 FLS and an interval type-2 FLS are designed to perform lifetime estimation of the sensor network. We compare our fuzzy approach with other nonfuzzy schemes in previous papers. Simulation results show that FLS offers a feasible method to analyze and estimate the sensor network lifetime and the interval type-2 FLS in which the antecedent and the consequent membership functions are modeled as Gaussian with uncertain std outperforms the singleton type-1 FLS and the nonfuzzy schemes.     

A New Fuzzy Support Vector Machine Based on the Weighted Margin

The ideas from fuzzy neural networks and support vector machine (SVM) are incorporated to make SVM classifiers perform better. The influence of the samples with high uncertainty can be decreased by employing the fuzzy membership to weigh the margin of each training vector. The linear separability, fuzzy margin, optimal hyperplane, generalization and soft fuzzy margin algorithms are discussed. A new optimization problem is obtained and SVM is then completely reformulated into a new fuzzy support vector machine (NFSVM). Moreover, the generation bound of NFSVM can be described. We also introduce the membership function in fuzzy neural networks to do some experiments. The results demonstrate that the proposed NFSVM can produce better results than regular SVM and Fuzzy Kernel Perceptron (FKP) in some real cases.     

基于图卷积神经网络的自注意力的融合节点分类框架

图神经网络因其强大的建模能力引起广泛关注, 常常被用来解决图上的节点分类任务. 现阶段常用的以图卷积神经网络 (graph convolutional network, GCN)为内核的模型解决此类问题, 但往往因为出现过拟合与过平滑而导致深层的节点嵌入表示效果并不好. 因此, 本文提出了一种基于GCN内核的结合残差连接与自注意力方法——GCNRN模型, 以提升GCN的泛化能力. 同时, 为了整合更深入的信息, 本文引入融合机制, 采用模糊积分融合多个分类器, 最终提高模型测试精度. 为了验证所提出方法的优越性, 本文采用ogbn-arxiv与常用的引文数据集进行了对比实验. GCNRN模型与多个以GCN为内核的现有模型相比, 节点分类准确率平均提高了2%, 且避免了传统的过拟合和过平滑现象. 此外, 实验结果表明, 增加了基于模糊积分的融合模块的多分类器模型比传统融合方法具有更好的分类效果.     

Comparison of artificial neural networks and support vector machine classifiers for land cover classification in Northern China using a SPOT-5 HRG image

This article presents a sufficient comparison of two types of advanced non-parametric classifiers implemented in remote sensing for land cover classification. A SPOT-5 HRG image of Yanqing County, Beijing, China, was used, in which agriculture and forest dominate land use. Artificial neural networks (ANNs), including the adaptive backpropagation (ABP) algorithm, Levenberg–Marquardt (LM) algorithm, Quasi-Newton (QN) algorithm and radial basis function (RBF) were carefully tested. The LM–ANN and RBF–ANN, which outperform the other two, were selected to make a detailed comparison with support vector machines (SVMs). The experiments show that those well-trained ANNs and SVMs have no significant difference in classification accuracy, but the SVM usually performs slightly better. Analysis of the effect of the training set size highlights that the SVM classifier has great tolerance on a small training set and avoids the problem of insufficient training of ANN classifiers. The testing also illustrates that the ANNs and SVMs can vary greatly with regard to training time. The LM–ANN can converge very quickly but not in a stable manner. By contrast, the training of RBF–ANN and SVM classifiers is fast and can be repeatable.     

Pattern classification with mixtures of weighted least-squares support vector machine experts

Support Vector Machine (SVM) classifiers are high-performance classification models devised to comply with the structural risk minimization principle and to properly exploit the kernel artifice of nonlinearly mapping input data into high-dimensional feature spaces toward the automatic construction of better discriminating linear decision boundaries. Among several SVM variants, Least-Squares SVMs (LS-SVMs) have gained increased attention recently due mainly to their computationally attractive properties coming as the direct result of applying a modified formulation that makes use of a sum-squared-error cost function jointly with equality, instead of inequality, constraints. In this work, we present a flexible hybrid approach aimed at augmenting the proficiency of LS-SVM classifiers with regard to accuracy/generalization as well as to hyperparameter calibration issues. Such approach, named as Mixtures of Weighted Least-Squares Support Vector Machine Experts, centers around the fusion of the weighted variant of LS-SVMs with Mixtures of Experts models. After the formal characterization of the novel learning framework, simulation results obtained with respect to both binary and multiclass pattern classification problems are reported, ratifying the suitability of the novel hybrid approach in improving the performance issues considered.     

OAO-SVMs的训练时间性能分析及算法改进

支持向量机(SVM)算法是统计学习理论中最年轻的分支。结构风险最小化原则使其具有良好的学习推广性。但在实际应用中,训练速度慢一直是支持向量机理论几个亟待解决的问题之一,这一点在SVM向多类问题领域推广时表现的尤为明显。文中将从样本分布与类别数量两方面入手,对传统的SVM多分类OAO算法进行训练时间性能上的分析,并引入分层的思想,提出传统OAO-SVMs算法的改进模型H-OAO-SVMs。通过与其他常见多分类SVMs训练时间的比较表明:改进后的H-OAO-SVMs模型具有更优的训练时间性能。     

Improving the SVM gender classification accuracy using clustering and incremental learning

Gender recognition has been playing a very important role in various applications such as human–computer interaction, surveillance, and security. Nonlinear support vector machines (SVMs) were investigated for the identification of gender using the Face Recognition Technology (FERET) image face database. It was shown that SVM classifiers outperform the traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, and nearest neighbour). In this context, this paper aims to improve the SVM classification accuracy in the gender classification system and propose new models for a better performance. We have evaluated different SVM learning algorithms; the SVM‐radial basis function with a 5% outlier fraction outperformed other SVM classifiers. We have examined the effectiveness of different feature selection methods. AdaBoost performs better than the other feature selection methods in selecting the most discriminating features. We have proposed two classification methods that focus on training subsets of images among the training images. Method 1 combines the outcome of different classifiers based on different image subsets, whereas method 2 is based on clustering the training data and building a classifier for each cluster. Experimental results showed that both methods have increased the classification accuracy.     

核对齐多核模糊支持向量机

支持向量机（SVMs）是当前被广泛使用的机器学习技术,其通过最优分割超平面来提高分类器的泛化能力,在实际应用中表现优异。然而SVM也存在易受噪声影响,以及核函数选择等难题。针对以上问题,本文将基于核对齐的多核学习方法引入到模糊支持向量机（fuzzy support vector machine, FSVM）中,提出了模糊多核支持向量机模型（multiple kernel fuzzy support vector machine,MFSVM）。MFSVM通过模糊粗糙集方法计算每一样例隶属度;其次,利用核对齐的多核方法计算每一单核权重,并将组合核引入到模糊支持向量机中。该方法不仅提高了支持向量机的抗噪声能力,也有效避免了核选择难题。在UCI数据库上进行实验,结果表明本文所提方法具有较高的分类精度,验证了该方法的可行性与有效性。     

Equalization of nonlinear time-varying channels using type-2 fuzzyadaptive filters

Presents a kind of adaptive filter: type-2 fuzzy adaptive filter (FAF); one that is realized using an unnormalized type-2 Takagi-Sugeno-Kang (TSK) fuzzy logic system (FLS). We apply this filter to equalization of a nonlinear time-varying channel and demonstrate that it can implement the Bayesian equalizer for such a channel, has a simple structure, and provides fast inference. A clustering method is used to adaptively design the parameters of the FAF. Two structures are used for the equalizer: transversal equalizer (TE) and decision feedback equalizer (DFE). A decision tree structure is used to implement the decision feedback equalizer, in which each leaf of the tree is a type-2 FAF. This DFE vastly reduces computational complexity as compared to a TE. Simulation results show that equalizers based on type-2 FAFs perform much better than nearest neighbor classifiers (NNC) or equalizers based on type-1 FAFs     

Empirical analysis of support vector machine ensemble classifiers

Ensemble classification – combining the results of a set of base learners – has received much attention in the machine learning community and has demonstrated promising capabilities in improving classification accuracy. Compared with neural network or decision tree ensembles, there is no comprehensive empirical research in support vector machine (SVM) ensembles. To fill this void, this paper analyses and compares SVM ensembles with four different ensemble constructing techniques, namely bagging, AdaBoost, Arc-X4 and a modified AdaBoost. Twenty real-world data sets from the UCI repository are used as benchmarks to evaluate and compare the performance of these SVM ensemble classifiers by their classification accuracy. Different kernel functions and different numbers of base SVM learners are tested in the ensembles. The experimental results show that although SVM ensembles are not always better than a single SVM, the SVM bagged ensemble performs as well or better than other methods with a relatively higher generality, particularly SVMs with a polynomial kernel function. Finally, an industrial case study of gear defect detection is conducted to validate the empirical analysis results.