Classification of hyperspectral remote sensing images with support vector machines

This paper addresses the problem of the classification of hyperspectral remote sensing images by support vector machines (SVMs). First, we propose a theoretical discussion and experimental analysis aimed at understanding and assessing the potentialities of SVM classifiers in hyperdimensional feature spaces. Then, we assess the effectiveness of SVMs with respect to conventional feature-reduction-based approaches and their performances in hypersubspaces of various dimensionalities. To sustain such an analysis, the performances of SVMs are compared with those of two other nonparametric classifiers (i.e., radial basis function neural networks and the K-nearest neighbor classifier). Finally, we study the potentially critical issue of applying binary SVMs to multiclass problems in hyperspectral data. In particular, four different multiclass strategies are analyzed and compared: the one-against-all, the one-against-one, and two hierarchical tree-based strategies. Different performance indicators have been used to support our experimental studies in a detailed and accurate way, i.e., the classification accuracy, the computational time, the stability to parameter setting, and the complexity of the multiclass architecture. The results obtained on a real Airborne Visible/Infrared Imaging Spectroradiometer hyperspectral dataset allow to conclude that, whatever the multiclass strategy adopted, SVMs are a valid and effective alternative to conventional pattern recognition approaches (feature-reduction procedures combined with a classification method) for the classification of hyperspectral remote sensing data.     

分类器组合技术的多样性研究

分类器组合技术可以提高模式识别的性能,受到了模式识别领域研究人员的广泛关注。实现成员分类器的多样性是提高分类器组合泛化能力主要手段。本文从成员分类器的生成介绍了实现成员分类器多样性的各种方法,同时介绍了度量成员分类器多样性的各种技术,并提出了一种如何训练多样性成员分类器的技术思路。     

Performance-based classifier combination in atlas-based image segmentation using expectation-maximization parameter estimation

It is well known in the pattern recognition community that the accuracy of classifications obtained by combining decisions made by independent classifiers can be substantially higher than the accuracy of the individual classifiers. We have previously shown this to be true for atlas-based segmentation of biomedical images. The conventional method for combining individual classifiers weights each classifier equally (vote or sum rule fusion). In this paper, we propose two methods that estimate the performances of the individual classifiers and combine the individual classifiers by weighting them according to their estimated performance. The two methods are multiclass extensions of an expectation-maximization (EM) algorithm for ground truth estimation of binary classification based on decisions of multiple experts (Warfield et al., 2004). The first method performs parameter estimation independently for each class with a subsequent integration step. The second method considers all classes simultaneously. We demonstrate the efficacy of these performance-based fusion methods by applying them to atlas-based segmentations of three-dimensional confocal microscopy images of bee brains. In atlas-based image segmentation, multiple classifiers arise naturally by applying different registration methods to the same atlas, or the same registration method to different atlases, or both. We perform a validation study designed to quantify the success of classifier combination methods in atlas-based segmentation. By applying random deformations, a given ground truth atlas is transformed into multiple segmentations that could result from imperfect registrations of an image to multiple atlas images. In a second evaluation study, multiple actual atlas-based segmentations are combined and their accuracies computed by comparing them to a manual segmentation. We demonstrate in both evaluation studies that segmentations produced by combining multiple individual registration-based segmentations are more accurate for the two classifier fusion methods we propose, which weight the individual classifiers according to their EM-based performance estimates, than for simple sum rule fusion, which weights each classifier equally.     

Supervised image classification by contextual AdaBoost based on posteriors in neighborhoods

AdaBoost, a machine learning technique, is employed for supervised classification of land-cover categories of geostatistical data. We introduce contextual classifiers based on neighboring pixels. First, posterior probabilities are calculated at all pixels. Then, averages of the log posteriors are calculated in different neighborhoods and are then used as contextual classification functions. Weights for the classification functions can be determined by minimizing the empirical risk with multiclass. Finally, a convex combination of classification functions is obtained. The classification is performed by a noniterative maximization procedure. The proposed method is applied to artificial multispectral images and benchmark datasets. The performance of the proposed method is excellent and is similar to the Markov-random-field-based classifier, which requires an iterative maximization procedure.     

基于特征级融合和支持向量机的飞机识别

提出一种新的基于组合不变量的飞机识别方法。对不同飞机机型图像,提取Hu矩、仿射矩和归一化傅里叶描述子（NFD）3类不变量进行特征级融合。针对组合不变量取值范围较大问题,提出采用4种归一化方法,结合支持向量机（SVM）以提高飞机识别系统的分类性能。仿真实验表明,提取飞机的组合不变量特征,采用传统神经网络或SVM构建分类器...     

Optimizing feature extraction for multiclass problems

Feature extraction has been an important research topic in pattern classification and has been studied extensively by many researchers. Most of the conventional feature extraction methods are performed using a criterion function defined between two classes or a global function. Although these methods work relatively well in most cases, it is generally not optimal in any sense for multiclass problems. In order to address this problem, the authors propose a method to optimize feature extraction for multiclass problems. The authors first investigate the distribution of classification accuracies of multiclass problems in the feature space and find that there exist much better feature sets that the conventional feature extraction algorithms fail to find. Then the authors propose an algorithm that finds such features. Experiments with remotely sensed data show that the proposed algorithm consistently provides better performances compared with the conventional feature extraction algorithms     

Performance Measures for Neyman–Pearson Classification

In the Neyman-Pearson (NP) classification paradigm, the goal is to learn a classifier from labeled training data such that the probability of a false negative is minimized while the probability of a false positive is below a user-specified level alpha isin (0,1). This work addresses the question of how to evaluate and compare classifiers in the NP setting. Simply reporting false positives and false negatives leaves some ambiguity about which classifier is best. Unlike conventional classification, however, there is no natural performance measure for NP classification. We cannot reject classifiers whose false positive rate exceeds a since, among other reasons, the false positive rate must be estimated from data and hence is not known with certainty. We propose two families of performance measures for evaluating and comparing classifiers and suggest one criterion in particular for practical use. We then present general learning rules that satisfy performance guarantees with respect to these criteria. As in conventional classification, the notion of uniform convergence plays a central role, and leads to finite sample bounds, oracle inequalities, consistency, and rates of convergence. The proposed performance measures are also applicable to the problem of anomaly prediction.     

基于混合多样性生成与修剪的集成单类分类算法

针对传统集成学习方法直接应用于单类分类器效果不理想的问题,该文首先证明了集成学习方法能够提升单类分类器的性能,同时证明了若基分类器集不经选择会导致集成后性能下降;接着指出了经典集成方法直接应用于单类分类器集成时存在基分类器多样性严重不足的问题,并提出了一种能够提高多样性的基单类分类器混合生成策略;最后从集成损失构成的角度拆分集成单类分类器的损失函数,针对性地构造了集成单类分类器修剪策略并提出一种基于混合多样性生成和修剪的单类分类器集成算法,简称为PHD-EOC。在UCI标准数据集和恶意程序行为检测数据集上的实验结果表明,PHD-EOC算法兼顾多样性与单类分类性能,在各种单类分类器评价指标上均较经典集成学习方法有更好的表现,并降低了决策阶段的时间复杂度。     

Multiclass support vector machines for EEG-signals classification.

In this paper, we proposed the multiclass support vector machine (SVM) with the error-correcting output codes for the multiclass electroencephalogram (EEG) signals classification problem. The probabilistic neural network (PNN) and multilayer perceptron neural network were also tested and benchmarked for their performance on the classification of the EEG signals. Decision making was performed in two stages: feature extraction by computing the wavelet coefficients and the Lyapunov exponents and classification using the classifiers trained on the extracted features. The purpose was to determine an optimum classification scheme for this problem and also to infer clues about the extracted features. Our research demonstrated that the wavelet coefficients and the Lyapunov exponents are the features which well represent the EEG signals and the multiclass SVM and PNN trained on these features achieved high classification accuracies.     

基于模糊规则的多分类器融合

用非线性方法解决多分类器融合问题能够取得比较高的识别率, 但是,当前被应用在多分类器融合领域中的非线性方法可理解性较差,给使用者带来一定的困难。而基于模糊规则的模式识别方法是一类可理解性好的非线性方法,但迄今为止还没有被应用于多分类器融合问题中。基于上述考虑,该文将模糊系统应用到多分类器融合中,并且研究了如何设计可理解性好、精度高的模糊系统的问题,提出了一种改进的基于支持向量的模糊系统设计方法。该方法在从ELENA项目数据库和UCI数据库中选出的4个数据集上进行了测试。实验结果表明,该方法能够用可理解性好的模糊系统实现低错误率的多分类器融合。     

Segmentation and classification of dot and non-dot-like fluorescence in situ hybridization signals for automated detection of cytogenetic abnormalities.

Signal segmentation and classification of fluorescence in situ hybridization (FISH) images are essential for the detection of cytogenetic abnormalities. Since current methods are limited to dot-like signal analysis, we propose a methodology for segmentation and classification of dot and non-dot-like signals. First, nuclei are segmented from their background and from each other in order to associate signals with specific isolated nuclei. Second, subsignals composing non-dot-like signals are detected and clustered to signals. Features are measured to the signals and a subset of these features is selected representing the signals to a multiclass classifier. Classification using a naive Bayesian classifier (NBC) or a multilayer perceptron is accomplished. When applied to a FISH image database, dot and non-dot-like signals were segmented almost perfectly and then classified with accuracy of approximately 80% by either of the classifiers.     

Neuroinspired Architecture for Robust Classifier Fusion of Multisensor Imagery

Two new algorithms for robust and fault-tolerant classifier combination are presented. The attractor dynamics (AD) algorithm models some properties of sensory integration in the central nervous system and is based on the application of the dynamical systems for classifier fusion. The classifier masking (CM) algorithm is a nonneural version of the AD algorithm based on finding intersecting classifier intervals. Both of the proposed algorithms employ the idea of consensus among individual classifiers. The individual classifiers have been trained using resampled feature sets. They fuse the information from advanced synthetic aperture radar, medium resolution imaging spectrometer, and advanced along track scanning radiometer envisat satellite sensors for the improved sea ice classification. The results of our experiments show that training and combing the individual classifier outputs in a multiple classifier system significantly improve the robustness and the fault tolerance of the classification system as compared to the single classifier combining all sources of information. The robustness of the single classifier has been largely reduced in cases of single sensor failures (87.9 % in normal conditions versus 64.8% and 66.1% for two artificially corrupted data sets), whereas the CM algorithm is more tolerant to the sensor and preprocessing errors (86.4% in normal conditions versus 78.9% and 73.6% for two artificially corrupted data sets). The performance of the CM algorithm is superior to those of the simple multiple classifier combination strategies based on classifier averaging and majority voting (78.9% versus 70.9% and 69.5%, respectively) because the AD and CM algorithms are able to discard the corrupted classifier outputs based on classifier agreement and, in fact, represent hybrid approaches combining the properties of classifier averaging and classifier selection methods.     

Fingerprint and speaker verification decisions fusion using a functional link network

By exploiting the specialist capabilities of each classifier, a combined classifier may yield results which would not be possible with a single classifier. In this paper, we propose to combine the fingerprint and speaker verification decisions using a functional link network. This is to circumvent the nontrivial trial-and-error and iterative training effort as seen in backpropagation neural networks which cannot guarantee global optimal solutions. In many data fusion applications, as individual classifiers to be combined would have attained a certain level of classification accuracy, the proposed functional link network can be used to combine these classifiers by taking their outputs as the inputs to the network. The proposed network is first applied to a pattern recognition problem to illustrate its approximation capability. The network is then used to combine the fingerprint and speaker verification decisions with much improved receiver operating characteristics performance as compared to several decision fusion methods from the literature.     

Real-time implementation and performance evaluation of speech classifiers in speech analysis-synthesis

In this work, six voiced/unvoiced speech classifiers based on the autocorrelation function (ACF), average magnitude difference function (AMDF), cepstrum, weighted ACF (WACF), zero crossing rate and energy of the signal (ZCR-E), and neural networks (NNs) have been simulated and implemented in real time using the TMS320C6713 DSP starter kit. These speech classifiers have been integrated into a linear-predictive-coding-based speech analysis-synthesis system and their performance has been compared in terms of the percentage of the voiced/unvoiced classification accuracy, speech quality, and computation time. The results of the percentage of the voiced/unvoiced classification accuracy and speech quality show that the NN-based speech classifier performs better than the ACF-, AMDF-, cepstrum-, WACF- and ZCR-E-based speech classifiers for both clean and noisy environments. The computation time results show that the AMDF-based speech classifier is computationally simple, and thus its computation time is less than that of other speech classifiers, while that of the NN-based speech classifier is greater compared with other classifiers.     

Bagging linear sparse Bayesian learning models for variable selection in cancer diagnosis.

This paper investigates variable selection (VS) and classification for biomedical datasets with a small sample size and a very high input dimension. The sequential sparse Bayesian learning methods with linear bases are used as the basic VS algorithm. Selected variables are fed to the kernel-based probabilistic classifiers: Bayesian least squares support vector machines (BayLS-SVMs) and relevance vector machines (RVMs). We employ the bagging techniques for both VS and model building in order to improve the reliability of the selected variables and the predictive performance. This modeling strategy is applied to real-life medical classification problems, including two binary cancer diagnosis problems based on microarray data and a brain tumor multiclass classification problem using spectra acquired via magnetic resonance spectroscopy. The work is experimentally compared to other VS methods. It is shown that the use of bagging can improve the reliability and stability of both VS and model prediction.     

Machine Learning Based Intrusion Detection Systems for IoT Applications

Internet of Things (IoT) and its applications are the most popular research areas at present. The characteristics of IoT on one side make it easily applicable to real-life applications, whereas on the other side expose it to cyber threats. Denial of Service (DoS) is one of the most catastrophic attacks against IoT. In this paper, we investigate the prospects of using machine learning classification algorithms for securing IoT against DoS attacks. A comprehensive study is carried on the classifiers which can advance the development of anomaly-based intrusion detection systems (IDSs). Performance assessment of classifiers is done in terms of prominent metrics and validation methods. Popular datasets CIDDS-001, UNSW-NB15, and NSL-KDD are used for benchmarking classifiers. Friedman and Nemenyi tests are employed to analyze the significant differences among classifiers statistically. In addition, Raspberry Pi is used to evaluate the response time of classifiers on IoT specific hardware. We also discuss a methodology for selecting the best classifier as per application requirements. The main goals of this study are to motivate IoT security researchers for developing IDSs using ensemble learning, and suggesting appropriate methods for statistical assessment of classifier’s performance.

     

基于神经网络的纠错输出编码方法研究

构造基于数据编码矩阵是目前利用纠错输出编码解决多类分类问题的研究重点.为此提出利用单层感知器作为学习框架,结合解码策略把输出编码矩阵各码元值映射为感知器网络中的权值,同时引入含权值取值约束的目标函数作为该网络代价函数,并对其进行学习,最终得到基于子类划分的数据编码矩阵.实验中利用人工数据集和UCI数据集并选择线性逻辑分类器作为基分类器分别进行测试,通过与几种经典编码方法比较,结果表明该编码方法能在编码长度较小情况下得到更好的分类效果.     

满足多重分类误差的Gensini参数识别分类算法

文中研究基于常规体检数据对体检者的Gensini参数范围进行分类的方法,区分体检者分别在Gensini参数3个区间中的一个。由于判定分类性能需要考虑各个类别的漏识别率、误识别率指标,需要对不同的错误指标设计对应的分类器,文中提出利用三个2种类型的分类器输出结果,进行组合的方式,获得能够满足多种分类误差条件的分类器设计方法,所提出的方法简化了分类器设计复杂度,通过分类器输出结果的映射变换,能够快速实现满足不同分类误差要求的分类。     

Application of neural networks to radar image classification

A number of methods have been developed to classify ground terrain types from fully polarimetric synthetic aperture radar (SAR) images, and these techniques are often grouped into supervised and unsupervised approaches. Supervised methods have yielded higher accuracy than unsupervised techniques, but suffer from the need for human interaction to determine classes and training regions. In contrast, unsupervised methods determine classes automatically, but generally show limited ability to accurately divide terrain into natural classes. In this paper, a new terrain classification technique is introduced to determine terrain classes in polarimetric SAR images, utilizing unsupervised neural networks to provide automatic classification, and employing an iterative algorithm to improve the performance. Several types of unsupervised neural networks are first applied to the classification of SAR images, and the results are compared to those of more conventional unsupervised methods. Results show that one neural network method-Learning Vector Quantization (LVQ)-outperforms the conventional unsupervised classifiers, but is still inferior to supervised methods. To overcome this poor accuracy, an iterative algorithm is proposed where the SAR image is reclassified using a maximum likelihood (ML) classifier. It is shown that this algorithm converges, and significantly improves classification accuracy     

基于最小距离的音频分类方法的研究

基于内容的音频分类是一个有趣并有重要意义的问题。音频分类技术包括音频特征抽取和分类器两个基本部分。如今,基于内容的音频自动分类技术已经有了很大的发展。然而,现有的基于内容的音频自动分类方法在分类的准确性、有效性和算法复杂度等诸多方面存在一定的不足,探索性能更佳的方法就成为了该领域的研究热点。提取了基于内容的音频分类所使用的音频特征,得到了基于帧的音频特征和基于片段的音频特征两个层次的特征,并提出了一种基于MFCC的简化的特征;选取了最小距离分类器中的最近邻分类器和K近邻分类器,对这几种典型的音频分类器进行研究,进行仿真实验,分析了实验结果;最后设计并仿真了经过改进的最小距离音频分类器,它的性能相对于原有的最近邻和K近邻分类器有一定的提高,并具有很低的系统复杂度和很短的分类时间。