Intelligent GPGPU Classification in Volume Visualization: A framework based on Error‐Correcting Output Codes

In volume visualization, the definition of the regions of interest is inherently an iterative trial‐and‐error process finding out the best parameters to classify and render the final image. Generally, the user requires a lot of expertise to analyze and edit these parameters through multi‐dimensional transfer functions. In this paper, we present a framework of intelligent methods to label on‐demand multiple regions of interest. These methods can be split into a two‐level GPU‐based labelling algorithm that computes in time of rendering a set of labelled structures using the Machine Learning Error‐Correcting Output Codes (ECOC) framework. In a pre‐processing step, ECOC trains a set of Adaboost binary classifiers from a reduced pre‐labelled data set. Then, at the testing stage, each classifier is independently applied on the features of a set of unlabelled samples and combined to perform multi‐class labelling. We also propose an alternative representation of these classifiers that allows to highly parallelize the testing stage. To exploit that parallelism we implemented the testing stage in GPU‐OpenCL. The empirical results on different data sets for several volume structures shows high computational performance and classification accuracy.     

基于单类分类器的半监督学习

提出一种结合单类学习器和集成学习优点的Ensemble one-class半监督学习算法.该算法首先为少量有标识数据中的两类数据分别建立两个单类分类器.然后用建立好的两个单类分类器共同对无标识样本进行识别,利用已识别的无标识样本对已建立的两个分类面进行调整、优化.最终被识别出来的无标识数据和有标识数据集合在一起训练一个基分类器,多个基分类器集成在一起对测试样本的测试结果进行投票.在5个UCI数据集上进行实验表明,该算法与tri-training算法相比平均识别精度提高4.5%,与仅采用纯有标识数据的单类分类器相比,平均识别精度提高8.9%.从实验结果可以看出,该算法在解决半监督问题上是有效的.     

Improving classical contextual classifications

This paper shows some combinations of classifiers that achieve high accuracy classifications. Traditionally the maximum likelihood classification is used as an initial classification for a contextual classifier. We show that by using different non-parametric spectral classifiers to obtain the initial classification, we can significatively improve the accuracy of the classification with a reasonable computational cost. In this work we propose the use of different spectral classifications as initial maps for a contextual classifier (ICM) in order to obtain some interesting combinations of spectral-contextual classifiers for remote sensing image classification with an acceptable trade-off between the accuracy of the final classification and the computational effort required.     

多种分类器融合的遥感影像分类

在遥感影像分类应用中,不同分类器的分类精度是不同的,而同一分类器对不同类别的分类精度也是不相同的。多分类器结合的思想就是利用现有分类器之间的互补性,通过适当的方法将不同的分类器之间进行优势互补,往往可以得到比单个分类器更好的分类结果。本文研究了如何在Matlab下采用最短距离分类器、贝叶斯分类器、BP神经网络分类器对影像进行分类,并采用投票法进行多种分类器结合的遥感影像分类,最后进行分类后处理。实验结果表明多分类器结合的遥感影像分类比单一分类器分类的精度高。     

Ensemble Online Classifier Based on the One-Class Base Classifiers for Mining Data Streams

The problem addressed in this study concerns mining data streams with concept drift. The goal of the article is to propose and validate a new approach to mining data streams with concept-drift using the ensemble classifier constructed from the one-class base classifiers. It is assumed that base classifiers of the proposed ensemble are induced from incoming chunks of the data stream. Each chunk consists of prototypes and information about whether the class prediction of these instances, carried-out at earlier steps, has been correct. Each data chunk can be updated by using the instance selection technique when new data arrive. When a new data chunk is formed, the ensemble model is also updated on the basis of weights assigned to each one-class classifier. In this article, two well-known instance-based learning algorithms—the CNN and the ENN—have been adopted to solve the one-class classification problems and, consequently, update the proposed classifier ensemble. The proposed approaches have been validated experimentally, and the computational experiment results are shown and discussed. The experiment results prove that the proposed approach using the ensemble classifier constructed from the one-class base classifiers with instance selection for chunk updating can outperform well-known approaches for data streams with concept drift.     

一种基于模糊关联分类的遥感图像分类方法

遥感图像分类是遥感领域的研究热点之一.提出了一种基于自适应区间划分的模糊关联遥感图像分类方法(fuzzy associative remote sensing classification,FARSC).算法根据遥感图像分类的特点,利用模糊C均值聚类算法自适应地建立连续型属性模糊区间,使用新的剪枝策略对项集进行筛选从而避免生成无用规则,采用一种新的规则重要性度量方法对多模糊分类规则进行融合,从而有效地提高分类效率和精确度.在UCI数据和遥感图像上所作实验结果表明,算法具有较高的分类精度以及对样本数量变化的不敏感性,对于解决遥感图像分类问题,FARSC算法具有较高的实用性,是一种有效的遥感图像分类方法.     

SVM在多源遥感图像分类中的应用研究

在利用遥感图像进行土地利用/覆盖分类过程中,可采用以下两种途径来提高分类精度:一是通过增加有利于分类的数据源,引入地理辅助数据和归一化植被指数(NDVI)来进行多源信息融合;二是选择更好的分类方法,例如支持向量机(SVM)学习方法,由于该方法克服了最大似然法和神经网络的弱点,非常适合高维、复杂的小样本多源数据的分类。为了提高多源遥感图像分类的精度,还研究了支持向量机在遥感图像分类中模型的选择,包括多类模型和核函数的选择。分类结果表明,支持向量机比传统的分类方法具有更高的精度,尤其是基于径向基核函数和一对一多类方法的支持向量机模型更适合多源遥感图像分类,因此,基于支持向量机的多源土地利用/覆盖分类能大大提高分类精度。     

A multi-one-class dynamic classifier for adaptive digitization of document streams

In this paper, we present a new dynamic classifier design based on a set of one-class independent SVM for image data stream categorization. Dynamic or continuous learning and classification has been recently investigated to deal with different situations, like online learning of fixed concepts, learning in non-stationary environments (concept drift) or learning from imbalanced data. Most of solutions are not able to deal at the same time with many of these specificities. Particularly, adding new concepts, merging or splitting concepts are most of the time considered as less important and are consequently less studied, whereas they present a high interest for stream-based document image classification. To deal with that kind of data, we explore a learning and classification scheme based on one-class SVM classifiers that we call mOC-iSVM (multi-one-class incremental SVM). Even if one-class classifiers are suffering from a lack of discriminative power, they have, as a counterpart, a lot of interesting properties coming from their independent modeling. The experiments presented in the paper show the theoretical feasibility on different benchmarks considering addition of new classes. Experiments also demonstrate that the mOC-iSVM model can be efficiently used for tasks dedicated to documents classification (by image quality and image content) in a context of streams, handling many typical scenarii for concepts extension, drift, split and merge.     

基于最近邻的遥感影像单类信息提取

遥感影像单类信息提取是一种特殊的分类,旨在训练和提取单一兴趣类别。研究了基于最近邻分类器的单类信息提取方法,包括类别划分和样本选择问题。首先分析论证了最近邻方法提取单类信息只与所选择的样本相关,而与类别划分无关,因此可以将单类信息提 取作为二类分类问题进行处理。然后在二类分类问题中,根据空间和特征邻近性选择非兴趣类别的部分训练样本,简化了分类过程。实验结果表明,所提出的方法可以有效实现遥感影像单类信息的提取。     

Linear classifier design under heteroscedasticity in Linear Discriminant Analysis

Under normality and homoscedasticity assumptions, Linear Discriminant Analysis (LDA) is known to be optimal in terms of minimising the Bayes error for binary classification. In the heteroscedastic case, LDA is not guaranteed to minimise this error. Assuming heteroscedasticity, we derive a linear classifier, the Gaussian Linear Discriminant (GLD), that directly minimises the Bayes error for binary classification. In addition, we also propose a local neighbourhood search (LNS) algorithm to obtain a more robust classifier if the data is known to have a non-normal distribution. We evaluate the proposed classifiers on two artificial and ten real-world datasets that cut across a wide range of application areas including handwriting recognition, medical diagnosis and remote sensing, and then compare our algorithm against existing LDA approaches and other linear classifiers. The GLD is shown to outperform the original LDA procedure in terms of the classification accuracy under heteroscedasticity. While it compares favourably with other existing heteroscedastic LDA approaches, the GLD requires as much as 60 times lower training time on some datasets. Our comparison with the support vector machine (SVM) also shows that, the GLD, together with the LNS, requires as much as 150 times lower training time to achieve an equivalent classification accuracy on some of the datasets. Thus, our algorithms can provide a cheap and reliable option for classification in a lot of expert systems.     

The performance of regularized discriminant analysis versus non-parametric classifiers applied to high-dimensional image classification

Classification of very-high-dimensional images is of the utmost interest in remote sensing applications. Storage space, and mainly the computational effort required for classifying these kinds of images, are the main drawbacks in practice. Moreover, it is well known that a number of spectral classifiers may not be useful (even not valid) in practice for classifying very-high-dimensional images. Even if they are valid, they do not provide high-accuracy classifications when the training sets are high-overlapping in the representation space due to the shape of the decision boundaries they impose. In these cases, it is preferable to adopt a classifier that may adjust the decision boundaries in a better fashion. To do so, classification based on regularized discriminant analysis (RDA) was compared with a number of non-parametric classifiers. Two synthetic image databases consisting of high-dimensional images were used for testing the performance of the classifiers. These datasets were created using a procedure proposed by the authors. The main conclusion of this paper is that RDA may be used successfully for classifying very-high-dimensional images with high-overlapping training sets. RDA also provides an excellent classification accuracy for classifying real datasets in which training sets are high-overlapping in the representation space.     

Transductive methods for the distributed ensemble classification problem

We consider ensemble classification for the case where there is no common labeled training data for jointly designing the individual classifiers and the function that aggregates their decisions. This problem, which we call distributed ensemble classification, applies when individual classifiers operate (perhaps remotely) on different sensing modalities and when combining proprietary or legacy classifiers. The conventional wisdom in this case is to apply fixed rules of combination such as voting methods or rules for aggregating probabilities. Alternatively, we take a transductive approach, optimizing the combining rule for an objective function measured on the unlabeled batch of test data. We propose maximum likelihood (ML) objectives that are shown to yield well-known forms of probabilistic aggregation, albeit with iterative, expectation-maximization-based adjustment to account for mismatch between class priors used by individual classifiers and those reflected in the new data batch. These methods are extensions, for the ensemble case, of the work of Saerens, Latinne, and Decaestecker (2002). We also propose an information-theoretic method that generally outperforms the ML methods, better handles classifier redundancies, and addresses some scenarios where the ML methods are not applicable. This method also well handles the case of missing classes in the test batch. On UC Irvine benchmark data, all our methods give improvements in classification accuracy over the use of fixed rules when there is prior mismatch.     

Decision tree regression for soft classification of remote sensing data

In recent years, decision tree classifiers have been successfully used for land cover classification from remote sensing data. Their implementation as a per-pixel based classifier to produce hard or crisp classification has been reported in the literature. Remote sensing images, particularly at coarse spatial resolutions, are contaminated with mixed pixels that contain more than one class on the ground. The per-pixel approach may result in erroneous classification of images dominated by mixed pixels. Therefore, soft classification approaches that decompose the pixel into its class constituents in the form of class proportions have been advocated. In this paper, we employ a decision tree regression approach to determine class proportions within a pixel so as to produce soft classification from remote sensing data. Classification accuracy achieved by decision tree regression is compared with those achieved by the most widely used maximum likelihood classifier, implemented in the soft mode, and a supervised version of the fuzzy c-means classifier. Root Mean Square Error (RMSE) and fuzzy error matrix based measures have been used for accuracy assessment of soft classification.     

并联结构组合分类器的误差分析

虽然遥感图像分类器发展迅速,但是单分类器分类精度仍然不能满足实际应用的需要,因此组合分类器便成为遥感分类技术研究的一个重要方面。与串联结构相比,并联结构是实际应用过程中研究较早、发展较充分和应用最广泛的,因而主要就并联结构组合分类器的误差进行分析。通过理论分析得出：并联结构组合分类器的精度变化与单分类器的误差集合分布模式有关。相离时,其精度提升幅度最大;相交时,组合分类器精度得到提升,其精度提升幅度大小与组合后的误差像元集合的大小成反比;相包含时,其精度位于两单分类器之间,且更靠近精度较高的单分类器的精度。同时以长沙市局部区域为实验区进行了验证性的实验,实验结果有效地验证了并联结构组合分类器的误差分析所得到的推论。最后从理论上讨论了组合分类器提高遥感图像分类精度的可行性,并指出单个类别的组合分类结果也与单分类器对这个类别的误差集合分布模式有关,为组合分类器发展提供了一个较好的突破方向。     

Remote sensing image classification based on neural network ensemble algorithm

The amounts and types of remote sensing data have increased rapidly, and the classification of these datasets has become more and more overwhelming for a single classifier in practical applications. In this paper, an ensemble algorithm based on Diversity Ensemble Creation by Oppositional Relabeling of Artificial Training Examples (DECORATEs) and Rotation Forest is proposed to solve the classification problem of remote sensing image. In this ensemble algorithm, the RBF neural networks are employed as base classifiers. Furthermore, interpolation technology for identical distribution is used to remold the input datasets. These remolded datasets will construct new classifiers besides the initial classifiers constructed by the Rotation Forest algorithm. The change of classification error is used to decide whether to add another new classifier. Therefore, the diversity among these classifiers will be enhanced and the accuracy of classification will be improved. Adaptability of the proposed algorithm is verified in experiments implemented on standard datasets and actual remote sensing dataset.     

遥感影像的分类自校正方法研究

遥感图像分类是遥感图像研究的主要内容之一,分类精度高低直接关系到遥感数据的可靠性和实用性。多分类器系统可以提高单分类器分类的精度,但往往要求组成的子分类器分类误差相互独立,子分类器选择困难。支持向量机是新发展起来的一种非参数分类器,其分类原理和传统的基于统计的分类方法不同,表现出一定的独立性。为此本文尝试基于支持向量机和目前使用最广泛的最大似然法,构建一个性能高效且组合方式简单的复合分类器(称为遥感影像分类自校正方法)。同时,为了验证该分类器的性能,在北京市2006年4月27日的SPOT2图像上选择了一个研究区,分别利用最大似然法、支持向量机法和分类自校正方法进行分类对比试验。结果显示分类自校正方法的总体分类精度最高,比最大似然法和支持向量机法分别提高了4.35%和6.6%,而且各种地物类型的分类精度相对最大似然和支持向量机法都有提高。本文提出的分类自校正方法是一种性能高效且操作简单的分类方法。     

A case-study on naïve labelling for the nearest mean and the linear discriminant classifiers

The abundance of unlabelled data alongside limited labelled data has provoked significant interest in semi-supervised learning methods. “Naïve labelling” refers to the following simple strategy for using unlabelled data in on-line classification. A new data point is first labelled by the current classifier and then added to the training set together with the assigned label. The classifier is updated before seeing the subsequent data point. Although the danger of a run-away classifier is obvious, versions of naïve labelling pervade in on-line adaptive learning. We study the asymptotic behaviour of naïve labelling in the case of two Gaussian classes and one variable. The analysis shows that if the classifier model assumes correctly the underlying distribution of the problem, naïve labelling will drive the parameters of the classifier towards their optimal values. However, if the model is not guessed correctly, the benefits are outweighed by the instability of the labelling strategy (run-away behaviour of the classifier). The results are based on exact calculations of the point of convergence, simulations, and experiments with 25 real data sets. The findings in our study are consistent with concerns about general use of unlabelled data, flagged up in the recent literature.     

Incremental data-driven learning of a novelty detection model for one-class classification with application to high-dimensional noisy data

Most conventional learning algorithms require both positive and negative training data for achieving accurate classification results. However, the problem of learning classifiers from only positive data arises in many applications where negative data are too costly, difficult to obtain, or not available at all. This paper describes a new machine learning approach, called ILoNDF (Incremental data-driven Learning of Novelty Detector Filter). The approach is inspired by novelty detection theory and its learning method, which typically requires only examples from one class to learn a model. One advantage of ILoNDF is the ability of its generative learning to capture the intrinsic characteristics of the training data by continuously integrating the information relating to the relative frequencies of the features of training data and their co-occurrence dependencies. This makes ILoNDF rather stable and less sensitive to noisy features which may be present in the representation of the positive data. In addition, ILoNDF does not require extensive computational resources since it operates on-line without repeated training, and no parameters need to be tuned. In this study we mainly focus on the robustness of ILoNDF in dealing with high-dimensional noisy data and we investigate the variation of its performance depending on the amount of data available for training. To make our study comparable to previous studies, we investigate four common methods: PCA residuals, Hotelling’s T ² test, an auto-associative neural network, and a one-class version of the SVM classifier (lately a favored method for one-class classification). Experiments are conducted on two real-world text corpora: Reuters and WebKB. Results show that ILoNDF tends to be more robust, is less affected by initial settings, and consistently outperforms the other methods.     

基于单类支持向量机的电子鼻在白酒真假识别中的应用

针对采用多类分类方法进行白酒真假识别时存在的真酒样本和假酒样本（正类样本和异类样本）数量无法均衡以及异类样本无法全部获取的问题,提出应用单类支持向量机分别单独对每一种品牌的白酒训练单类分类器进行真假识别的方法。首先采用自主设计的电子鼻系统对不同品牌白酒进行采样测试;采样后的传感器阵列数据依次经过数据预处理、特征生成、特征选择降维处理,得到可用于分类的白酒样本;再通过格点搜索获取每种白酒单类分类器的最优参数;最后测试各个单类分类器对相应品牌白酒的真假识别效果。各单类分类器的真假识别率分布在93%~98%之间,结果表明,采用自主设计的电子鼻结合单类支持向量机可以很好地对白酒真假进行识别。