Decision tree regression for soft classification of remote sensing data

In recent years, decision tree classifiers have been successfully used for land cover classification from remote sensing data. Their implementation as a per-pixel based classifier to produce hard or crisp classification has been reported in the literature. Remote sensing images, particularly at coarse spatial resolutions, are contaminated with mixed pixels that contain more than one class on the ground. The per-pixel approach may result in erroneous classification of images dominated by mixed pixels. Therefore, soft classification approaches that decompose the pixel into its class constituents in the form of class proportions have been advocated. In this paper, we employ a decision tree regression approach to determine class proportions within a pixel so as to produce soft classification from remote sensing data. Classification accuracy achieved by decision tree regression is compared with those achieved by the most widely used maximum likelihood classifier, implemented in the soft mode, and a supervised version of the fuzzy c-means classifier. Root Mean Square Error (RMSE) and fuzzy error matrix based measures have been used for accuracy assessment of soft classification.     

Accuracy statistics for judging soft classification

In the literature one can find different accuracy measures that are built from the error matrix. However, standard accuracy assessment, which is based on the error matrix, is incomplete when dealing with fuzzy sets or when errors do not have the same importance. In this paper, we propose an extension of the error concept for soft (or crisp) classification that will be able to extend standard accuracy measures (e.g., overall, producer's, user's or Kappa statistic) that can be used in any framework: errors with different importance, soft classifier and crisp reference data (expert) or with a fuzzy expert. In particular, a weighted measure is built that takes into account the preferences of the decision maker in order to differentiate some errors that must not be considered equal.     

Defending against XML-related attacks in e-commerce applications with predictive fuzzy associative rules

Security administrators need to prioritise which feature to focus on amidst the various possibilities and avenues of attack, especially via Web Service in e-commerce applications. This study addresses the feature selection problem by proposing a predictive fuzzy associative rule model (FARM). FARM validates inputs by segregating the anomalies based fuzzy associative patterns discovered from five attributes in the intrusion datasets. These associative patterns leads to the discovery of a set of 18 interesting rules at 99% confidence and subsequently, categorisation into not only certainly allow/deny but also probably deny access decision class. FARM's classification provides 99% classification accuracy and less than 1% false alarm rate. Our findings indicate two benefits to using fuzzy datasets. First, fuzzy enables the discovery of fuzzy association patterns, fuzzy association rules and more sensitive classification. In addition, the root mean squared error (RMSE) and classification accuracy for fuzzy and crisp datasets do not differ much when using the Random Forest classifier. However, when other classifiers are used with increasing number of instances on the fuzzy and crisp datasets, the fuzzy datasets perform much better. Future research will involve experimentation on bigger data sets on different data types.     

Affect classification using genetic-optimized ensembles of fuzzy ARTMAPs

Training neural networks in distinguishing different emotions from physiological signals frequently involves fuzzy definitions of each affective state. In addition, manual design of classification tasks often uses sub-optimum classifier parameter settings, leading to average classification performance. In this study, an attempt to create a framework for multi-layered optimization of an ensemble of classifiers to maximize the system's ability to learn and classify affect, and to minimize human involvement in setting optimum parameters for the classification system is proposed. Using fuzzy adaptive resonance theory mapping (ARTMAP) as the classifier template, genetic algorithms (GAs) were employed to perform exhaustive search for the best combination of parameter settings for individual classifier performance. Speciation was implemented using subset selection of classification data attributes, as well as using an island model genetic algorithms method. Subsequently, the generated population of optimum classifier configurations was used as candidates to form an ensemble of classifiers. Another set of GAs were used to search for the combination of classifiers that would result in the best classification ensemble accuracy. The proposed methodology was tested using two affective data sets and was able to produce relatively small ensembles of fuzzy ARTMAPs with excellent affect recognition accuracy.     

Compact fuzzy association rule-based classifier

Classification is one of the most popular data mining techniques applied to many scientific and industrial problems. The efficiency of a classification model is evaluated by two parameters, namely the accuracy and the interpretability of the model. While most of the existing methods claim their accurate superiority over others, their models are usually complex and hardly understandable for the users. In this paper, we propose a novel classification model that is based on easily interpretable fuzzy association rules and fulfils both efficiency criteria. Since the accuracy of a classification model can be largely affected by the partitioning of numerical attributes, this paper discusses several fuzzy and crisp partitioning techniques. The proposed classification method is compared to 15 previously published association rule-based classifiers by testing them on five benchmark data sets. The results show that the fuzzy association rule-based classifier presented in this paper, offers a compact, understandable and accurate classification model.     

Neutrosophic classifier: An extension of fuzzy classifer

Fuzzy classification has become of great interest because of its ability to utilize simple linguistically interpretable rules and has overcome the limitations of symbolic or crisp rule based classifiers. This paper introduces an extension to fuzzy classifier: a neutrosophic classifier, which would utilize neutrosophic logic for its working. Neutrosophic logic is a generalized logic that is capable of effectively handling indeterminacy, stochasticity acquisition errors that fuzzy logic cannot handle. The proposed neutrosophic classifier employs neutrosophic logic for its working and is an extension of commonly used fuzzy classifier. It is compared with the commonly used fuzzy classifiers on the following parameters: nature of membership functions, number of rules and indeterminacy in the results generated. It is proved in the paper that extended fuzzy classifier: neutrosophic classifier; optimizes the said parameters in comparison to the fuzzy counterpart. Finally the paper is concluded with justifying that neutrosophic logic though in its nascent stage still holds the potential to be experimented for further exploration in different domains.     

Increasing soft classification accuracy through the use of an ensemble of classifiers

Although soft classification analyses can reduce problems such as those associated with mixed pixels that impact negatively on conventional hard classifications their accuracy is often low. One approach to increasing the accuracy of soft classifications is the use of an ensemble of classifiers, an approach which has been successful for hard classifications but rarely applied for soft classifications. Four methods for combining soft classifications to increase soft classification accuracy were assessed. These methods were based on (i) the selection of the most accurate predictions on a class‐specific basis, (ii) the average of the outputs of the individual classifications for each case, (iii) the direct combination of classifications using evidential reasoning and (iv) the adaptation of the outputs to enable the use of a conventional (hard classification) ensemble approach. These four approaches were assessed with classifications of National Oceanic and Atmospheric Administration (NOAA) Advanced Very High‐Resolution Radiometer (AVHRR) imagery of Australia. The data were classified using two neural networks and a probabilistic classifier. All four ensemble approaches applied to the outputs of these three classifiers were found to increase classification accuracy. Relative to the most accurate individual classification, the increases in overall accuracy derived ranged from 2.20% to 4.45%, increases that were statistically significant at 95% level of confidence. The results highlight that ensemble approaches may be used to significantly increase soft classification accuracy.     

Diversity-based combination of non-parametric classifiers for EMG signal decomposition

Non-parametric classification procedures based on a certainty measure and nearest neighbour rule for motor unit potential classification (MUP) during electromyographic (EMG) signal decomposition were explored. A diversity-based classifier fusion approach is developed and evaluated to achieve improved classification performance. The developed system allows the construction of a set of non-parametric base classifiers and then automatically chooses, from the pool of base classifiers, subsets of classifiers to form candidate classifier ensembles. The system selects the classifier ensemble members by exploiting a diversity measure for selecting classifier teams. The kappa statistic is used as the diversity measure to estimate the level of agreement between base classifier outputs, i.e., to measure the degree of decision similarity between base classifiers. The pool of base classifiers consists of two kinds of classifiers: adaptive certainty-based classifiers (ACCs) and adaptive fuzzy k-NN classifiers (AFNNCs) and both utilize different types of features. Once the patterns are assigned to their classes, by the classifier fusion system, firing pattern consistency statistics for each class are calculated to detect classification errors in an adaptive fashion. Performance of the developed system was evaluated using real and simulated EMG signals and was compared with the performance of the constituent base classifiers and the performance of the fixed ensemble containing the full set of base classifiers. Across the EMG signal data sets used, the diversity-based classifier fusion approach had better average classification performance overall, especially in terms of reducing classification errors.     

一种基于模糊关联分类的遥感图像分类方法

遥感图像分类是遥感领域的研究热点之一.提出了一种基于自适应区间划分的模糊关联遥感图像分类方法(fuzzy associative remote sensing classification,FARSC).算法根据遥感图像分类的特点,利用模糊C均值聚类算法自适应地建立连续型属性模糊区间,使用新的剪枝策略对项集进行筛选从而避免生成无用规则,采用一种新的规则重要性度量方法对多模糊分类规则进行融合,从而有效地提高分类效率和精确度.在UCI数据和遥感图像上所作实验结果表明,算法具有较高的分类精度以及对样本数量变化的不敏感性,对于解决遥感图像分类问题,FARSC算法具有较高的实用性,是一种有效的遥感图像分类方法.     

A hybrid approach to urban land use/cover mapping using Landsat 7 Enhanced Thematic Mapper Plus (ETM+) images

A hybrid method that incorporates the advantages of supervised and unsupervised approaches as well as hard and soft classifications was proposed for mapping the land use/cover of the Atlanta metropolitan area using Landsat 7 Enhanced Thematic Mapper Plus (ETM+) data. The unsupervised ISODATA clustering method was initially used to segment the image into a large number of clusters of pixels. With reference to ground data based on 1?:?40?000 colour infrared aerial photographs in the form of Digital Orthophoto Quarter Quadrangle (DOQQ), homogeneous clusters were labelled. Clusters that could not be labelled because of mixed pixels were clipped out and subjected to a supervised fuzzy classification. A final land use/cover map was obtained by a union overlay of the two partial land use/cover maps. This map was evaluated by comparing with maps produced using unsupervised ISODATA clustering, supervised fuzzy and supervised maximum likelihood classification methods. It was found that the hybrid approach was slightly better than the unsupervised ISODATA clustering in land use/cover classification accuracy, most probably because of the supervised fuzzy classification, which effectively dealt with the mixed pixel problem in the low-density urban use category of land use/cover. It was suggested that this hybrid approach can be economically implemented in a standard image processing software package to produce land use/cover maps with higher accuracy from satellite images of moderate spatial resolution in a complex urban environment, where both discrete and continuous land cover elements occur side by side.     

A Dempster Shafer-based fuzzy multisensor fusion system using airborne lidar and hyperspectral imagery

ABSTRACT

Nowadays, accurate spectral reflectance information is provided by hyperspectral (HS) data while light detection and ranging (lidar) data provides precise information about the height and geometrical properties of the surfaces. In the most research papers, data fusion of disparate sensors significantly improves object classification performance compared to that of just an individual sensor. Previous researches on fusion of these two sensors had problems such as crisp classifiers or simple fuzzy decision-making systems. This article tries to overcome these weaknesses by accurate support vector machine (SVM) and Fuzzy SVM as classifiers in crisp and fuzzy decision fusion system and fusion of two sensors by two different methods based on precise theories of Bayesian and Shafer. Also, the proposed method tries to compare the results of fusion of both data using decision fusion system with stacked features strategy. This study focuses on HS and lidar fusion through three main phases. The first phase is based on the using of Noise Weighted Harsanyi-Farrand-Chang method and principal component analysis to overcome the high dimensionality problem of HS data. The second phase is based on the feature extraction and selection strategy on lidar data. Finally, fuzzy SVM and Dempster Shafer methods are applied as fuzzy classification and fuzzy decision fusion strategies on the feature spaces. A co-registered HS and lidar data set from Houston of U.S.A. by 15 classes was available to examine the effectiveness of the proposed method. The results of this study highlight that the combination of HS and lidar data enable reliable mapping of land cover.     

A fuzzy classifier to deal with similarity between labels on automatic prosodic labeling

This paper presents an original approach to automatic prosodic labeling. Fuzzy logic techniques are used for representing situations of high uncertainty with respect to the category to be assigned to a given prosodic unit. The Fuzzy Integer technique is used to combine the output of different base classifiers. The resulting fuzzy classifier benefits from the different capabilities of the base classifiers for identifying different types of prosodic events. At the same time, the fuzzy classifier identifies the events that are potentially more difficult to be labeled. The classifier has been applied to the identification of ToBI pitch accents. The state of the art on pitch accent multiclass classification reports around 70% accuracy rate. In this paper we describe a fuzzy classifier which assigns more than one label in confusing situations. We show that the pairs of labels that appear in these uncertain situations are consistent with the most confused pairs of labels reported in manual prosodic labeling experiments. Our fuzzy classifier obtains a soft classification rate of 81.8%, which supports the potential of the proposed system for computer assisted prosodic labeling.     

基于多分类器集成的“北京一号”小卫星遥感影像分类研究

将多分类器集合应用于"北京一号"小卫星多光谱遥感数据土地覆盖分类,首先构建分类器集合,应用最小距离分类、最大似然分类、支持向量机(SVM)、BP神经网络、RBF神经网络和决策树等进行土地覆盖分类,然后利用Bagging、Boosting、投票法、证据理论和模糊积分法等分类器集成方法,得到综合不同分类器输出的最终分类结果。试验表明,多分类器集成能够有效提高"北京一号"小卫星土地覆盖分类的精度,具有广泛的应用前景。     

Comparison of fuzzy c-means classification,linear mixture modelling and MLC probabilities as tools for unmixing coarse pixels

Three different 'soft' classifiers (fuzzy c-means classifier, linear mixture model, and probability values from a maximum likelihood classification) were used for unmixing of coarse pixel signatures to identify four land cover classes (i.e., supervised classifications). The coarse images were generated from a 30m Thematic Mapper (TM) image; one set by mean filtering, and another using an asymmetric filter kernel to simulate Multi-Spectral Scanner (MSS) sensor sampling. These filters collapsed together windows of up to 11 11 pixels. The fractional maps generated by the three classifiers were compared to truth maps at the corresponding scales, and to the results of a hard maximum likelihood classification. Overall, the fuzzy c-means classifier gave the best predictions of sub-pixel landcover areas, followed by the linear mixture model. The probabilities differed little from the hard classification, suggesting that the clusters should be modelled more loosely. This paper demonstrates successful methods for use and comparison of the classifiers that should ideally be extended to a real dataset.     

The role of digital bathymetry in mapping shallow marine vegetation from hyperspectral image data

Hyperspectral remote sensing is a proven technology for measurement of coastal ocean colour, including sea‐bed mapping in optically shallow waters. Using hyperspectral imagery of shallow (<15 m deep) sea bed acquired with the Compact Airborne Spectrographic Imager (CASI‐550), we examined how changes in the spatial resolution of bathymetric grids, created from sonar data (echosounding) and input to conventional image classifiers, affected the accuracy of distributional maps of invasive (Codium fragile ssp. tomentosoides) and native (kelp) seaweeds off the coast of Nova Scotia, Canada. The addition of a low‐resolution bathymetric grid, interpolated from soundings by the Canadian Hydrographic Service, improved the overall classification accuracies by up to ～10%. However, increasing the bathymetric resolution did not increase the accuracy of classification maps produced with the supervised (Maximum Likelihood) classifier as shown by a slightly lower accuracy (2%) when using an intermediate‐resolution bathymetric grid interpolated from soundings with a recreational fish finder. Supervised classifications using the first three eigenvectors from a principal‐components analysis were consistently more accurate (by at least 27%) than unsupervised (K‐means classifier) schemes with similar data compression. With an overall accuracy of 76%, the most reliable scheme was a supervised classification with low‐resolution bathymetry. However, the supervised approach was particularly sensitive, and variations in accuracy of 2% resulted in overestimations of up to 53% in the extent of C. fragile and kelp. The use of a passive optical bathymetric algorithm to derive a high‐resolution bathymetric grid from the CASI data showed promise, although fundamental differences between this grid and those created with the sonar data limited the conclusions. The bathymetry (at any spatial resolution) appeared to improve the accuracy of the classifications both by reducing the confusion among the spectral classes and by removing noise in the image data. Variations in the accuracy of depth estimates and inescapable positional inaccuracies in the imagery and ground data largely accounted for the observed differences in the classification accuracies. This study provides the first detailed demonstration of the advantages and limitations of integrating digital bathymetry with hyperspectral data for the mapping of benthic assemblages in optically shallow waters.     

Performance evaluation of pattern classifiers for handwritten character recognition

This paper describes a performance evaluation study in which some efficient classifiers are tested in handwritten digit recognition. The evaluated classifiers include a statistical classifier (modified quadratic discriminant function, MQDF), three neural classifiers, and an LVQ (learning vector quantization) classifier. They are efficient in that high accuracies can be achieved at moderate memory space and computation cost. The performance is measured in terms of classification accuracy, sensitivity to training sample size, ambiguity rejection, and outlier resistance. The outlier resistance of neural classifiers is enhanced by training with synthesized outlier data. The classifiers are tested on a large data set extracted from NIST SD19. As results, the test accuracies of the evaluated classifiers are comparable to or higher than those of the nearest neighbor (1-NN) rule and regularized discriminant analysis (RDA). It is shown that neural classifiers are more susceptible to small sample size than MQDF, although they yield higher accuracies on large sample size. As a neural classifier, the polynomial classifier (PC) gives the highest accuracy and performs best in ambiguity rejection. On the other hand, MQDF is superior in outlier rejection even though it is not trained with outlier data. The results indicate that pattern classifiers have complementary advantages and they should be appropriately combined to achieve higher performance. Received: July 18, 2001 / Accepted: September 28, 2001     

Classification of weld flaws with imbalanced class data

This paper presents research results of our investigation of the imbalanced data problem in the classification of different types of weld flaws, a multi-class classification problem. The one-against-all scheme is adopted to carry out multi-class classification and three algorithms including minimum distance, nearest neighbors, and fuzzy nearest neighbors are employed as the classifiers. The effectiveness of 22 data preprocessing methods for dealing with imbalanced data is evaluated in terms of eight evaluation criteria to determine whether any method would emerge to dominate the others. The test results indicate that: (1) nearest neighbor classifiers outperform the minimum distance classifier; (2) some data preprocessing methods do not improve any criterion and they vary from one classifier to another; (3) the combination of using the AHC_KM data preprocessing method with the 1-NN classifier is the best because they together produce the best performance in six of eight evaluation criteria; and (4) the most difficult weld flaw type to recognize is crack.     

Approaches for the production and evaluation of fuzzy land cover classifications from remotely-sensed data

Remote sensing is an attractive source of data for land cover mapping applications. Mapping is generally achieved through the application of a conventional statistical classification, which allocates each image pixel to a land cover class. Such approaches are inappropriate for mixed pixels, which contain two or more land cover classes, and a fuzzy classification approach is required. When pixels may have multiple and partial class membership measures of the strength of class membership may be output and, if strongly related to the land cover composition, mapped to represent such fuzzy land cover. This type of representation can be derived by softening the output of a conventional ‘hard’ classification or using a fuzzy classification. The accuracy of the representation provided by a fuzzy classification is, however, difficult to evaluate. Conventional measures of classification accuracy cannot be used as they are appropriate only for ‘hard’ classifications. The accuracy of a classification may, however, be indicated by the way in which the strength of class membership is partitioned between the classes and how closely this represents the partitioning of class membership on the ground. In this paper two measures of the closeness of the land cover representation derived from a classification to that on the ground were used to evaluate a set of fuzzy classifications. The latter were based on measures of the strength of class membership output from classifications by a discriminant analysis, artificial neural network and fuzzy c-means classifiers. The results show the importance of recognising and accommodating for the fuzziness of the land cover on the ground. The accuracy assessment methods used were applicable to pure and mixed pixels and enabled the identification of the most accurate land cover representation derived. The results showed that the fuzzy representations were more accurate than the ‘hard’ classifications. Moreover, the outputs derived from the artificial neural network and the fuzzy c-means algorithm in particular were strongly related to the land cover on the ground and provided the most accurate land cover representations. The ability to appropriately represent fuzzy land cover and evaluate the accuracy of the representation should facilitate the use of remote sensing as a source of land cover data.     

Application of a fuzzy classification technique in computer gradingof fish products

This work presents the enhancement and application of a fuzzy classification technique for automated grading of fish products. Common features inherent in grading-type data and their specific requirements in processing for classification are identified. A fuzzy classifier with a four-level hierarchy is developed based on the “generalized K-nearest neighbor rules”. Both conventional and fuzzy classifiers are examined using a realistic set of herring roe data (collected from the fish processing industry) to compare the classification performance in terms of accuracy and computational cost. The classification results show that the generalized fuzzy classifier provides the best accuracy at 89%. The grading system can be tuned through two parameters-the threshold of fuzziness and the cost weighting of error types-to achieve higher classification accuracy. An optimization scheme is also incorporated into the system for automatic determination of these parameter values with respect to a specific optimization function that is based on process renditions, including the product price and labor cost. Since the primary common features are accommodated in the classification algorithm, the method presented here provides a general capability for both grading and sorting-type problems in food processing     

Editorial

This study investigates the potential of applying the radial basis function (RBF) neural network architecture for the classification of multispectral very high spatial resolution satellite images into 13 classes of various scales. For the development of the RBF classifiers, the innovative fuzzy means training algorithm is utilized, which is based on a fuzzy partition of the input space. The method requires only a short amount of time to select both the structure and the parameters of the RBF classifier. The new technique was applied to the area of Lake Kerkini, which is a wetland of great ecological value, located in northern Greece. Eleven experiments were carried out in total in order to investigate the performance of the classifier using different input parameters (spectral and textural) as well as different window sizes and neural network complexities. For comparison purposes the same satellite scene was classified using the maximum likelihood (MLH) classification with the same set of training samples. Overall, the neural network classifiers outperformed the MLH classification by 10–17%, reaching a maximum overall accuracy of 78%. Analysis showed that the selection of input parameters is vital for the success of the classifiers. On the other hand, the incorporation of textural analysis and/or modification of the window size do not affect the performance substantially.