The effect to diagnostic accuracy of decision tree classifier of fuzzy and k-NN based weighted pre-processing methods to diagnosis of erythemato-squamous diseases

This paper presents a novel method for differential diagnosis of erythemato-squamous disease. The proposed method is based on fuzzy weighted pre-processing, k-NN (nearest neighbor) based weighted pre-processing, and decision tree classifier. The proposed method consists of three parts. In the first part, we have used decision tree classifier to diagnosis erythemato-squamous disease. In the second part, first of all, fuzzy weighted pre-processing, which can improved by ours, is a new method and applied to inputs erythemato-squamous disease dataset. Then, the obtained weighted inputs were classified using decision tree classifier. In the third part, k-NN based weighted pre-processing, which can improved by ours, is a new method and applied to inputs erythemato-squamous disease dataset. Then, the obtained weighted inputs were classified via decision tree classifier. The employed decision tree classifier, fuzzy weighted pre-processing decision tree classifier, and k-NN based weighted pre-processing decision tree classifier have reached to 86.18, 97.57, and 99.00% classification accuracies using 20-fold cross validation, respectively.     

Development of a knowledge based decision tree classifier using hybrid polarimetric SAR observables

ABSTRACT

Fast, efficient and accurate classification of the land cover using Synthetic aperture radar (SAR) observables extracted from hybrid polarimetric SAR data is achieved in this research. The proposed knowledge-based tree classifier utilizes little apriori real-time field survey information along with just four features, namely, backscattering coefficient (σ_rh+σ_rv), scattering mechanism (α), diffuse scattering and odd bounce (surface) scattering feature from m- α decomposition method in a sequential manner. We have exploited the Separability Index (SI) criterion for identifying these 4 features among the available 16 features. The overall accuracy (OA), user accuracy (UA), producer accuracy (PA), kappa coefficient (κ), precision (P), recall (R) and score (F) of the proposed classifier underscores its merits. Further, for the sake of fair comparison with the existing approaches, we have built the layer stacked images using these four features and applied them to the supervised maximum likelihood estimator classifier (MLE) as well as the unsupervised k-means classifier. It is found that the proposed classifier has better performance in terms of OA, UA, PA and κ on different SAR data sets consisting of different areas.     

Using k-nn and discriminant analyses to classify rain forest types in a Landsat TM image over northern Costa Rica

Conservation and land use planning in humid tropical lowland forests urgently need accurate remote sensing techniques to distinguish among floristically different forest types. We investigated the degree to which floristically and structurally defined Costa Rican lowland rain forest types can be accurately discriminated by a non-parametric k nearest neighbors (k-nn) classifier or linear discriminant analysis. Pixel values of Landsat Thematic Mapper (TM) image and Shuttle Radar Topography Mission (SRTM) elevation model extracted from segments or from 5 × 5 pixel windows were employed in the classifications. 104 field plots were classified into three floristic and one structural type of forest (regrowth forest). Three floristically defined forest types were formed through clustering the old-growth forest plots (n = 52) by their species specific importance values. An error assessment of the image classification was conducted via cross-validation and error matrices, and overall percent accuracy and Kappa scores were used as measures of accuracy. Image classification of the four forest types did not adequately distinguish two old-growth forest classes, so they were merged into a single forest class. The resulting three forest classes were most accurately classified by the k-nn classifier using segmented image data (overall accuracy 91%). The second best method, with respect to accuracy, was the k-nn with 5 × 5 pixel windows data (89% accuracy), followed by the canonical discriminant analysis using the 5 × 5 pixel window data (86%) and the segment data (82%). We conclude the k-nn classifier can accurately distinguish floristically and structurally different rain forest types. The classification accuracies were higher for the k-nn classifier than for the canonical discriminant analysis, but the differences in Kappa scores were not statistically significant. The segmentation did not increase classification accuracy in this study.     

Evaluation of k-Nearest Neighbor classifier performance for direct marketing

Text data mining is a process of exploratory data analysis. Classification maps data into predefined groups or classes. It is often referred to as supervised learning because the classes are determined before examining the data. This paper describes the proposed k-Nearest Neighbor classifier that performs comparative cross-validation for the existing k-Nearest Neighbor classifier. The feasibility and the benefits of the proposed approach are demonstrated by means of data mining problem: direct marketing. Direct marketing has become an important application field of data mining. Comparative cross-validation involves estimation of accuracy by either stratified k-fold cross-validation or equivalent repeated random subsampling. While the proposed method may have a high bias; its performance (accuracy estimation in our case) may be poor due to a high variance. Thus the accuracy with the proposed k-Nearest Neighbor classifier was less than that with the existing k-Nearest Neighbor classifier, and the smaller the improvement in runtime the larger the improvement in precision and recall. In our proposed method we have determined the classification accuracy and prediction accuracy where the prediction accuracy is comparatively high.     

基于Fuzzy ARTMAP神经网络的高分辨率图象土地覆盖分类及其评价

主要讨论了基于Fuzzy ARTMAP神经网络的高分辨率遥感图象土地覆盖分类方法及其实践．首先介绍了Fuzzy ARTMAP神经网络的原理，然后用SPOT XS图象试验数据进行土地覆盖分类．分类结果与传统的最大似然监督分类(MLC)、反馈式(Back Propagation，BP)神经网络的分类结果进行了比较．通过抽取500个样点对3种分类结果进行精度评价表明，Fuzzy ARTMAP神经网络相对其他两种方法，分类精度均有不同程度的改善，具有更好的分类结果，总分类精度比MLC和BP算法分别提高17．41％、7．32％．最后，对不同分类方法对于土地覆盖分类结果的影响进行了评价和分析．试验表明，Fuzzy ARTMAP神经网络用于高分辨图象土地覆盖分类研究可以获得相对较好的分类结果．     

Some issues in the classification of DAIS hyperspectral data

Classification accuracy depends on a number of factors, of which the nature of the training samples, the number of bands used, the number of classes to be identified relative to the spatial resolution of the image and the properties of the classifier are the most important. This paper evaluates the effects of these factors on classification accuracy using a test area in La Mancha, Spain. High spectral and spatial resolution DAIS data were used to compare the performance of four classification procedures (maximum likelihood, neural network, support vector machines and decision tree). There was no evidence to support the view that classification accuracy inevitably declines as the data dimensionality increases. The support vector machine classifier performed well with all test data sets. The use of the orthogonal MNF transform resulted in a decline in classification accuracy. However, the decision‐tree approach to feature selection worked well. Small increases in classifier accuracy may be obtained using more sophisticated techniques, but it is suggested here that greater attention should be given to the collection of training and test data that represent the range of land surface variability at the spatial scale of the image.     

Compressive sensing based electronic nose platform

Electronic nose (EN) systems play a significant role for gas monitoring and identification in gas plants. Using an EN system which consists of an array of sensors provides a high performance. Nevertheless, this performance is bottlenecked by the high system complexity incorporated with the high number of sensors. In this paper a new EN system is proposed using data sets collected from an in-house fabricated $4\times 4$ tin-oxide gas array sensor. The system exploits the theory of compressive sensing (CS) and distributed compressive sensing (DCS) to reduce the storage capacity and power consumption. The obtained results have shown that compressing the transmitted data to 20% of its original size will preserve the information by achieving a high reconstruction quality. Moreover, exploiting DCS will maintain the same reconstruction quality for just 15% of the original size. This high quality of reconstruction is explored for classification using several classifiers such as decision tree (DT), K-nearest neighbour (KNN) and extended nearest neighbour (ENN) along with linear discrimination analysis (LDA) as feature reduction technique. CS-based reconstructed data has achieved a 95% classification accuracy. Furthermore, DCS-based reconstructed data achieved a 98.33% classification accuracy which is the same as using original data without compression.     

A fast approach for detection of erythemato-squamous diseases based on extreme learning machine with maximum relevance minimum redundancy feature selection

In this paper, a novel hybrid method, which integrates an effective filter maximum relevance minimum redundancy (MRMR) and a fast classifier extreme learning machine (ELM), has been introduced for diagnosing erythemato-squamous (ES) diseases. In the proposed method, MRMR is employed as a feature selection tool for dimensionality reduction in order to further improve the diagnostic accuracy of the ELM classifier. The impact of the type of activation functions, the number of hidden neurons and the size of the feature subsets on the performance of ELM have been investigated in detail. The effectiveness of the proposed method has been rigorously evaluated against the ES disease dataset, a benchmark dataset, from UCI machine learning database in terms of classification accuracy. Experimental results have demonstrated that our method has achieved the best classification accuracy of 98.89% and an average accuracy of 98.55% via 10-fold cross-validation technique. The proposed method might serve as a new candidate of powerful methods for diagnosing ES diseases.     

Wireless electronic nose system for real-time quantitative analysis of gas mixtures using micro-gas sensor array and neuro-fuzzy network

A wireless electronic nose system (WENS) is designed for the real-time quantification of ammonia (NH₃), hydrogen sulfide (H₂S), and their mixtures. The WENS hardware consists of a microcontroller for obtaining measurement data from a micro-gas sensor array, and an RF transceiver for transmitting the data sets to a master sensor node. Meanwhile, the WENS software analyses the binary gas mixtures using a fuzzy ARTMAP classifier and a fuzzy ART-based concentration estimator with multiplicative drift correction based on reference gases. A virtual instrument is developed in the LabVIEW environment for monitoring the analyzed gas mixtures. The performance of the proposed WENS is also assessed and compared with the minimum and product inference methods. The proposed WENS adopting the weighted inference method produces the best concentration estimations as regards the root mean square error.     

A comparison of supervised maximum likelihood and decision tree classification for crop cover estimation from multitemporal LANDSAT MSS data

Abstract

Supervised maximum likelihood classification was compared with a supervised binary decision tree for crop classification from multitemporal LANDSAT MSS data. Similar levels of classification accuracy were obtained using both algorithms, but the ease of training and computational simplicity of the binary decision tree suggest that this algorithm may be a viable alternative to the maximum likelihood for the analysis of data sets with high dimensionality such as multitemporal LANDSAT MSS data.     

An efficient scheme of target classification and information fusion in wireless sensor networks

In this paper, an efficient target classification and fusion scheme for wireless sensor networks (WSNs) is proposed and evaluated. When a classification algorithm for WSN nodes is designed, parametric approaches such as Gaussian mixture model (GMM) should be more preferred to non-parametric ones due to the hard limitation in resources. The GMM algorithm not only shows good performances for target classification in WSNs but it also requires very small resources. Based on the classifier, a decision tree generated by the classification and regression tree algorithm is used to fuse the information from heterogeneous sensors. This node-level classification scheme provides a satisfactory classification rate, 94.10%, with little resources. Finally, a confidence-based fusion algorithm improves the overall accuracy by fusing the information among sensor nodes. Our experimental results show that the proposed group-level fusion algorithm improves the accuracy by an average of 4.17% accuracy with randomly selected nodes.     

Decision tree regression for soft classification of remote sensing data

In recent years, decision tree classifiers have been successfully used for land cover classification from remote sensing data. Their implementation as a per-pixel based classifier to produce hard or crisp classification has been reported in the literature. Remote sensing images, particularly at coarse spatial resolutions, are contaminated with mixed pixels that contain more than one class on the ground. The per-pixel approach may result in erroneous classification of images dominated by mixed pixels. Therefore, soft classification approaches that decompose the pixel into its class constituents in the form of class proportions have been advocated. In this paper, we employ a decision tree regression approach to determine class proportions within a pixel so as to produce soft classification from remote sensing data. Classification accuracy achieved by decision tree regression is compared with those achieved by the most widely used maximum likelihood classifier, implemented in the soft mode, and a supervised version of the fuzzy c-means classifier. Root Mean Square Error (RMSE) and fuzzy error matrix based measures have been used for accuracy assessment of soft classification.     

Pattern recognition for sensor array signals using Fuzzy ARTMAP

A Fuzzy ARTMAP classifier for pattern recognition in chemical sensor array was developed based on Fuzzy Set Theory and Adaptive Resonance Theory. In contrast to most current classifiers with difficulty in detecting new analytes, the Fuzzy ARTMAP system can identify untrained analytes with comparatively high probability. And to detect presence of new analyte, the Fuzzy ARTMAP classifier does not need retraining process that is necessary for most traditional neural network classifiers. In this study, principal component analysis (PCA) was first implemented for feature extraction purpose, followed by pattern recognition using Fuzzy ARTMAP classifiers. To construct the classifier with high recognition rate, parameter sensitive analysis was applied to find critical factors and Pareto optimization was used to locate the optimum parameter setting for the classifier. The test result shows that the proposed method can not only maintain satisfactory correct classification rate for trained analytes, but also be able to detect untrained analytes at a high recognition rate. Also the Pareto optimal values of the most important parameter have been identified, which could help constructing Fuzzy ARTMAP classifiers with good classification performance in future application.     

Combining cluster analysis with classifier ensembles to predict financial distress

The ability to accurately predict business failure is a very important issue in financial decision-making. Incorrect decision-making in financial institutions is very likely to cause financial crises and distress. Bankruptcy prediction and credit scoring are two important problems facing financial decision support. As many related studies develop financial distress models by some machine learning techniques, more advanced machine learning techniques, such as classifier ensembles and hybrid classifiers, have not been fully assessed. The aim of this paper is to develop a novel hybrid financial distress model based on combining the clustering technique and classifier ensembles. In addition, single baseline classifiers, hybrid classifiers, and classifier ensembles are developed for comparisons. In particular, two clustering techniques, Self-Organizing Maps (SOMs) and k-means and three classification techniques, logistic regression, multilayer-perceptron (MLP) neural network, and decision trees, are used to develop these four different types of bankruptcy prediction models. As a result, 21 different models are compared in terms of average prediction accuracy and Type I & II errors. By using five related datasets, combining Self-Organizing Maps (SOMs) with MLP classifier ensembles performs the best, which provides higher predication accuracy and lower Type I & II errors.     

DF-LDA tree: a nonlinear multilevel classifier for pattern recognition

This article presents a new nonlinear classifier by arranging linear classifiers in a tree structure. The proposed classifier, called the direct fractional-step linear discriminant (DF-LDA) tree, adopts a tree structure containing a DF-LDA at each node. The structure of the tree classifier evolves as the training proceeds, so there is no need to decide any parameters as a priori. Due to the many DF-LDAs arranged in the tree structure, classification performance of the proposed classifier is improved over single-shot DF-LDA. The proposed DF-LDA tree is tested on various synthetic and real datasets. Experimental results show that the proposed classifier leads to very satisfactory results in terms of classification accuracy.     

A supervised learning approach for heading detection

As the popularity of the portable document format (PDF) file format increases, research that facilitates PDF text analysis or extraction is necessary. Heading detection is a crucial component of PDF-based text classification processes. This research involves training a supervised learning model to detect headings by systematically testing and selecting classifier features using recursive feature elimination. Results indicate that decision tree is the best classifier with an accuracy of 95.83%, sensitivity of 0.981, and a specificity of 0.946. This research into heading detection contributes to the field of PDF-based text extraction and can be applied to the automation of large scale PDF text analysis in a variety of professional and policy-based contexts.     

A hierarchical multiclass support vector machine incorporated with holistic triple learning units

This paper proposes a new hierarchical learning structure, namely the holistic triple learning (HTL), for extending the binary support vector machine (SVM) to multi-classification problems. For an N-class problem, a HTL constructs a decision tree up to a depth of éN/3ù+1\lceil N/3\rceil+1. A leaf node of the decision tree is allowed to be placed with a holistic triple learning unit whose generalisation abilities are assessed and approved. Meanwhile, the remaining nodes in the decision tree each accommodate a standard binary SVM classifier. The holistic triple classifier is a regression model trained on three classes, whose training algorithm is originated from a recently proposed implementation technique, namely the least-squares support vector machine (LS-SVM). A major novelty with the holistic triple classifier is the reduced number of support vectors in the solution. For the resultant HTL-SVM, an upper bound of the generalisation error can be obtained. The time complexity of training the HTL-SVM is analysed, and is shown to be comparable to that of training the one-versus-one (1-vs.-1) SVM, particularly on small-scale datasets. Empirical studies show that the proposed HTL-SVM achieves competitive classification accuracy with a reduced number of support vectors compared to the popular 1-vs-1 alternative.     

Accurate and reliable diagnosis and classification using probabilistic ensemble simplified fuzzy ARTMAP

In this paper, an accurate and effective probabilistic plurality voting method to combine outputs from multiple simplified fuzzy ARTMAP (SFAM) classifiers is presented. Five ELENA benchmark problems and five medical benchmark data sets have been used to evaluate the applicability and performance of the proposed probabilistic ensemble simplified fuzzy ARTMAP (PESFAM) network. Among the five benchmark problems in ELENA project, PESFAM outperforms the SFAM and multi-layer perceptron (MLP) classifier. In addition, the effectiveness of the proposed PESFAM is delineated in medical diagnosis applications. For the medical diagnosis and classification problems, PESFAM achieves 100 percent in accuracy, specificity, and sensitivity based on the 10-fold crossvalidation and these results are superior to those from other classification algorithms. In addition, a posteri probability of the predicted class can be used to measure the prediction reliability of PESFAM. The experiments demonstrate the potential of the proposed multiple SFAM classifiers in offering an optimal solution to the data-ordering problem of SFAM implementation and also as an intelligent medical diagnosis tool.     

A decision tree algorithm for surface soil freeze/thaw classification over China using SSM/I brightness temperature

This paper reports the development of a decision tree algorithm to classify the surface soil freeze/thaw states. The algorithm uses SSM/I brightness temperatures recorded in the early morning. Three critical indices are used as classification criteria—the scattering index (SI), the 37 GHz vertical polarization brightness temperature (T_37V), and the 19 GHz polarization difference (PD₁₉). The thresholds of these criteria were obtained from samples of frozen soil, thawed soil, desert, and snow. The algorithm is capable of distinguishing between frozen soil, thawed soil, desert, and precipitation. In-situ 4-cm deep soil temperatures on the Qinghai-Tibetan Plateau were used to validate the classification results, and the average classification accuracy was found to be 87%. Regarding the misclassified pixels, about 40% and 73% of them appeared when the surface soil temperature ranged from − 0.5 °C to 0.5 °C and from − 2.0 °C to 2.0 °C, respectively, which means that most misclassifications occurred near the soil freezing point. In addition, misclassifications mainly occurred from April to May and September to October, the transition periods between warm and cold seasons. A grid-to-grid Kappa analysis was also conducted to evaluate the consistency between the map of the actual number of frozen days obtained using the decision tree classification algorithm and the reference map of geocryological regionalization and classification in China. The overall classification accuracy was 91.7%, and the Kappa index was 80.5%. The boundary between the frozen and thawed soil was consistent with the southern limit of seasonally frozen ground from the reference map. The statistics show that the maximum area of frozen soil is about 6.82 × 10⁶ km² in late January, accounting for 69% of total Chinese land area.