A Generalized Functional Network for a Classifier‐Quantifiers Scheme in a Gas‐Sensing System

This paper discusses a new computational scheme based on functional networks and applies it to the problem of classification and quantification of gas species in a mixture. A generalized functional network as a new classifier is proposed to improve the potentialities of the standard functional network classifier. Both methodology and learning algorithm are derived. The performance of this new classifier is examined by using experimental applications. A comparative study with the most common classification algorithms is carried out by showing the high‐quality performance of the proposed classifier. The classifier interacts with some quantifiers, again based on functional networks and finite differences. The scheme of the quantifiers was previously proposed for single gas exposure applications and is here extended to the multigas case. Numerical results show that our approach behaves quite satisfactorily.     

Change Detection Using Adaptive Fuzzy Neural Networks: Environmental Damage Assessment after the Gulf War

This article introduces an adaptive fuzzy neural network classifier for environmental change detection and classification applied to monitor landcover changes resulting from the Gulf War. In this study, landcover change is treated as a qualitative shift between landcover categories. The Change Detection Adaptive Fuzzy (CDAF) network learns fuzzy membership functions for each landcover class present at the first image date based on a sample of the image data. An image from a later date is then classified using this network to recognize change among familiar classes as well as change to unfamiliar landcover classes. The CDAF network predicts landcover change with 86% accuracy representing an improvement over both a standard multidate K-means technique which performed at 70% accuracy and a hybrid approach using a maximum likelihood classifier (MLC)/K-means which achieved 65% accuracy. In this study, we developed a hybrid classifier based on conventional statistical methods (MLC/K-means classifier) for comparison purposes to help evaluate the performance of the CDAF network. The CDAF compared with existing change detection methodology has two features that lead to significant performance improvements: 1) new landcover types created by a change event automatically lead to the establishment of new landcover categories through an unsupervised learning strategy, and 2) for each pixel the distribution of fuzzy membership values across possible categories are compared to determine whether a significant change has occurred.     

Aerial scene classification via an ensemble extreme learning machine classifier based on discriminative hybrid convolutional neural networks features

Identifying a discriminative feature can effectively improve the classification performance of aerial scene classification. Deep convolutional neural networks (DCNN) have been widely used in aerial scene classification for its learning discriminative feature ability. The DCNN feature can be more discriminative by optimizing the training loss function and using transfer learning methods. To enhance the discriminative power of a DCNN feature, the improved loss functions of pretraining models are combined with a softmax loss function and a centre loss function. To further improve performance, in this article, we propose hybrid DCNN features for aerial scene classification. First, we use DCNN models with joint loss functions and transfer learning from pretrained deep DCNN models. Second, the dense DCNN features are extracted, and the discriminative hybrid features are created using linear connection. Finally, an ensemble extreme learning machine (EELM) classifier is adopted for classification due to its general superiority and low computational cost. Experimental results based on the three public benchmark data sets demonstrate that the hybrid features obtained using the proposed approach and classified by the EELM classifier can result in remarkable performance.     

动态置信度的序列选择增量学习方法

贝叶斯在训练样本不完备的情况下,对未知类别新增训练集进行增量学习时,会将分类错误的训练样本过早地加入到分类器中而降低其性能,另外增量学习采用固定的置信度评估参数会使其效率低下,泛化性能不稳定.为解决上述问题,提出一种动态置信度的序列选择增量学习方法.首先,在现有的分类器基础上选出分类正确的文本组成新增训练子集.其次,利用置信度动态监控分类器性能来对新增训练子集进行批量实例选择.最后,通过选择合理的学习序列来强化完备数据的积极影响,弱化噪声数据的消极影响,并实现对测试文本的分类.实验结果表明,本文提出的方法在有效提高分类精度的同时也能明显改善增量学习效率.     

Prediction of disorder with new computational tool: BVDEA

Recognizing that many intrinsically disordered regions in proteins play key roles in vital functions and also in some diseases, identification of the disordered regions has became a demanding process for structure prediction and functional characterization of proteins. Therefore, many studies have been motivated on accurate prediction of disorder. Mostly, machine learning techniques have been used for dealing with the prediction problem of disorder due to the capability of extracting the complex relationships and correlations hidden in large data sets. In this study, a novel method, named Border Vector Detection and Extended Adaptation (BVDEA) was developed for predicting disorder as an alternative accurate classifier. The classifier performs the predictions by using three types of structural features belonging to proteins. For attesting the performance of the method, three computational learning techniques and eleven specific tools were used for comparison. Training was executed based on the data by 5-fold cross validation. When compared with the two learning methods of LVQ and BVDA, the proposed method gives the best success on classification. The BVDEA also provides faster and more robust learning as compared to the others. The new method provides a significant contribution to predicting disorder and order regions of proteins.     

Active learning with adaptive regularization

In classification problems, active learning is often adopted to alleviate the laborious human labeling efforts, by finding the most informative samples to query the labels. One of the most popular query strategy is selecting the most uncertain samples for the current classifier. The performance of such an active learning process heavily relies on the learned classifier before each query. Thus, stepwise classifier model/parameter selection is quite critical, which is, however, rarely studied in the literature. In this paper, we propose a novel active learning support vector machine algorithm with adaptive model selection. In this algorithm, before each new query, we trace the full solution path of the base classifier, and then perform efficient model selection using the unlabeled samples. This strategy significantly improves the active learning efficiency with comparatively inexpensive computational cost. Empirical results on both artificial and real world benchmark data sets show the encouraging gains brought by the proposed algorithm in terms of both classification accuracy and computational cost.     

Graph-based semisupervised learning

Graph-based learning provides a useful approach for modeling data in classification problems. In this modeling scenario, the relationship between labeled and unlabeled data impacts the construction and performance of classifiers, and therefore a semi-supervised learning framework is adopted. We propose a graph classifier based on kernel smoothing. A regularization framework is also introduced, and it is shown that the proposed classifier optimizes certain loss functions. Its performance is assessed on several synthetic and real benchmark data sets with good results, especially in settings where only a small fraction of the data are labeled.     

基于多视图的半监督集成学习方法

针对集成学习方法中分类器差异性不足以及已标记样本少的问题,提出了一种新的半监督集成学习算法,将半监督方法引入到集成学习中,利用大量未标记样本的信息来细化每个基分类器,并且构造差异性更大的基分类器,首先通过多视图方法选取合适的未标记样本,并使用多视图方法将大量繁杂的特征属性分类,使用不同的特征降维方法对不同的视图进行降维...     

Support vector learning for fuzzy rule-based classification systems

To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, the support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet a certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.     

Regularized margin-based conditional log-likelihood loss for prototype learning

The classification performance of nearest prototype classifiers largely relies on the prototype learning algorithm. The minimum classification error (MCE) method and the soft nearest prototype classifier (SNPC) method are two important algorithms using misclassification loss. This paper proposes a new prototype learning algorithm based on the conditional log-likelihood loss (CLL), which is based on the discriminative model called log-likelihood of margin (LOGM). A regularization term is added to avoid over-fitting in training as well as to maximize the hypothesis margin. The CLL in the LOGM algorithm is a convex function of margin, and so, shows better convergence than the MCE. In addition, we show the effects of distance metric learning with both prototype-dependent weighting and prototype-independent weighting. Our empirical study on the benchmark datasets demonstrates that the LOGM algorithm yields higher classification accuracies than the MCE, generalized learning vector quantization (GLVQ), soft nearest prototype classifier (SNPC) and the robust soft learning vector quantization (RSLVQ), and moreover, the LOGM with prototype-dependent weighting achieves comparable accuracies to the support vector machine (SVM) classifier.     

Fuzzy classifier design using genetic algorithms

A new method for design of a fuzzy-rule-based classifier using genetic algorithms (GAs) is discussed. The optimal parameters of the fuzzy classifier including fuzzy membership functions and the size and structure of fuzzy rules are extracted from the training data using GAs. This is done by introducing new representation schemes for fuzzy membership functions and fuzzy rules. An effectiveness measure for fuzzy rules is developed that allows for systematic addition or deletion of rules during the GA optimization process. A clustering method is utilized for generating new rules to be added when additions are required. The performance of the classifier is tested on two real-world databases (Iris and Wine) and a simulated Gaussian database. The results indicate that highly accurate classifiers could be designed with relatively few fuzzy rules. The performance is also compared to other fuzzy classifiers tested on the same databases.     

On the use of evolutionary feature selection for improving fuzzy rough set based prototype selection

The k-nearest neighbors classifier is a widely used classification method that has proven to be very effective in supervised learning tasks. In this paper, a fuzzy rough set method for prototype selection, focused on optimizing the behavior of this classifier, is presented. The hybridization with an evolutionary feature selection method is considered to further improve its performance, obtaining a competent data reduction algorithm for the 1-nearest neighbors classifier. This hybridization is performed in the training phase, by using the solution of each preprocessing technique as the starting condition of the other one, within a cycle. The results of the experimental study, which have been contrasted through nonparametric statistical tests, show that the new hybrid approach obtains very promising results with respect to classification accuracy and reduction of the size of the training set.     

一种基于粗糙集的网页分类方法

Internet的迅速发展带来了一个新的问题，如何有效，迅速地从浩瀚的Web网页中找到所需要的信息，机器学习的发展给这个问题的解决提供了一个新的方向，本文将粗糙集理论应用于网页分类，提出了一种基于粗糙集的决策表约简的增量式学习算法，并利用该算法实现了一个Web网页的分类器，实验结果表明该分类器具有良好的性能。     

Applying a new localized generalization error model to design neural networks trained with extreme learning machine

High accuracy and low overhead are two key features of a well-designed classifier for different classification scenarios. In this paper, we propose an improved classifier using a single-hidden layer feedforward neural network (SLFN) trained with extreme learning machine. The novel classifier first utilizes principal component analysis to reduce the feature dimension and then selects the optimal architecture of the SLFN based on a new localized generalization error model in the principal component space. Experimental and statistical results on the NSL-KDD data set demonstrate that the proposed classifier can achieve a significant performance improvement compared with previous classifiers.     

Ensemble learning for independent component analysis

It is well known that the applicability of independent component analysis (ICA) to high-dimensional pattern recognition tasks such as face recognition often suffers from two problems. One is the small sample size problem. The other is the choice of basis functions (or independent components). Both problems make ICA classifier unstable and biased. In this paper, we propose an enhanced ICA algorithm by ensemble learning approach, named as random independent subspace (RIS), to deal with the two problems. Firstly, we use the random resampling technique to generate some low dimensional feature subspaces, and one classifier is constructed in each feature subspace. Then these classifiers are combined into an ensemble classifier using a final decision rule. Extensive experimentations performed on the FERET database suggest that the proposed method can improve the performance of ICA classifier.     

Measure oriented training: a targeted approach to imbalanced classification problems

Since the overall prediction error of a classifier on imbalanced problems can be potentially misleading and biased, alternative performance measures such as G-mean and F-measure have been widely adopted. Various techniques including sampling and cost sensitive learning are often employed to improve the performance of classifiers in such situations. However, the training process of classifiers is still largely driven by traditional error based objective functions. As a result, there is clearly a gap between themeasure according to which the classifier is evaluated and how the classifier is trained. This paper investigates the prospect of explicitly using the appropriate measure itself to search the hypothesis space to bridge this gap. In the case studies, a standard threelayer neural network is used as the classifier, which is evolved by genetic algorithms (GAs) with G-mean as the objective function. Experimental results on eight benchmark problems show that the proposed method can achieve consistently favorable outcomes in comparison with a commonly used sampling technique. The effectiveness of multi-objective optimization in handling imbalanced problems is also demonstrated.     

基于网络与统计方法的集成分类器及其应用

分析了各单一方法的分类性能，提出了神经网络与统计方法相集成的策略，由此提出MLFN-CCA-Fisher集成分类器。通过网络的自适应学习，将原样本模式经加权S型变换，映射到新的模式空间，能被线性分类，然后用相关成分分析方法提取特征，再建立Fisher判别模型。在性能测试与实际应用中，集成分类器均取得了良好的效果。     

Automatic rule learning using roughset for fuzzy classifier in fault categorization of mono-block centrifugal pump

Rule learning based approach to fault detection and diagnosis is becoming very popular, mainly due to their high accuracy when compared to older statistical methods. Fault detection and diagnosis of various mechanical components of centrifugal pump is essential to increase the productivity and reduce the breakdowns. This paper presents the use of rough sets to generate the rules from statistical features extracted from vibration signals under good and faulty conditions of a centrifugal pump. A fuzzy inference system (FIS) is built using rough set rules and tested using test data. The effect of different types of membership functions on the FIS performance is also presented. Finally, the performance of this classifier is compared to that of a fuzzy-antminer classifier and to multi-layer perceptron (MLP) based classifiers.     

A novel PBL-McRBFN-RFE approach for identification of critical brain regions responsible for Parkinson’s disease

In this paper, we present a novel approach for the identification of critical brain regions responsible for Parkinson’s disease (PD) based on magnetic resonance images (MRI) using meta-cognitive radial basis function network (McRBFN) classifier with Recursive Feature Elimination (RFE). The McRBFN classifier uses voxel based morphometric (VBM) features extracted from MRI and employs a projection based learning (PBL) algorithm. The meta-cognitive learning present in PBL-McRBFN helps in selecting proper samples to learn based on its current knowledge and evolve the architecture automatically. Since, the classifier developed using PBL-McRBFN is efficient, we propose recursive feature elimination approach (called PBL-McRBFN-RFE) to identify most relevant brain regions responsible for PD prediction.The study has been conducted using the Parkinson’s Progression Markers Initiative (PPMI) data set. First, we conducted the study on PD prediction using the PBL-McRBFN classifier on the PPMI data set. We have also compared the results of the PBL-McRBFN classifier with the support vector machine (SVM) classifier. The study results clearly show that the PBL-McRBFN classifier produces better generalization performance on PD prediction. Finally, we use RFE approach with PBL-McRBFN to identify the brain regions responsible for PD. The PBL-McRBFN-RFE selected features indicate that the loss of gray matter in the superior temporal gyrus region may be responsible for the onset of PD, and is consistent with the earlier findings from medical research studies.