Particle swarm optimization-based feature selection in sentiment classification

Sentiment classification is one of the important tasks in text mining, which is to classify documents according to their opinion or sentiment. Documents in sentiment classification can be represented in the form of feature vectors, which are employed by machine learning algorithms to perform classification. For the feature vectors, the feature selection process is necessary. In this paper, we will propose a feature selection method called fitness proportionate selection binary particle swarm optimization (F-BPSO). Binary particle swarm optimization (BPSO) is the binary version of particle swam optimization and can be applied to feature selection domain. F-BPSO is a modification of BPSO and can overcome the problems of traditional BPSO including unreasonable update formula of velocity and lack of evaluation on every single feature. Then, some detailed changes are made on the original F-BPSO including using fitness sum instead of average fitness in the fitness proportionate selection step. The modified method is, thus, called fitness sum proportionate selection binary particle swarm optimization (FS-BPSO). Moreover, further modifications are made on the FS-BPSO method to make it more suitable for sentiment classification-oriented feature selection domain. The modified method is named as SCO-FS-BPSO where SCO stands for “sentiment classification-oriented”. Experimental results show that in benchmark datasets original F-BPSO is superior to traditional BPSO in feature selection performance and FS-BPSO outperforms original F-BPSO. Besides, in sentiment classification domain, SCO-FS-BPSO which is modified specially for sentiment classification is superior to traditional feature selection methods on subjective consumer review datasets.     

Adaptive latent fingerprint segmentation using feature selection and random decision forest classification

Latent fingerprints are important evidences used by law enforcement agencies. However, current state-of-the-art for automatic latent fingerprint recognition is not as reliable as live-scan fingerprints and advancements are required in every step of the recognition pipeline. This research focuses on automatically segmenting latent fingerprints to distinguish between ridge and non-ridge patterns. There are three major contributions of this research: (i) a machine learning algorithm for combining five different categories of features for automatic latent fingerprint segmentation, (ii) a feature selection technique using modified RELIEF formulation for analyzing the influence of multiple category features on latent fingerprint segmentation, and (iii) a novel SIVV based metric to measure the effect of the segmentation algorithm without the requirement to perform the entire matching process. The image is tessellated into local patches and saliency based features along with image, gradient, ridge, and quality based features are extracted. Feature selection is performed to study the contribution of the various category features towards foreground ridge pattern representation. Using these selected features, a trained Random Decision Forest based algorithm classifies the local patches as background or foreground. The results on three publicly available databases demonstrate the efficacy of the proposed algorithm.     

A new hybrid feature selection approach using feature association map for supervised and unsupervised classification

Feature selection, both for supervised as well as for unsupervised classification is a relevant problem pursued by researchers for decades. There are multiple benchmark algorithms based on filter, wrapper and hybrid methods. These algorithms adopt different techniques which vary from traditional search-based techniques to more advanced nature inspired algorithm based techniques. In this paper, a hybrid feature selection algorithm using graph-based technique has been proposed. The proposed algorithm has used the concept of Feature Association Map (FAM) as an underlying foundation. It has used graph-theoretic principles of minimal vertex cover and maximal independent set to derive feature subset. This algorithm applies to both supervised and unsupervised classification. The performance of the proposed algorithm has been compared with several benchmark supervised and unsupervised feature selection algorithms and found to be better than them. Also, the proposed algorithm is less computationally expensive and hence has taken less execution time for the publicly available datasets used in the experiments, which include high-dimensional datasets.     

A machine learning approach for feature selection traffic classification using security analysis

Class imbalance has become a big problem that leads to inaccurate traffic classification. Accurate traffic classification of traffic flows helps us in security monitoring, IP management, intrusion detection, etc. To address the traffic classification problem, in literature, machine learning (ML) approaches are widely used. Therefore, in this paper, we also proposed an ML-based hybrid feature selection algorithm named WMI_AUC that make use of two metrics: weighted mutual information (WMI) metric and area under ROC curve (AUC). These metrics select effective features from a traffic flow. However, in order to select robust features from the selected features, we proposed robust features selection algorithm. The proposed approach increases the accuracy of ML classifiers and helps in detecting malicious traffic. We evaluate our work using 11 well-known ML classifiers on the different network environment traces datasets. Experimental results showed that our algorithms achieve more than 95% flow accuracy results.     

不平衡数据集上的文本分类特征选择新方法*

为了有效消除声发射信号中的噪声,将广义S变换滤波方法应用于声发射信号去噪,分别采用广义S变换中的充零法、基于带通滤波器设计滤波算子法以及时频滤波法进行滤波比较,针对信号的不同时频特性设计了相应的时频滤波算子。结果表明,基于S变换的三种时频滤波法对声发射信号的去噪均有较好的效果,克服了传统滤波方法滤波因子不能随时间、频率变化而变化的缺陷。其中时频滤波法在高信噪比和低信噪比情况下都能更好地去除噪声,可以满足信号处理的要求。     

Study of image retrieval and classification based on adaptive features using genetic algorithm feature selection

This paper proposes a genetic algorithm feature selection (GAFS) for image retrieval systems and image classification. Two texture features of adaptive motifs co-occurrence matrix (AMCOM) and gradient histogram for adaptive motifs (GHAM) and color feature of an adaptive color histogram for K-means (ACH) were used in this paper. In this paper, the feature selections have adopted sequential forward selection (SFS), sequential backward selection (SBS), and genetic algorithms feature selection (GAFS). Image retrieval and classification performance mainly build from three features: ACH, AMCOM and GHAM, where the classification system is used for two-class SVM classification. In the experimental results, we can find that all the methods regarding feature extraction mentioned in this study can contribute to better results with regard to image retrieval and image classification. The GAFS can provide a more robust solution at the expense of increased computational effort. By applying GAFS to image retrieval systems, not only could the number of features be effectively reduced, but higher image retrieval accuracy is elicited.     

An expert model for self-care problems classification using probabilistic neural network and feature selection approach

Self-care problems classification is one of the important challenges for occupational therapists. Extent and variety of disorders make the self-care problems classification process complex and time-consuming. To overcome this challenge, an expert model is proposed innovatively in this research. The proposed model is based on Probabilistic Neural Network (PNN) and Genetic Algorithm (GA) for classifying self-care problems of children with physical and motor disability. In this model, PNN is employed as a classifier and GA is applied for feature selection. The PNN is trained by using a standard ICF-CY dataset. Based on ICF-CY, occupational therapists must evaluate many features to diagnose self-care problems. According to the experiences of occupational therapists, these features have different effects on classification. Hence, GA is employed to select relevant and important features in self-care problems classification. Since the classification rules are important for occupational therapists, the self-care problems classification rules are extracted additionally by using the CART algorithm. The experimental results show that by using the feature selection algorithm, the accuracy and time complexity of classification are improved in comparison to other models. The proposed model can classify self-care problems of children with 94.28% accuracy by using only 16.5% of all features.     

具有特征选择的多源自适应分类框架

对于现有的多源自适应学习方案无法有效区分多个源域中的有用信息并迁移至目标域的问题,提出一种具有特征选择的多源自适应分类框架（MACFFS）,并将特征选择和共享特征子空间学习整合到统一框架中进行联合特征学习。具体来说,MACFFS将来自多个源域的特征数据投影至不同的潜在空间中来学习得到多个源域分类模型,实现目标域的分类。然后,将得到的多个分类结果进行整合用于目标域分类模型的学习。此外,框架还利用L_2,1范数稀疏回归代替传统的基于L₂范数的最小二乘回归来提高鲁棒性。最后,把多种现有方法在两项任务中与MACFFS进行实验比较分析。实验结果表明,与现有方法中表现最好的DSM相比,MACFFS节省了接近1/4的计算时间,并且提升了大约2%的识别率。总的来说,MACFFS结合了机器学习、统计学习等相关知识,为多源自适应方法提供了一个新的思路,且该方法在现实场景下的识别应用中比现有方法具有更好的性能。     

具有特征选择的多源自适应分类框架

对于现有的多源自适应学习方案无法有效区分多个源域中的有用信息并迁移至目标域的问题，提出一种具有特征选择的多源自适应分类框架（MACFFS），并将特征选择和共享特征子空间学习整合到统一框架中进行联合特征学习。具体来说，MACFFS将来自多个源域的特征数据投影至不同的潜在空间中来学习得到多个源域分类模型，实现目标域的分类。然后，将得到的多个分类结果进行整合用于目标域分类模型的学习。此外，框架还利用L_2，1范数稀疏回归代替传统的基于L₂范数的最小二乘回归来提高鲁棒性。最后，把多种现有方法在两项任务中与MACFFS进行实验比较分析。实验结果表明，与现有方法中表现最好的DSM相比，MACFFS节省了接近1/4的计算时间，并且提升了大约2%的识别率。总的来说，MACFFS结合了机器学习、统计学习等相关知识，为多源自适应方法提供了一个新的思路，且该方法在现实场景下的识别应用中比现有方法具有更好的性能。     

A new variance-based approach for discriminative feature extraction in machine hearing classification using spectrogram features

Machine hearing is an emerging research field that is analogous to machine vision in that it aims to equip computers with the ability to hear and recognise a variety of sounds. It is a key enabler of natural human–computer speech interfacing, as well as in areas such as automated security surveillance, environmental monitoring, smart homes/buildings/cities. Recent advances in machine learning allow current systems to accurately recognise a diverse range of sounds under controlled conditions. However doing so in real-world noisy conditions remains a challenging task. Several front–end feature extraction methods have been used for machine hearing, employing speech recognition features like MFCC and PLP, as well as image-like features such as AIM and SIF. The best choice of feature is found to be dependent upon the noise environment and machine learning techniques used. Machine learning methods such as deep neural networks have been shown capable of inferring discriminative classification rules from less structured front–end features in related domains. In the machine hearing field, spectrogram image features have recently shown good performance for noise-corrupted classification using deep neural networks. However there are many methods of extracting features from spectrograms. This paper explores a novel data-driven feature extraction method that uses variance-based criteria to define spectral pooling of features from spectrograms. The proposed method, based on maximising the pooled spectral variance of foreground and background sound models, is shown to achieve very good performance for robust classification.     

Object-based feature selection using class-pair separability for high-resolution image classification

ABSTRACT

With the increasing popularity of object-based image analysis (OBIA) since 2006, numerous classification and mapping tasks were reported to benefit from this evolving paradigm. In these studies, segments are firstly created, followed by classification based on segment-level information. However, the feature space formed by segment-level feature variables can be very large and complex, posing challenges to obtaining satisfactory classification performance. Accordingly, this work attempts to develop a new feature selection approach for segment-level features. Based on the principle of class-pair separability, the segment-level features are grouped according to their types. For each group, the contribution of each segment-level feature to the separation of a pair of classes is quantified. With the information of all feature groups and class pairs, the separability ranking and appearance frequency are considered to compute importance score for each feature. Higher importance score means larger appropriateness to select a feature. By using two Gaofen-2 multi-spectral images, the proposed method is validated. The experimental results show the advantages of the proposed technique over some state-of-the-art feature selection approaches: (1) it can better reduce the number of segment-level features and effectively avoid redundant information; (2) the feature subset obtained by the proposed scheme has good potential to improve classification accuracy.     

Simultaneous feature selection and classification using kernel-penalized support vector machines

We introduce an embedded method that simultaneously selects relevant features during classifier construction by penalizing each feature’s use in the dual formulation of support vector machines (SVM). This approach called kernel-penalized SVM (KP-SVM) optimizes the shape of an anisotropic RBF Kernel eliminating features that have low relevance for the classifier. Additionally, KP-SVM employs an explicit stopping condition, avoiding the elimination of features that would negatively affect the classifier’s performance. We performed experiments on four real-world benchmark problems comparing our approach with well-known feature selection techniques. KP-SVM outperformed the alternative approaches and determined consistently fewer relevant features.     

A Naive Bayes approach for URL classification with supervised feature selection and rejection framework

Web page classification has become a challenging task due to the exponential growth of the World Wide Web. Uniform Resource Locator (URL)‐based web page classification systems play an important role, but high accuracy may not be achievable as URL contains minimal information. Nevertheless, URL‐based classifiers along with rejection framework can be used as a first‐level filter in a multistage classifier, and a costlier feature extraction from contents may be done in later stages. However, noisy and irrelevant features present in URL demand feature selection methods for URL classification. Therefore, we propose a supervised feature selection method by which relevant URL features are identified using statistical methods. We propose a new feature weighting method for a Naive Bayes classifier by embedding the term goodness obtained from the feature selection method. We also propose a rejection framework to the Naive Bayes classifier by using posterior probability for determining the confidence score. The proposed method is evaluated on the Open Directory Project and WebKB data sets. Experimental results show that our method can be an effective first‐level filter. McNemar tests confirm that our approach significantly improves the performance.     

A feature selection bayesian approach for extracting classification rules with a clustering genetic algorithm

This work describes a method that combines a Bayesian feature selection approach with a clustering genetic algorithm to get classification rules in data-mining applications. A Bayesian network is generated from a data set and the Markov blanket of the class variable is applied to the feature subset selection task. The general rule extraction method is simple and consists of employing the clustering process in the examples of each class separately. In this way, clusters of similar examples are found for each class. These clusters can be viewed as subclasses and can, consequently, be modeled into logical rules. In this context, the problem of finding the optimal number of classification rules can be viewed as the problem of finding the best number of clusters. The Clustering Genetic Algorithm can find the best clustering in a data set, according to the Average Silhouette Width criterion, and it was applied to extract classification rules. The proposed methodology is illustrated by means of simulations in three data sets that are benchmarks for data-mining methods--Wisconsin Breast Cancer, Mushroom, and Congressional Voting Records. The rules extracted with all the attributes are compared to those extracted with the features belonging to the Markov blanket and the obtained results show that the proposed method is very promising.     

Ensemble feature selection with the simple Bayesian classification

A popular method for creating an accurate classifier from a set of training data is to build several classifiers, and then to combine their predictions. The ensembles of simple Bayesian classifiers have traditionally not been a focus of research. One way to generate an ensemble of accurate and diverse simple Bayesian classifiers is to use different feature subsets generated with the random subspace method. In this case, the ensemble consists of multiple classifiers constructed by randomly selecting feature subsets, that is, classifiers constructed in randomly chosen subspaces. In this paper, we present an algorithm for building ensembles of simple Bayesian classifiers in random subspaces. The EFS_SBC algorithm includes a hill-climbing-based refinement cycle, which tries to improve the accuracy and diversity of the base classifiers built on random feature subsets. We conduct a number of experiments on a collection of 21 real-world and synthetic data sets, comparing the EFS_SBC ensembles with the single simple Bayes, and with the boosted simple Bayes. In many cases the EFS_SBC ensembles have higher accuracy than the single simple Bayesian classifier, and than the boosted Bayesian ensemble. We find that the ensembles produced focusing on diversity have lower generalization error, and that the degree of importance of diversity in building the ensembles is different for different data sets. We propose several methods for the integration of simple Bayesian classifiers in the ensembles. In a number of cases the techniques for dynamic integration of classifiers have significantly better classification accuracy than their simple static analogues. We suggest that a reason for that is that the dynamic integration better utilizes the ensemble coverage than the static integration.     

Drug/nondrug classification using Support Vector Machines with various feature selection strategies

In conjunction with the advance in computer technology, virtual screening of small molecules has been started to use in drug discovery. Since there are thousands of compounds in early-phase of drug discovery, a fast classification method, which can distinguish between active and inactive molecules, can be used for screening large compound collections. In this study, we used Support Vector Machines (SVM) for this type of classification task. SVM is a powerful classification tool that is becoming increasingly popular in various machine-learning applications. The data sets consist of 631 compounds for training set and 216 compounds for a separate test set. In data pre-processing step, the Pearson's correlation coefficient used as a filter to eliminate redundant features. After application of the correlation filter, a single SVM has been applied to this reduced data set. Moreover, we have investigated the performance of SVM with different feature selection strategies, including SVM–Recursive Feature Elimination, Wrapper Method and Subset Selection. All feature selection methods generally represent better performance than a single SVM while Subset Selection outperforms other feature selection methods. We have tested SVM as a classification tool in a real-life drug discovery problem and our results revealed that it could be a useful method for classification task in early-phase of drug discovery.     

Toward an efficient and scalable feature selection approach for internet traffic classification

There is significant interest in the network management and industrial security community about the need to identify the “best” and most relevant features for network traffic in order to properly characterize user behaviour and predict future traffic. The ability to eliminate redundant features is an important Machine Learning (ML) task because it helps to identify the best features in order to improve the classification accuracy as well as to reduce the computational complexity related to the construction of the classifier. In practice, feature selection (FS) techniques can be used as a preprocessing step to eliminate irrelevant features and as a knowledge discovery tool to reveal the “best” features in many soft computing applications. In this paper, we investigate the advantages and disadvantages of such FS techniques with new proposed metrics (namely goodness, stability and similarity). We continue our efforts toward developing an integrated FS technique that is built on the key strengths of existing FS techniques. A novel way is proposed to identify efficiently and accurately the “best” features by first combining the results of some well-known FS techniques to find consistent features, and then use the proposed concept of support to select a smallest set of features and cover data optimality. The empirical study over ten high-dimensional network traffic data sets demonstrates significant gain in accuracy and improved run-time performance of a classifier compared to individual results produced by some well-known FS techniques.