Correlation feature selection based improved-Binary Particle Swarm Optimization for gene selection and cancer classification

DNA microarray technology has emerged as a prospective tool for diagnosis of cancer and its classification. It provides better insights of many genetic mutations occurring within a cell associated with cancer. However, thousands of gene expressions measured for each biological sample using microarray pose a great challenge. Many statistical and machine learning methods have been applied to get most relevant genes prior to cancer classification. A two phase hybrid model for cancer classification is being proposed, integrating Correlation-based Feature Selection (CFS) with improved-Binary Particle Swarm Optimization (iBPSO). This model selects a low dimensional set of prognostic genes to classify biological samples of binary and multi class cancers using Naive–Bayes classifier with stratified 10-fold cross-validation. The proposed iBPSO also controls the problem of early convergence to the local optimum of traditional BPSO. The proposed model has been evaluated on 11 benchmark microarray datasets of different cancer types. Experimental results are compared with seven other well known methods, and our model exhibited better results in terms of classification accuracy and the number of selected genes in most cases. In particular, it achieved up to 100% classification accuracy for seven out of eleven datasets with a very small sized prognostic gene subset (up to <1.5%) for all eleven datasets.     

Cancer classification from serial analysis of gene expression with event models

Cancer class prediction and discovery is beneficial to imperfect non-automated cancer diagnoses which affect patient cancer treatments. Serial Analysis of Gene Expression (SAGE) is a relatively new method for monitoring gene expression levels and is expected to contribute significantly to the progress in cancer treatment by enabling an automatic, precise and early diagnosis. A promising application of SAGE gene expression data is classification of cancers. In this paper, we build three event models (the multivariate Bernoulli model, the multinomial model and the normalized multinomial model) for SAGE gene expression profiles. The event models based methods are compared with the standard Naïve Bayes method. Both binary classification and multicategory classification are investigated. Experiments results on several SAGE datasets show that event models are better than standard Naïve Bayes in general. Normalized Information Gain (NIG), an extension of Information Gain (IG), is proposed for gene selection. The impact of gene correlation on the classification performance is investigated.     

基于统计方法的面向旅游问句分类实验研究

根据旅游领域知识的特点,设计了面向旅游问句的分类体系。利用信息增益、互信息、交叉熵和X^2统计四种特征选择方法及支持向量机分类器,对网上常见的旅游真实问句分类进行了实验研究,实验结果表明：在现有问句分类体系下,信息增益的特征选择方法在特征空间维数为550维时,分类旅游问句的结果是最佳的。     

SVM-ABC based cancer microarray (gene expression) hybrid method for data classification

Microarray technology presents a challenge due to the large dimensionality of the data, which can be difficult to interpret. To address this challenge, the article proposes a feature extraction-based cancer classification technique coupled with artificial bee colony optimization (ABC) algorithm. The ABC-support vector machine (SVM) method is used to classify the lung cancer datasets and compared them with existing techniques in terms of precision, recall, F-measure, and accuracy. The proposed ABC-SVM has the advantage of dealing with complex nonlinear data, providing good flexibility. Simulation analysis was conducted with 30% of the data reserved for testing the proposed method. The results indicate that the proposed attribute classification technique, which uses fewer genes, performs better than other modalities. The classifiers, such as naïve Bayes, multi-class SVM, and linear discriminant analysis, were also compared and the proposed method outperformed these classifiers and state-of-the-art techniques. Overall, this study demonstrates the potential of using intelligent algorithms and feature extraction techniques to improve the accuracy of cancer diagnosis using microarray gene expression data.     

基于内容的垃圾短信过滤

研究一种基于最小风险贝叶斯决策的垃圾短信过滤方法。对于以文本信息为主的短信,采用信息增益的方法进行特征选择,使用基于最小风险贝叶斯决策方法进行分类。通过自建短信语料库对该方法进行了实验。实验结果表明,该方法能够准确地对短信进行分类,降低合法短信的分类错误率,分类正确率达到99.3%,符合了短信分类要求。     

深度可分离卷积在Android恶意软件分类的应用研究

传统机器学习在恶意软件分析上需要复杂的特征工程,不适用于大规模的恶意软件分析。为提高在Android恶意软件上的检测效率,将Android恶意软件字节码文件映射成灰阶图像,综合利用深度可分离卷积（depthwise separable convolution,DSC）和注意力机制提出基于全局注意力模块（GCBAM）的Android恶意软件分类模型。从APK文件中提取字节码文件,将字节码文件转换为对应的灰阶图像,通过构建基于GCBAM的分类模型对图像数据集进行训练,使其具有Android恶意软件分类能力。实验表明,该模型对Android恶意软件家族能有效分类,在获取的7 630个样本上,分类准确率达到98.91%,相比机器学习算法在准确率、召回率等均具有较优效果。     

Genetic Programming with a Genetic Algorithm for Feature Construction and Selection

The use of machine learning techniques to automatically analyse data for information is becoming increasingly widespread. In this paper we primarily examine the use of Genetic Programming and a Genetic Algorithm to pre-process data before it is classified using the C4.5 decision tree learning algorithm. Genetic Programming is used to construct new features from those available in the data, a potentially significant process for data mining since it gives consideration to hidden relationships between features. A Genetic Algorithm is used to determine which such features are the most predictive. Using ten well-known datasets we show that our approach, in comparison to C4.5 alone, provides marked improvement in a number of cases. We then examine its use with other well-known machine learning techniques.     

Cloud computing-based parallel genetic algorithm for gene selection in cancer classification

Cancer classification is one of the main steps during patient healing process. This fact enforces modern clinical researchers to use advanced bioinformatics methods for cancer classification. Cancer classification is usually performed using gene expression data gained in microarray experiment and advanced machine learning methods. Microarray experiment generates huge amount of data, and its processing via machine learning methods represents a big challenge. In this study, two-step classification paradigm which merges genetic algorithm feature selection and machine learning classifiers is utilized. Genetic algorithm is built in MapReduce programming spirit which makes this algorithm highly scalable for Hadoop cluster. In order to improve the performance of the proposed algorithm, it is extended into a parallel algorithm which process on microarray data in distributed manner using the Hadoop MapReduce framework. In this paper, the algorithm was tested on eleven GEMS data sets (9 tumors, 11 tumors, 14 tumors, brain tumor 1, lung cancer, brain tumor 2, leukemia 1, DLBCL, leukemia 2, SRBCT, and prostate tumor) and its accuracy reached 100% for less than 25 selected features. The proposed cloud computing-based MapReduce parallel genetic algorithm performed well on gene expression data. In addition, the scalability of the suggested algorithm is unlimited because of underlying Hadoop MapReduce platform. The presented results indicate that the proposed method can be effectively implemented for real-world microarray data in the cloud environment. In addition, the Hadoop MapReduce framework demonstrates substantial decrease in the computation time.

     

Can classification performance be predicted by complexity measures? A study using microarray data

Data complexity analysis enables an understanding of whether classification performance could be affected, not by algorithm limitations, but by intrinsic data characteristics. Microarray datasets based on high numbers of gene expressions combined with small sample sizes represent a particular challenge for machine learning researchers. This type of data also has other particularities that may negatively affect the generalization capacity of classifiers, such as overlaps between classes and class imbalance. Making use of several complexity measures, we analyzed the intrinsic complexity of several microarray datasets with and without feature selection and then explored the connection with the empirical results obtained by four widely used classifiers. Experimental results for 21 binary and multiclass datasets demonstrate that a correlation exists between microarray data complexity and the classification error rates.     

肿瘤识别过程中特征基因的选取

基于肿瘤基因表达数据,运用信息科学的方法和技术建立肿瘤的预测分类模型,对肿瘤的识别具有重要意义。在建立模型的过程中,如何能够有效地排除噪声基因进而挑选出分类特征基因对肿瘤预测的准确性有很大的影响。针对该类问题,提出了一种新的特征基因选取方法—CLUSTER_S2N法。该方法采取了“信噪比”指标与聚类相结合的方法来挑选特征基因,并分别以前列腺癌和急性白血病的基因表达谱为例,用支持向量机作为分类器进行了肿瘤的分类预测实验。实验结果表明该方法的可行性。     

A multiscale feature fusion approach for classification of very high resolution satellite imagery based on wavelet transform

A novel methodology based on multiscale spectral and spatial information fusion using wavelet transform is proposed in order to classify very high resolution (VHR) satellite imagery. Conventional wavelet‐based feature extraction methods employ single windows of a fixed size, which are not satisfactory as the VHR imagery contains complex and multiscale objects. In this paper, spectral and spatial features are extracted based on a set of concentric windows around a central pixel in order to integrate the information across different windows/scales. The proposed method is made up of three blocks: (1) the conventional wavelet‐based feature extraction methods are extended from single band processing to multispectral bands, and from single window to multi‐windows, (2) two multiscale fusion algorithms are proposed to exploit the multiscale spectral and spatial information and (3) a support vector machine (SVM), a relatively new method of machine learning, is used to classify the multiscale spectral–spatial feature sets. The proposed classification method is evaluated on two VHR datasets and the results show that the multiscale approach can improve the classification accuracy in homogeneous areas while simultaneously preserving accuracy in edge regions.     

一种基于微阵列数据的集成分类方法*

针对现有的微阵列数据集成分类方法分类精度不高这一问题,提出了一种Bagging-PCA-SVM方法。该方法首先采用Bootstrap技术对训练样本集重复取样,构成大量训练样本子集,然后在每个子集上进行特征选择和主成分分析以消除噪声基因与冗余基因;最后利用支持向量机作为分类器,采用多数投票的方法预测样本的类属。通过三个数据集进行了测试,测试结果表明了该方法的有效性和可行性。     

Ensemble Rough Hypercuboid Approach for Classifying Cancers

Cancer classification is the critical basis for patient-tailored therapy. Conventional histological analysis tends to be unreliable because different tumors may have similar appearance. The advances in microarray technology make individualized therapy possible. Various machine learning methods can be employed to classify cancer tissue samples based on microarray data. However, few methods can be elegantly adopted for generating accurate and reliable as well as biologically interpretable rules. In this paper, we introduce an approach for classifying cancers based on the principle of minimal rough fringe. For training rough hypercuboid classifiers from gene expression data sets, the method dynamically evaluates all available genes and sifts the genes with the smallest implicit regions as the dimensions of implicit hypercuboids. An unseen object is predicted to be a certain class if it falls within the corresponding class hypercuboid. Based upon the method, ensemble rough hypercuboid classifiers are subsequently constructed. Experimental results on some open cancer gene expression data sets show that the proposed method is capable of generating accurate and interpretable rules compared with some other machine learning methods. Hence, it is a feasible way of classifying cancer tissues in biomedical applications.     

Cancer classification by gradient LDA technique using microarray gene expression data

Cancer classification is one of the major applications of the microarray technology. When standard machine learning techniques are applied for cancer classification, they face the small sample size (SSS) problem of gene expression data. The SSS problem is inherited from large dimensionality of the feature space (due to large number of genes) compared to the small number of samples available. In order to overcome the SSS problem, the dimensionality of the feature space is reduced either through feature selection or through feature extraction. Linear discriminant analysis (LDA) is a well-known technique for feature extraction-based dimensionality reduction. However, this technique cannot be applied for cancer classification because of the singularity of the within-class scatter matrix due to the SSS problem. In this paper, we use Gradient LDA technique which avoids the singularity problem associated with the within-class scatter matrix and shown its usefulness for cancer classification. The technique is applied on three gene expression datasets; namely, acute leukemia, small round blue-cell tumour (SRBCT) and lung adenocarcinoma. This technique achieves lower misclassification error as compared to several other previous techniques.     

Classifier ensemble construction with rotation forest to improve medical diagnosis performance of machine learning algorithms

Improving accuracies of machine learning algorithms is vital in designing high performance computer-aided diagnosis (CADx) systems. Researches have shown that a base classifier performance might be enhanced by ensemble classification strategies. In this study, we construct rotation forest (RF) ensemble classifiers of 30 machine learning algorithms to evaluate their classification performances using Parkinson's, diabetes and heart diseases from literature.While making experiments, first the feature dimension of three datasets is reduced using correlation based feature selection (CFS) algorithm. Second, classification performances of 30 machine learning algorithms are calculated for three datasets. Third, 30 classifier ensembles are constructed based on RF algorithm to assess performances of respective classifiers with the same disease data. All the experiments are carried out with leave-one-out validation strategy and the performances of the 60 algorithms are evaluated using three metrics; classification accuracy (ACC), kappa error (KE) and area under the receiver operating characteristic (ROC) curve (AUC).Base classifiers succeeded 72.15%, 77.52% and 84.43% average accuracies for diabetes, heart and Parkinson's datasets, respectively. As for RF classifier ensembles, they produced average accuracies of 74.47%, 80.49% and 87.13% for respective diseases.RF, a newly proposed classifier ensemble algorithm, might be used to improve accuracy of miscellaneous machine learning algorithms to design advanced CADx systems.     

Breast Cancer Diagnosis Using Feature Selection Approaches and Bayesian Optimization

Breast cancer seriously affects many women. If breast cancer is detected at an early stage, it may be cured. This paper proposes a novel classification model based improved machine learning algorithms for diagnosis of breast cancer at its initial stage. It has been used by combining feature selection and Bayesian optimization approaches to build improved machine learning models. Support Vector Machine, K-Nearest Neighbor, Naive Bayes, Ensemble Learning and Decision Tree approaches were used as machine learning algorithms. All experiments were tested on two different datasets, which are Wisconsin Breast Cancer Dataset (WBCD) and Mammographic Breast Cancer Dataset (MBCD). Experiments were implemented to obtain the best classification process. Relief, Least Absolute Shrinkage and Selection Operator (LASSO) and Sequential Forward Selection were used to determine the most relevant features, respectively. The machine learning models were optimized with the help of Bayesian optimization approach to obtain optimal hyperparameter values. Experimental results showed the unified feature selection-hyperparameter optimization method improved the classification performance in all machine learning algorithms. Among the various experiments, LASSO-BO-SVM showed the highest accuracy, precision, recall and F1-score for two datasets (97.95%, 98.28%, 98.28%, 98.28% for MBCD and 98.95%, 97.17%, 100%, 98.56% for MBCD), yielding outperforming results compared to recent studies.     

Feature Interaction Maximisation

Feature selection plays an important role in classification algorithms. It is particularly useful in dimensionality reduction for selecting features with high discriminative power. This paper introduces a new feature-selection method called Feature Interaction Maximisation (FIM), which employs three-way interaction information as a measure of feature redundancy. It uses a forward greedy search to select features which have maximum interaction information with the features already selected, and which provide maximum relevance. The experiments conducted to verify the performance of the proposed method use three datasets from the UCI repository. The method is compared with four other well-known feature-selection methods: Information Gain (IG), Minimum Redundancy Maximum Relevance (mRMR), Double Input Symmetrical Relevance (DISR), and Interaction Gain Based Feature Selection (IGFS). The average classification accuracy of two classifiers, Naïve Bayes and K-nearest neighbour, is used to assess the performance of the new feature-selection method. The results show that FIM outperforms the other methods.     

基于轻量级迁移学习的无人机航拍视频图像天气场景分类

针对传统航拍视频图像CNN模型天气分类效果差、无法满足移动设备应用以及现有天气图像数据集匮乏且场景单一的问题, 构建了晴天、雨天、雪天、雾天4类面向多场景的无人机航拍天气图像数据集, 并提出了基于轻量级迁移学习的无人机航拍视频图像天气场景分类模型. 该模型采用迁移学习的方法, 在ImageNet数据集上训练好两种轻量级CNN, 并设计3个轻量级CNN分支进行特征提取. 特征提取首先采用ECANet注意力机制改进的EfficientNet-b0作为主分支提取整幅图像特征, 并使用两个MobileNetv2分支分别对天空和非天空局部独有的深层特征进行提取. 其次, 通过Concatenate将这3个区域进行特征融合. 最后, 使用Softmax层对4类天气场景实现分类. 实验结果表明, 该方法应用于移动等计算受限设备时对于天气场景分类的识别准确率达到了97.3%, 有着较好的分类效果.     

Testing Different Ensemble Configurations for Feature Selection

In recent years, ensemble learning has become a prolific area of study in pattern recognition, based on the assumption that using and combining different learning models in the same problem could lead to better performance results than using a single model. This idea of ensemble learning has traditionally been used for classification tasks, but has more recently been adapted to other machine learning tasks such as clustering and feature selection. We propose several feature selection ensemble configurations based on combining rankings of features from individual rankers according to the combination method and threshold value used. The performance of each proposed ensemble configuration was tested for synthetic datasets (to assess the adequacy of the selection), real classical datasets (with more samples than features), and DNA microarray datasets (with more features than samples). Five different classifiers were studied in order to test the suitability of the proposed ensemble configurations and assess the results.     

Classification of bioinformatics dataset using finite impulse response extreme learning machine for cancer diagnosis

In this paper, the classification of the two binary bioinformatics datasets, leukemia and colon tumor, is further studied by using the recently developed neural network-based finite impulse response extreme learning machine (FIR-ELM). It is seen that a time series analysis of the microarray samples is first performed to determine the filtering properties of the hidden layer of the neural classifier with FIR-ELM for feature identification. The linear separability of the data patterns in the microarray datasets is then studied. For improving the robustness of the neural classifier against noise and errors, a frequency domain gene feature selection algorithm is also proposed. It is shown in the simulation results that the FIR-ELM algorithm has an excellent performance for the classification of bioinformatics data in comparison with many existing classification algorithms.