Multiple classifier systems for classification of multifrequency PolSAR images with limited training samples

ABSTRACT

The aim of this article is to improve land-cover classification accuracy from multifrequency full-polarimetric synthetic aperture radar (PolSAR) observations using multiple classifier systems (MCSs) when limited training samples are available. Two types of popular MCSs, tree-based MCSs and neural-based MCSs, were compared with individual decision tree (DT) and neural network methods. Moreover, an objective majority voting (OMV) was proposed and compared with majority voting (MV) and weighted MV (WMV) to fuse the results of the MCSs. Experimental tests were performed on three benchmark PolSAR data sets with different frequencies (X, C, and L) over the San Francisco Bay, CA. The results indicated (1) tree-based MCSs and neural-based MCSs, in general, produced higher overall, producer?s and user?s accuracies than the related individual methods, i.e. DT and NN, with limited training samples; (2) tree-based MCSs were also often more accurate and much faster than neural-based MCSs; (3) regarding robustness, among the MCSs, random forest showed higher stability while bagging showed lower stability in the classification of three PolSAR data sets; (4) the OMV proposed in this article usually outperformed its competitors, i.e. MV and WMV; (5) the results obtained by the methods from the C-band data set were more accurate and more reliable than those obtained from the X- and L-band data sets.     

Predicting the Results of RNA Molecular Specific Hybridization Using Machine Learning

Ribonucleic acid (RNA) hybridization is widely used in popular RNA simulation software in bioinformatics. However, limited by the exponential computational complexity of combinatorial problems, it is challenging to decide, within an acceptable time, whether a specific RNA hybridization is effective. We hereby introduce a machine learning based technique to address this problem. Sample machine learning (ML) models tested in the training phase include algorithms based on the boosted tree (BT), random forest (RF), decision tree (DT) and logistic regression (LR), and the corresponding models are obtained. Given the RNA molecular coding training and testing sets, the trained machine learning models are applied to predict the classification of RNA hybridization results. The experiment results show that the optimal predictive accuracies are 96.2%, 96.6%, 96.0% and 69.8% for the RF, BT, DT and LR-based approaches, respectively, under the strong constraint condition, compared with traditional representative methods. Furthermore, the average computation efficiency of the RF, BT, DT and LR-based approaches are 208 679, 269 756, 184 333 and 187 458 times higher than that of existing approach, respectively. Given an RNA design, the BT-based approach demonstrates high computational efficiency and better predictive accuracy in determining the biological effectiveness of molecular hybridization.      

A novel algorithm for land use and land cover classification using RADARSAT-2 polarimetric SAR data

This study proposes a new four-component algorithm for land use and land cover (LULC) classification using RADARSAT-2 polarimetric SAR (PolSAR) data. These four components are polarimetric decomposition, PolSAR interferometry, object-oriented image analysis, and decision tree algorithms. First, polarimetric decomposition can be used to support the classification of PolSAR data. It is aimed at extracting polarimetric parameters related to the physical scattering mechanisms of the observed objects. Second, PolSAR interferometry is used to extract polarimetric interferometric information to support LULC classification. Third, the main purposes of object-oriented image analysis are delineating image objects, as well as extracting various textural and spatial features from image objects to improve classification accuracy. Finally, a decision tree algorithm provides an efficient way to select features and implement classification. A comparison between the proposed method and the Wishart supervised classification which is based on the coherency matrix was made to test the performance of the proposed method. The overall accuracy of the proposed method was 86.64%, whereas that of the Wishart supervised classification was 69.66%. The kappa value of the proposed method was 0.84, much higher than that of the Wishart supervised classification, which exhibited a kappa value of 0.65. The results indicate that the proposed method exhibits much better performance than the Wishart supervised classification for LULC classification. Further investigation was carried out on the respective contribution of the four components to LULC classification using RADARSAT-2 PolSAR data, and it indicates that all the four components have important contribution to the classification. Polarimetric information has significant implications for identifying different vegetation types and distinguishing between vegetation and urban/built-up. The polarimetric interferometric information extracted from repeat-pass RADARSAT-2 images is important in reducing the confusion between urban/built-up and vegetation and that between barren/sparsely vegetated land and vegetation. Object-oriented image analysis is very helpful in reducing the effect of speckle in PolSAR images by implementing classification based on image objects, and the textural information extracted from image objects is helpful in distinguishing between water and lawn. The decision tree algorithm can achieve higher classification accuracy than the nearest neighbor classification implemented using Definiens Developer 7.0, and the accuracy of the decision tree algorithm is similar with that of the support vector classification which is implemented based on the features selected using genetic algorithms. Compared with the nearest neighbor and support vector classification, the decision tree algorithm is more efficient to select features and implement classification. Furthermore, the decision tree algorithm can provide clear classification rules that can be easily interpreted based on the physical meaning of the features used in the classification. This can provide physical insight for LULC classification using PolSAR data.     

针对不平衡数据的过采样和随机森林改进算法

针对数据不平衡带来的少数类样本识别率低的问题,提出通过加权策略对过采样和随机森林进行改进的算法,从数据预处理和算法两个方面降低数据不平衡对分类器的影响。数据预处理阶段应用合成少数类过采样技术(Synthetic Minority Oversampling Technique,SMOTE)降低数据不平衡度,每个少数类样本根据其相对于剩余样本的欧氏距离分配权重,使每个样本合成不同数量的新样本。算法改进阶段利用Kappa系数评价随机森林中决策树训练后的分类效果,并赋予每棵树相应的权重,使分类能力更好的树在投票阶段有更大的投票权,提高随机森林算法对不平衡数据的整体分类性能。在KEEL数据集上的实验表明,与未改进算法相比,改进后的算法对少数类样本分类准确率和整体样本分类性能有所提升。     

基于新型不纯度度量的代价敏感随机森林分类器

针对不平衡数据集的有效分类问题,提出一种结合代价敏感学习和随机森林算法的分类器。首先提出了一种新型不纯度度量,该度量不仅考虑了决策树的总代价,还考虑了同一节点对于不同样本的代价差异;其次,执行随机森林算法,对数据集作K次抽样,构建K个基础分类器;然后,基于提出的不纯度度量,通过分类回归树(CART)算法来构建决策树,从而形成决策树森林;最后,随机森林通过投票机制做出数据分类决策。在UCI数据库上进行实验,与传统随机森林和现有的代价敏感随机森林分类器相比,该分类器在分类精度、AUC面积和Kappa系数这3种性能度量上都具有良好的表现。     

基于平均特征重要性和集成学习的异常检测

异常检测系统在网络空间安全中起着至关重要的作用,为网络安全提供有效的保障.对于复杂的网络流量信息,传统的单一的分类器往往无法同时具备较高检测精确度和较强的泛化能力.此外,基于全特征的异常检测模型往往会受到冗余特征的干扰,影响检测的效率和精度.针对这些问题,本文提出了一种基于平均特征重要性的特征选择和集成学习的模型,选取决策树(DT)、随机森林(RF)、额外树(ET)作为基分类器,建立投票集成模型,并基于基尼系数计算基分类器的平均特征重要性进行特征选择.在多个数据集上的实验评估结果表明,本文提出的集成模型优于经典集成学习模型及其他著名异常检测集成模型.且提出的基于平均特征重要性的特征选择方法可以使集成模型准确率平均进一步提升约0.13%,训练时间平均节省约30%.     

结合纹理与极化分解的面向对象极化SAR水体提取方法

合成孔径雷达(Synthetic Aperture Radar,SAR)拥有全天时全天候的工作能力,能够有效地连续对地观测,是土地管理、水体监测、灾害评估等多种应用的稳定数据来源。基于面向对象的思想,提出一种高精度、低虚警率的极化SAR(Polarimetric SAR,PolSAR)水体提取方法。此方法首先对极化SAR图像进行分割,再结合纹理与极化分解特征,对分割区域进行投票,识别水体区域。利用Radarsat-2数据和TerraSAR-X数据开展实验,并将提出方法与基于单一纹理和基于极化分解等水体提取方法进行对比,结果表明该方法在两种数据中均具有最高的总分类精度,其中基于分割技术能够保持完整的水陆边界,纹理与极化特征能够区分浅草、裸地和阴影等与水体相似的地物,结合投票方法能够提高小型水体检测率。     

Knowledge-based system for text classification using ID6NB algorithm

This paper presents a novel algorithm named ID6NB for extending decision tree induced by Quinlan’s non-incremental ID3 algorithm. The presented approach is aimed at suggesting the solutions for few unhandled exceptions of the Decision tree induction algorithms such as (i) the situation in which the majority voting makes incorrect decision (generating two different types of rules for same data), and (ii) in case of dimensionality reduction by decision tree induction algorithms, the determination of appropriate attribute at a node where two or more attributes have equal highest information gain. Exception due to majority voting is handled with the help of Naive Bayes algorithm and also novel solutions are given for dimensionality reduction. As a result, the classification accuracy has drastically improved. An extensive experimental evaluation on a number of real and synthetic databases shows that ID6NB is a state-of-the-art classification algorithm that outperforms well than other methods of decision tree learning.     

一种模型决策森林算法

随机森林(RF)具有抗噪能力强,预测准确率高,能够处理高维数据等优点,因此在机器学习领域得到了广泛的应用。模型决策树(MDT)是一种加速的决策树算法,虽然能够提高决策树算法的训练效率,但是随着非纯伪叶结点规模的增大,模型决策树的精度也在下降。针对上述问题,提出了一种模型决策森林算法(MDF)以提高模型决策树的分类精度。MDF算法将MDT作为基分类器,利用随机森林的思想,生成多棵模型决策树。算法首先通过旋转矩阵得到不同的样本子集,然后在这些样本子集上训练出多棵不同的模型决策树,再将这些树通过投票的方式进行集成,最后根据得到的模型决策森林给出分类结果。在标准数据集上的实验结果表明,提出的模型决策森林在分类精度上明显优于模型决策树算法,并且MDF在树的数量较少时也能取到不错的精度,避免了因树的数量增加时间复杂度增高的问题。     

基于改进随机森林算法的工业过程运行状态评价

运行状态评价是指在过程正常生产的前提下, 进一步判断生产过程运行状态的优劣. 针对复杂工业过程定量信息与定性信息共存的情况, 本文提出了一种基于随机森林的工业过程运行状态评价方法. 针对随机森林中决策树信息存在冗余的问题, 基于互信息将传统随机森林中的决策树进行分组, 并选出每组中最优的决策树组成新的随机森林. 同时为了强化评价精度高的决策树和弱化评价精度低的决策树对最终评价结果的影响, 使用加权投票机制取代传统众数投票方法, 最终构成一种基于互信息的加权随机森林算法(Mutual information weighted random forest, MIWRF). 对于在线评价, 本文通过计算在线数据处于各个等级的概率, 并且结合提出的在线评价策略, 判定当前样本运行状态等级. 为了验证所提算法的有效性, 将所提方法应用于湿法冶金浸出过程, 实验结果表明, 相对于传统随机森林算法, MIWRF 降低了模型的复杂度, 同时提高了运行状态评价精度.     

A random forest-based framework for crop mapping using temporal,spectral, textural and polarimetric observations

Combining optical and polarimetric synthetic aperture radar (PolSAR) earth observations offers a complementary data set with a significant number of spectral, textural, and polarimetric features for crop mapping and monitoring. Moreover, a temporal combination of both sources of information may lead to obtaining more reliable results compared to the use of single-time observations. In this paper, an operational framework based on the stacked generalization of random forest (RF), which efficiently employed bi-temporal observations of optical and radar data, was proposed for crop mapping. In the first step, various spectral, vegetation index, textural, and polarimetric features were extracted from both data sources and placed into several groups. Each group was classified separately using a single RF classifier. Then, several additional classification tasks were accomplished by another RF classifier. The earth observations used in this paper were collected by RapidEye satellites and the Unmanned Aerial Vehicle Synthetic Aperture Radar (UAVSAR) system over an agricultural region near Winnipeg, Manitoba, Canada. The results confirmed that the proposed methodology was able to provide a higher overall accuracy and kappa coefficient than traditional stacking method, and also than all the individual RFs using each group. These accuracy metrics were also better than those of the RFs using the stacked features. Moreover, only the proposed methodology could achieve standard accuracy (F-score ≥85%) for all crop types in the study area. The visual comparison also demonstrated that the crop maps produced by the proposed methodology had more homogeneous, uniform appearances. Moreover, the mixed pixels of crop types, which abundantly existed in the traditional stacking and individual RFs? maps, were significantly eliminated.     

Artificial bee colony-based support vector machines with feature selection and parameter optimization for rule extraction

Support vector machine (SVM) is a state-of-art classification tool with good accuracy due to its ability to generate nonlinear model. However, the nonlinear models generated are typically regarded as incomprehensible black-box models. This lack of explanatory ability is a serious problem for practical SVM applications which require comprehensibility. Therefore, this study applies a C5 decision tree (DT) to extract rules from SVM result. In addition, a metaheuristic algorithm is employed for the feature selection. Both SVM and C5 DT require expensive computation. Applying these two algorithms simultaneously for high-dimensional data will increase the computational cost. This study applies artificial bee colony optimization (ABC) algorithm to select the important features. The proposed algorithm ABC–SVM–DT is applied to extract comprehensible rules from SVMs. The ABC algorithm is applied to implement feature selection and parameter optimization before SVM–DT. The proposed algorithm is evaluated using eight datasets to demonstrate the effectiveness of the proposed algorithm. The result shows that the classification accuracy and complexity of the final decision tree can be improved simultaneously by the proposed ABC–SVM–DT algorithm, compared with genetic algorithm and particle swarm optimization algorithm.     

Texture classification of logged forests in tropical Africa using machine-learning algorithms

This Letter describes a procedure that incorporates textural measures in the classification of logged forests from Landsat Thematic Mapper data. The objective was to increase classification accuracy by applying recently developed algorithms in machine learning that are fast in training. Three voting classification algorithms, Arc-4x, Adaboost and bagging were also tested. Initial results using a decision tree classifier showed that adding selected textural measures increased the accuracy of logged forest classification by almost 40%, although the class accuracy for logged forests was only approximately 50% when using spectral and textural features combined. No further significant increase in the classification of logged forests was obtained by voting classification.     

Change detection using distance-based algorithms between synthetic aperture radar polarimetric decompositions

The present study introduces distance based change detection (CD) algorithms in polarimetric synthetic aperture radar (PolSAR) data. PolSAR images, due to interactions between electromagnetic waves and target and because of the high spatial resolution, can be used to study changes in the Earth’s surface. The purpose of this paper is to use features extracted from the fully-polarimetry imaging radar that involved Yamaguchi four-component and H/A/α decomposition based on the distance between the vectors of features for CD. We first extract features from polarimetric decompositions of multi-looked covariance (or coherency) matrix data. We then use two well-known distance measures namely Canberra and Euclidean distances for measuring the similarity between the vectors of polarimetric decompositions at different times. Assessment of incorporated methods is performed using different criteria, such as overall accuracy, area under the receiver operating characteristic curve, and false alarms rate. The results of the experiments show that Canberra distance has better performance with high overall accuracy and low false alarm rate than Euclidean distance and other compared algorithms to detect changes.     

The multispectral separability of Costa Rican rainforest types with support vector machines and Random Forest decision trees

Estimating the extent of tropical rainforest types is needed for biodiversity assessment and carbon accounting. In this study, we used statistical comparisons to determine the ability of Landsat Thematic Mapper (TM) bands and spectral vegetation indices to discriminate composition and structural types. A total of 144 old-growth forest plots established in northern Costa Rica were categorized via cluster analysis and ordination. Locations for palm swamps, forest regrowth and tree plantations were also acquired, making 11 forest types for separability analysis. Forest types classified using support vector machines (SVM), a theoretically superior method for solving complex classification problems, were compared with the random forest decision tree classifier (RF). Separability comparisons demonstrate that spectral data are sensitive to differences among forest types when tree species and structural similarity is low. SVM class accuracy was 66.6% for all forest types, minimally higher than the RF classifier (65.3%). TM bands and the Normalized Difference Vegetation Index (NDVI) combined with digital elevation data notably increased accuracies for SVM (84.3%) and RF (86.7%) classifiers. Rainforest types discriminated here are typically limited to one or two categories for remote sensing classifications. Our results indicate that TM bands and ancillary data combined via machine learning algorithms can yield accurate and ecologically meaningful rainforest classifications important to national and international forest monitoring protocols.     

Mapping the land-cover distribution in arid and semiarid urban landscapes with Landsat Thematic Mapper imagery

Mapping the land-cover distribution in arid and semiarid urban landscapes using medium spatial resolution imagery is especially difficult due to the mixed-pixel problem in remotely sensed data and the confusion of spectral signatures among bare soils, sparse density shrub lands, and impervious surface areas (ISAs hereafter). This article explores a hybrid method consisting of linear spectral mixture analysis (LSMA), decision tree classifier, and cluster analysis for mapping land-cover distribution in two arid and semiarid urban landscapes: Urumqi, China, and Phoenix, USA. The Landsat Thematic Mapper (TM) imagery was unmixed into four endmember fraction images (i.e. high-albedo object, low-albedo object, green vegetation (GV), and soil) using the LSMA approach. New variables from these fraction images and TM spectral bands were used to map seven land-cover classes (i.e. forest, shrub, grass, crop, bare soil, ISA, and water) using the decision tree classifier. The cluster analysis was further used to modify the classification results. QuickBird imagery in Urumqi and aerial photographs in Phoenix were used to assess classification accuracy. Overall classification accuracies of 86.0% for Urumqi and 88.7% for Phoenix were obtained, much higher accuracies than those utilizing the traditional maximum likelihood classifier (MLC). This research demonstrates the necessity of using new variables from fraction images to distinguish between ISA and bare soils and between shrub and other vegetation types. It also indicates the different effects of spatial patterns of land-cover composition in arid and semiarid landscapes on urban land-cover classification.     

A comparison of pixel-based decision tree and object-based Support Vector Machine methods for land-cover classification based on aerial images and airborne lidar data

Precisely monitoring land cover/use is crucial for urban environmental assessment and management. Various classification techniques such as pixel-based and object-based approaches have advantages and disadvantages. In this article, based on our experiment data from an unmanned platform carried lidar scanner system and camera, we explored and compared classi?cation accuracies of pixel-based decision tree (DT) and object-based Support Vector Machine (SVM) approaches. Lidar height information can improve classification accuracy based on either object-based SVM or pixel-based DT. From total classification accuracy, object-based SVM was higher than that of pixel-based DT classification, and total accuracy and kappa coefficient of the former were 92.71% and 0.899, respectively. However, pixel-based DT outperformed object-based SVM when classifying small ‘scatter’ tree along roads. Additionally, in order to evaluate the accuracy of pixel-based DT and object-based SVM, we added benchmark data of ISPRS to compare the classification results of two methods. Object-based SVM classification methods by combining aerial imagery with lidar height information can achieve higher classification accuracy. And, accurately extracting tree class of different landscape pattern should select appropriate machine-learning algorithms. Comparison of the results on two methods will provide a reference for selecting a particular classification approaches according to local conditions.     

处理不平衡数据的聚类欠采样加权随机森林算法

针对现有欠采样处理算法中存在样本缺少代表性、分类性能差等问题,提出了一种基于聚类欠采样的加权随机森林算法(weighted random forest algorithm based on clustering under-sampling,CUS-WRF)。利用K-means算法对多数类样本聚类,引入欧氏距离作为欠采样时分配样本个数的权重依据,使采样后的多数类样本与少数类样本形成一个平衡的样本集,以CART决策树为基分类器,加权随机森林为整体框架,同时将测试样本的准确率作为每棵树的权值来完成对结果的最终投票,有效提高了整体分类性能。选择八组KEEL数据集进行实验,结果表明,与其余四种基于随机森林的不平衡数据处理算法相比,CUS-WRF算法的分类性能及稳定性更具优势。     

Crop discrimination based on polarimetric correlation coefficients optimization for PolSAR data

Crop discrimination is a necessary step for most agricultural monitoring systems. Radar polarimetric responses from various crops strongly relate to the types and orientations of the local scatterers, which makes the discrimination still difficult using the polarimetric synthetic aperture radar (PolSAR) technique. This work provides a new approach by investigating and utilizing the characteristics of polarimetric correlation coefficients in the rotation domain along the radar line of sight. The theoretical basis lies in that polarimetric correlation coefficients can reflect the different responses and can be enhanced at different levels for various land-cover types with suitable rotation angles in the rotation domain. In this vein, a polarimetric correlation coefficient optimization framework is established and new polarimetric features are extracted therein. Demonstration with multi-frequency (P-, L-, and C-bands) airborne synthetic aperture radar (AIRSAR) PolSAR data over crop areas validates that polarimetric correlation coefficients are crop dependent and the optimized polarimetric correlation coefficient parameters can better discriminate them. Then, a crop discrimination scheme is proposed using the derived polarimetric features. A flow chart for the optimal discrimination feature set selection and determination is provided and is validated by the real data with seven typical crop types. All these crop types are successfully discriminated for the P- and L-band data, whereas only two types of crops are slightly overlapped in the feature space for the C-band data. Experimental studies demonstrate the efficiency and potential of the established methodology.     

Improved airborne PolSAR calibration algorithm based on time-variant attitude compensation

The polarimetric synthetic aperture radar (PolSAR) usually has to be calibrated before practical application, so as to compensate for polarimetric distortion. The varying platform attitude is one of the factors causing distortion but has rarely been considered in existing polarimetric calibration algorithms. With the resolution of PolSAR systems improving and the synthetic aperture time prolonging, this factor cannot simply be ignored. The varying attitude will distort the polarimetric information by rotating the polarimetric orientation angle, and such distortion changes with azimuth time. In this article, we modified the conventional polarimetric system model to take account of the time-variant impact of the unstable platform attitude. A calibration algorithm is proposed to compensate the time-variant attitude impact on the raw return data. The proposed calibration algorithm is tested on the data collected by Institute of Electronics, Chinese Academy of Sciences P-band PolSAR system. Results show that it can achieve better performance by reducing crosstalk error than two conventional methods.