Accurate ensemble pruning with PL-bagging

Ensemble pruning deals with the selection of base learners prior to combination in order to improve prediction accuracy and efficiency. In the ensemble literature, it has been pointed out that in order for an ensemble classifier to achieve higher prediction accuracy, it is critical for the ensemble classifier to consist of accurate classifiers which at the same time diverse as much as possible. In this paper, a novel ensemble pruning method, called PL-bagging, is proposed. In order to attain the balance between diversity and accuracy of base learners, PL-bagging employs positive Lasso to assign weights to base learners in the combination step. Simulation studies and theoretical investigation showed that PL-bagging filters out redundant base learners while it assigns higher weights to more accurate base learners. Such improved weighting scheme of PL-bagging further results in higher classification accuracy and the improvement becomes even more significant as the ensemble size increases. The performance of PL-bagging was compared with state-of-the-art ensemble pruning methods for aggregation of bootstrapped base learners using 22 real and 4 synthetic datasets. The results indicate that PL-bagging significantly outperforms state-of-the-art ensemble pruning methods such as Boosting-based pruning and Trimmed bagging.     

Surface and normal ensembles for surface reconstruction

The majority of the existing techniques for surface reconstruction and the closely related problem of normal reconstruction are deterministic. Their main advantages are the speed and, given a reasonably good initial input, the high quality of the reconstructed surfaces. Nevertheless, their deterministic nature may hinder them from effectively handling incomplete data with noise and outliers. An ensemble is a statistical technique which can improve the performance of deterministic algorithms by putting them into a statistics based probabilistic setting. In this paper, we study the suitability of ensembles in normal and surface reconstruction. We experimented with a widely used normal reconstruction technique [Hoppe H, DeRose T, Duchamp T, McDonald J, Stuetzle W. Surface reconstruction from unorganized points. Computer Graphics 1992;71-8] and Multi-level Partitions of Unity implicits for surface reconstruction [Ohtake Y, Belyaev A, Alexa M, Turk G, Seidel H-P. Multi-level partition of unity implicits. ACM Transactions on Graphics 2003;22(3):463-70], showing that normal and surface ensembles can successfully be combined to handle noisy point sets.     

Random Forests with ensemble of feature spaces

Random Forests receive much attention from researchers because of their excellent performance. As Breiman suggested, the performance of Random Forests depends on the strength of the weak learners in the forests and the diversity among them. However, in the literature, many researchers only considered pre-processing of the data or post-processing of the Random Forests models. In this paper, we propose a new method to increase the diversity of each tree in the forests and thereby improve the overall accuracy. During the training process of each individual tree in the forest, different rotation spaces are concatenated into a higher space at the root node. Then the best split is exhaustively searched within this higher space. The location where the best split lies decides which rotation method to be used for all subsequent nodes. The performance of the proposed method here is evaluated on 42 benchmark data sets from various research fields and compared with the standard Random Forests. The results show that the proposed method improves the performance of the Random Forests in most cases.     

Ensemble clustering by means of clustering embedding in vector spaces

Ensemble clustering is a recently evolving research direction in cluster analysis and has found several different application domains. In this work the complex ensemble clustering problem is reduced to the well-known Euclidean median problem by clustering embedding in vector spaces. The Euclidean median problem is solved by the Weiszfeld algorithm and an inverse transformation maps the Euclidean median back into the clustering domain. In the experiment study different evaluation strategies are considered. The proposed embedding strategy is compared to several state-of-art ensemble clustering algorithms and demonstrates superior performance.     

基于集成学习的离子通道药物靶点预测

新药研制成功的关键在于药物靶点的发现和准确定位.在已知的药物靶点中,离子通道蛋白是一类广受欢迎的靶点,它与免疫系统、心血管等疾病密切相关.对于靶点的发现,传统生物方法成本高、耗时久.因此,探讨了基于机器学习的离子通道蛋白药物靶点的挖掘,以加快药物靶点发现过程,节约经费.由于药物靶点相关序列的长度不一致,考虑了蛋白质序列编码的13种特征,它们能将不等长的蛋白质序列转化成等长序列.通过数值实验筛选能够较好地区分靶点和非靶点的特征子集,并采用集成学习的方法整合特征得到预测模型.通过与已有工作的比较表明,提出的集成模型能得到较高的准确率,具有很好的应用前景.     

Retrieving soil temperature profile by assimilating MODIS LST products with ensemble Kalman filter

Proper estimation of initial state variables and model parameters are vital importance for determining the accuracy of numerical model prediction. In this work, we develop a one-dimensional land data assimilation scheme based on ensemble Kalman filter and Common Land Model version 3.0 (CoLM). This scheme is used to improve the estimation of soil temperature profile. The leaf area index (LAI) is also updated dynamically by MODIS LAI production and the MODIS land surface temperature (LST) products are assimilated into CoLM. The scheme was tested and validated by observations from four automatic weather stations (BTS, DRS, MGS, and DGS) in Mongolian Reference Site of CEOP during the period of October 1, 2002 to September 30, 2003. Results indicate that data assimilation improves the estimation of soil temperature profile about 1 K. In comparison with simulation, the assimilation results of soil heat fluxes also have much improvement about 13 W m^− 2 at BTS and DGS and 2 W m^− 2 at DRS and MGS, respectively. In addition, assimilation of MODIS land products into land surface model is a practical and effective way to improve the estimation of land surface variables and fluxes.     

Boosted Bayesian network classifiers

The use of Bayesian networks for classification problems has received a significant amount of recent attention. Although computationally efficient, the standard maximum likelihood learning method tends to be suboptimal due to the mismatch between its optimization criteria (data likelihood) and the actual goal of classification (label prediction accuracy). Recent approaches to optimizing classification performance during parameter or structure learning show promise, but lack the favorable computational properties of maximum likelihood learning. In this paper we present boosted Bayesian network classifiers, a framework to combine discriminative data-weighting with generative training of intermediate models. We show that boosted Bayesian network classifiers encompass the basic generative models in isolation, but improve their classification performance when the model structure is suboptimal. We also demonstrate that structure learning is beneficial in the construction of boosted Bayesian network classifiers. On a large suite of benchmark data-sets, this approach outperforms generative graphical models such as naive Bayes and TAN in classification accuracy. Boosted Bayesian network classifiers have comparable or better performance in comparison to other discriminatively trained graphical models including ELR and BNC. Furthermore, boosted Bayesian networks require significantly less training time than the ELR and BNC algorithms.     

Light gradient boosting machine with optimized hyperparameters for identification of malicious access in IoT network

In this paper, an advanced and optimized Light Gradient Boosting Machine (LGBM) technique is proposed to identify the intrusive activities in the Internet of Things (IoT) network. The followings are the major contributions: i) An optimized LGBM model has been developed for the identification of malicious IoT activities in the IoT network; ii) An efficient evolutionary optimization approach has been adopted for finding the optimal set of hyper-parameters of LGBM for the projected problem. Here, a Genetic Algorithm (GA) with k-way tournament selection and uniform crossover operation is used for efficient exploration of hyper-parameter search space; iii) Finally, the performance of the proposed model is evaluated using state-of-the-art ensemble learning and machine learning-based model to achieve overall generalized performance and efficiency. Simulation outcomes reveal that the proposed approach is superior to other considered methods and proves to be a robust approach to intrusion detection in an IoT environment.     

Ensemble feature selection and classification methods for machine learning-based coronary artery disease diagnosis

Coronary artery disease (CAD) is a condition in which the heart is not fed sufficiently as a result of the accumulation of fatty matter. As reported by the World Health Organization, around 32% of the total deaths in the world are caused by CAD, and it is estimated that approximately 23.6 million people will die from this disease in 2030. CAD develops over time, and the diagnosis of this disease is difficult until a blockage or a heart attack occurs. In order to bypass the side effects and high costs of the current methods, researchers have proposed to diagnose CADs with computer-aided systems, which analyze some physical and biochemical values at a lower cost. In this study, for the CAD diagnosis, (i) seven different computational feature selection (FS) methods, one domain knowledge-based FS method, and different classification algorithms have been evaluated; (ii) an exhaustive ensemble FS method and a probabilistic ensemble FS method have been proposed. The proposed approach is tested on three publicly available CAD data sets using six different classification algorithms and four different variants of voting algorithms. The performance metrics have been comparatively evaluated with numerous combinations of classifiers and FS methods. The multi-layer perceptron classifier obtained satisfactory results on three data sets. Performance evaluations show that the proposed approach resulted in 91.78%, 85.55%, and 85.47% accuracy for the Z-Alizadeh Sani, Statlog, and Cleveland data sets, respectively.     

F-BIDS: Federated-Blending based Intrusion Detection System

The rapid development of network communication along with the drastic increase in the number of smart devices has triggered a surge in network traffic, which can contain private data and in turn affect user privacy. Recently, Federated Learning (FL) has been proposed in Intrusion Detection Systems (IDS) to ensure attack detection, privacy preservation, and cost reduction, which are crucial issues in traditional centralized machine-learning-based IDS. However, FL-based approaches still exhibit vulnerabilities that can be exploited by adversaries to compromise user data. At the same time, meta-models (including the blending models) have been recognized as one of the solutions to improve generalization for attack detection and classification since they enhance generalization and predictive performances by combining multiple base models. Therefore, in this paper, we propose a Federated Blending model-driven IDS framework for the Internet of Things (IoT) and Industrial IoT (IIoT), called F-BIDS, in order to further protect the privacy of existing ML-based IDS. The proposition consists of a Decision Tree (DT) and Random Forest (RF) as base classifiers to first produce the meta-data. Then, the meta-classifier, which is a Neural Networks (NN) model, uses the meta-data during the federated training step, and finally, it makes the final classification on the test set. Specifically, in contrast to the classical FL approaches, the federated meta-classifier is trained on the meta-data (composite data) instead of user-sensitive data to further enhance privacy. To evaluate the performance of F-BIDS, we used the most recent and open cyber-security datasets, called Edge-IIoTset (published in 2022) and InSDN (in 2020). We chose these datasets because they are recent datasets and contain a large amount of network traffic including both malicious and benign traffic.