Feature selection based on artificial bee colony and gradient boosting decision tree

Data from many real-world applications can be high dimensional and features of such data are usually highly redundant. Identifying informative features has become an important step for data mining to not only circumvent the curse of dimensionality but to reduce the amount of data for processing. In this paper, we propose a novel feature selection method based on bee colony and gradient boosting decision tree aiming at addressing problems such as efficiency and informative quality of the selected features. Our method achieves global optimization of the inputs of the decision tree using the bee colony algorithm to identify the informative features. The method initializes the feature space spanned by the dataset. Less relevant features are suppressed according to the information they contribute to the decision making using an artificial bee colony algorithm. Experiments are conducted with two breast cancer datasets and six datasets from the public data repository. Experimental results demonstrate that the proposed method effectively reduces the dimensions of the dataset and achieves superior classification accuracy using the selected features.     

基于粒度粗糙熵与改进蜂群算法的特征选择

经典的人工蜂群(artificial bee colony, ABC)算法面临着收敛速度慢、易陷入局部最优等不足,因此基于该算法来进行特征选择还存在很多问题.对此,提出了一种基于粒度粗糙熵与改进蜂群算法的特征选择方法FS＿GREIABC.首先,将粗糙集中的知识粒度与粗糙熵有机地结合起来,提出一种新的信息熵模型——粒度粗糙熵;其次,将粒度粗糙熵应用于ABC算法中,提出一种基于粒度粗糙熵的适应度函数,从而获得了一种新的适应度计算策略;第三,为了提高ABC算法的局部搜索能力,将云模型引入到跟随蜂阶段.在多个UCI数据集以及软件缺陷预测数据集上的实验表明,相对于现有的特征选择算法, FS＿GREIABC不仅能够选择较少的特征,而且具有更好的分类性能.     

HColonies: a new hybrid metaheuristic for medical data classification

Medical data feature a number of characteristics that make their classification a complex task. Yet, the societal significance of the subject and the computational challenge it presents has caused the classification of medical datasets to be a popular research area. A new hybrid metaheuristic is presented for the classification task of medical datasets. The hybrid ant–bee colonies (HColonies) consists of two phases: an ant colony optimization (ACO) phase and an artificial bee colony (ABC) phase. The food sources of ABC are initialized into decision lists, constructed during the ACO phase using different subsets of the training data. The task of the ABC is to optimize the obtained decision lists. New variants of the ABC operators are proposed to suit the classification task. Results on a number of benchmark, real-world medical datasets show the usefulness of the proposed approach. Classification models obtained feature good predictive accuracy and relatively small model size.     

基于差分进化生物地理学优化的多层感知器训练方法

针对生物地理学优化训练多层感知器存在的早熟收敛以及初始化灵敏等问题,提出一种基于差分进化生物地理学优化的多层感知器训练方法。将生物地理学优化（Biogeography-based Optimization,BBO）与差分进化(Differential Evolution,DE)算法相结合,形成改进的混合DE_BBO算法;采用改进的DE_BBO来训练多层感知器（Multi-Layer Perceptron,MLP）,并应用于虹膜、乳腺癌、输血、钞票验证等4类数据分类。与BBO、PSO、GA、ACO、ES、PBIL等6种主流启发式算法的实验结果进行比较表明,DE_BBO_MLP算法在分类精度和收敛速度等方面优于已有方法。     

基于混合人工蜂群算法的并行测试任务优化研究

并行测试技术可以同时进行多个任务的测试,提高资源利用率,节约测试成本;并行测试调度问题是一种复杂的组合优化问题,是并行测试技术的核心要素;并行测试系统作为并行测试技术的载体,自身的性能和求解效率尤其重要;对并行测试完成时间极限定理进行了研究,建立了并行测试任务调度的数学模型,分析了传统元启发式算法求解并行测试问题的不足,提出了基于动态规划的递归搜索技术和人工蜂群算法相结合的混合人工蜂群算法,并采用整数规划精确算法和遗传算法对混合人工蜂群算法进行验证;得出结论采用混合人工蜂群算法进行并行测试任务的调度节约了接近50%的时间,降低了约20%的硬件资源占用,提高了测试效率,可以满足工程实际的应用。     

A hybrid approach to medical decision support systems: combining feature selection, fuzzy weighted pre-processing and AIRS

This paper presents a hybrid approach based on feature selection, fuzzy weighted pre-processing and artificial immune recognition system (AIRS) to medical decision support systems. We have used the heart disease and hepatitis disease datasets taken from UCI machine learning database as medical dataset. Artificial immune recognition system has shown an effective performance on several problems such as machine learning benchmark problems and medical classification problems like breast cancer, diabetes, and liver disorders classification. The proposed approach consists of three stages. In the first stage, the dimensions of heart disease and hepatitis disease datasets are reduced to 9 from 13 and 19 in the feature selection (FS) sub-program by means of C4.5 decision tree algorithm (CBA program), respectively. In the second stage, heart disease and hepatitis disease datasets are normalized in the range of [0,1] and are weighted via fuzzy weighted pre-processing. In the third stage, weighted input values obtained from fuzzy weighted pre-processing are classified using AIRS classifier system. The obtained classification accuracies of our system are 92.59% and 81.82% using 50-50% training-test split for heart disease and hepatitis disease datasets, respectively. With these results, the proposed method can be used in medical decision support systems.     

一种双种群差分蜂群算法

人工蜂群算法(ABC)是一种基于蜜蜂群智能搜索行为的随机优化算法.为了有效改善人工蜂群算法的性能,结合差分进化算法,提出一种新的双种群差分蜂群算法(BDABC).该算法首先通过基于反向学习的策略初始化种群,使得初始化的个体尽可能均匀分布在搜索空间,然后将种群中的个体随机分成两组,每组采用不同的优化策略同时进行寻优,并通过在两群体之间引入交互学习的思想,来提高算法的收敛速度.基于6个标准测试函数的仿真实验表明,BDABC算法能有效避免早熟收敛,全局优化能力和收敛速率都有显著提高.     

改进人工蜂群优化BP神经网络的分类研究

针对BP神经网络对初始权重敏感,容易陷入局部最优,人工蜂群算法局部搜索能力和开发能力相对较弱等问题,提出一种基于改进人工蜂群和反向传播的神经网络训练方法。引进差分进化思想改进人工蜂群算法,并对跟随蜂的搜索行为进行更准确的描述。用改进的人工蜂群全局搜索神经网络的初始权重,防止神经网络陷入局部最优。用新的方法对神经网络训练进行分类。实验结果表明,该算法相对于标准的BP神经网络,有效提高了分类正确率,泛化能力较强。     

Volterra kernel based face recognition using artificial bee colonyoptimization

The present paper describes a novel method of implementation of a stochastic optimization technique for the face recognition problem. The method proposed divides the original images into patches in space, and seeks a non-linear functional mapping using second-order Volterra kernels. The artificial bee colony optimization technique, a modern stochastic optimization algorithm, is used to derive optimal Volterra kernels during training to simultaneously maximize inter-class distances and minimize intra-class distances in the feature space. During testing, a voting procedure is used in conjunction with a nearest neighbor classifier to decide to which class each individual patch belongs. Finally, the aggregate classification results of all patches in an image are used to determine the overall recognition outcome for the given image. The utility of the proposed scheme is aptly demonstrated by implementing it on two popular benchmark face recognition datasets, and comparing the effectiveness of the proposed approach vis-à-vis other statistical learning procedures in facial recognition and also several other methods developed so far. The effectiveness of the artificial bee colony optimization technique and its Levy-mutated variation in optimizing Volterra kernels is conclusively proven in this paper by significantly outperforming many popular contemporary algorithms.     

Fast artificial bee colony and its application to stereo correspondence

Nature-inspired meta-heuristics have gained popularity for solutions to many real-world complex problems, and the artificial bee colony algorithm is one of the most powerful optimisation methods among meta-heuristics. However, inefficient exploitation of onlooker bees prevents the artificial bee colony algorithm from finding the final result accurately and efficiently for complex problems. In this paper, a novel optimisation method is proposed based on the artificial bee colony algorithm. The proposed optimisation method adaptively exploits onlooker bees over generations. In addition, the proposed optimisation method is applied to a stereo-matching problem to minimise the segment-based integer energy function, which is also introduced in this paper. The experimental results show that the proposed optimisation method outperforms state-of-the-art population-based meta-heuristics, such as the genetic algorithm, differential evolution, conventional artificial bee colony, and clonal selection algorithm, for benchmark functions as well as for the stereo-matching problem.     

混合离散人工蜂群算法求解含不相关并行机的分布式柔性流水线调度

研究每阶段含不相关并行机的分布式柔性流水线调度问题.考虑顺序相关准备时间和工件动态到达时间,以最小化总加权提前/拖期惩罚为目标建立整数规划模型,提出一种融合离散差分进化算法、变邻域下降算法和局域搜索的混合离散人工蜂群算法以获取近优解.该算法采用基于工厂-工件号的编码以及基于机器最早空闲时间的动态解码机制,通过随机规则和均衡分派策略生成初始工厂-工件序列群,在引领蜂阶段引入离散差分进化算法产生优质工厂-工件序列,在跟随蜂阶段利用变邻域下降算法在被选择序列附近继续搜索以得到邻域序列,在侦察蜂阶段设计基于关键/非关键工厂间插入的局域搜索提高算法搜索能力.通过仿真实验测试不同规模的算例,实验结果表明,所提出的混合离散人工蜂群算法表现出较好的求解性能.     

A binary ABC algorithm based on advanced similarity scheme for feature selection

Feature selection is the basic pre-processing task of eliminating irrelevant or redundant features through investigating complicated interactions among features in a feature set. Due to its critical role in classification and computational time, it has attracted researchers’ attention for the last five decades. However, it still remains a challenge. This paper proposes a binary artificial bee colony (ABC) algorithm for the feature selection problems, which is developed by integrating evolutionary based similarity search mechanisms into an existing binary ABC variant. The performance analysis of the proposed algorithm is demonstrated by comparing it with some well-known variants of the particle swarm optimization (PSO) and ABC algorithms, including standard binary PSO, new velocity based binary PSO, quantum inspired binary PSO, discrete ABC, modification rate based ABC, angle modulated ABC, and genetic algorithms on 10 benchmark datasets. The results show that the proposed algorithm can obtain higher classification performance in both training and test sets, and can eliminate irrelevant and redundant features more effectively than the other approaches. Note that all the algorithms used in this paper except for standard binary PSO and GA are employed for the first time in feature selection.     

Comparison of evolutionary-based optimization algorithms for structural design optimization

In this paper, a comparison of evolutionary-based optimization techniques for structural design optimization problems is presented. Furthermore, a hybrid optimization technique based on differential evolution algorithm is introduced for structural design optimization problems. In order to evaluate the proposed optimization approach a welded beam design problem taken from the literature is solved. The proposed approach is applied to a welded beam design problem and the optimal design of a vehicle component to illustrate how the present approach can be applied for solving structural design optimization problems. A comparative study of six population-based optimization algorithms for optimal design of the structures is presented. The volume reduction of the vehicle component is 28.4% using the proposed hybrid approach. The results show that the proposed approach gives better solutions compared to genetic algorithm, particle swarm, immune algorithm, artificial bee colony algorithm and differential evolution algorithm that are representative of the state-of-the-art in the evolutionary optimization literature.     

Analysing stable feature selection through an augmented marine predator algorithm based on opposition-based learning

Retrieving the relevant information from the high-dimensional dataset enhances the classification accuracy of a predictive model. This research critique has devised an improved marine predator algorithm based on opposition learning for stable feature selection to overcome the problem of high-dimensionality. Marine predator algorithm is a population-based meta-heuristics optimization algorithm that works on the ‘survival-of-the-fittest’ theory. Classical marine predator algorithm explores the search space merely in one direction, affecting its converging capacity while being responsible for stagnation at local minima. The proposed opposition-based learning nuances enhance the exploration capacity of marine predator algorithm and productively converges the model to global optima. The proposed OBL-based marine predator algorithm selects stable, substantial elements from six different high-dimensional microarray datasets. The performance of the proposed method is investigated using five predominantly used classifiers. From the result, it is understood that the proposed approach outperforms other conventional feature selection techniques in terms of converging capability, classification accuracy, and stable feature selection.     

A feature selection method based on modified binary coded ant colony optimization algorithm

Feature selection is a significant task for data mining and pattern recognition. It aims to select the optimal feature subset with the minimum redundancy and the maximum discriminating ability. In the paper, a feature selection approach based on a modified binary coded ant colony optimization algorithm (MBACO) combined with genetic algorithm (GA) is proposed. The method comprises two models, which are the visibility density model (VMBACO) and the pheromone density model (PMBACO). In VMBACO, the solution obtained by GA is used as visibility information; on the other hand, in PMBACO, the solution obtained by GA is used as initial pheromone information. In the method, each feature is treated as a binary bit and each bit has two orientations, one is for selecting the feature and another is for deselecting. The proposed method is also compared with that of GA, binary coded ant colony optimization (BACO), advanced BACO (ABACO), binary coded particle swarm optimization (BPSO), binary coded differential evolution (BDE) and a hybrid GA-ACO algorithm on some well-known UCI datasets; furthermore, it is also compared with some other existing techniques such as minimum Redundancy Maximum Relevance (mRMR), Relief algorithm for a comprehensive comparison. Experimental results display that the proposed method is robust, adaptive and exhibits the better performance than other methods involved in the paper.     

A new hybrid artificial bee colony algorithm for robust optimal design and manufacturing

The purpose of this paper is to develop a novel hybrid optimization method (HRABC) based on artificial bee colony algorithm and Taguchi method. The proposed approach is applied to a structural design optimization of a vehicle component and a multi-tool milling optimization problem.A comparison of state-of-the-art optimization techniques for the design and manufacturing optimization problems is presented. The results have demonstrated the superiority of the HRABC over the other techniques like differential evolution algorithm, harmony search algorithm, particle swarm optimization algorithm, artificial immune algorithm, ant colony algorithm, hybrid robust genetic algorithm, scatter search algorithm, genetic algorithm in terms of convergence speed and efficiency by measuring the number of function evaluations required.     

改进近邻人工蜂群算法求解柔性作业车间调度问题

为了更好地解决以最小化最大完工时间为目标的柔性作业车间调度问题,提出了一种改进的人工蜂群算法。首先,采用随机选择和反向学习策略来提高初始蜜源的质量。同时,设计了一种新颖的特征表示方式,用于计算蜜源之间的距离。在引领蜂阶段,通过引入交叉和变异策略来优化种群中的近距离蜜源。在探索蜂阶段,引入了六种变邻域方法,以扩大解空间的搜索范围。而在侦查蜂阶段,则根据蜜源的潜力值剔除局部最优个体。在15个数据集上进行了广泛实验,实验结果表明,该改进算法性能明显优于其他四种著名的群智能优化算法。该研究为解决柔性作业车间调度问题提供了一种新的有效方法,对于实际生产调度具有重要的实用价值。     

基于最优ABC-SVM算法的P2P流量识别

目前对等网络（Peer-to-Peer,P2P）流量的识别是网络管理研究的热门话题。基于支持向量机（Support Vector Machine , SVM）的P2P流量识别方法是常用的P2P流量识别方法之一。然而SVM的性能主要受参数和其使用特征的影响,而传统的方法则是将SVM的参数优化和特征选择问题分开处理,因此这样很难获得整体性能最优的SVM分类器。本论文提出了一种基于最优人工蜂群算法和支持向量机相结合的P2P流量识别方法,利用人工蜂群算法,将SVM的参数和特征选择问题视为最优化问题同步处理,可以获得整体性能最优的参数和特征子集。在真实的P2P数据上的实验结果表明提出的方法具有很好的自适应性和分类精度,能够同时获取特征子集和SVM参数的最优解,提高SVM分类器的整体性能。     

求解连续空间优化问题的改进蜂群算法

为了有效地解决人工蜂群算法容易陷入局部最优的缺陷,提出了一种改进蜂群算法。首先,利用反向学习方法构建初始种群,以提高初始化解的质量。同时,利用分布估计算法构造优秀个体解空间的概率模型以进行邻域搜索,以改善算法的搜索性能并防止陷入局部最优。对连续空间优化问题进行了仿真实验,结果表明改进算法具有较快的收敛速度,全局寻优能力显著提高。     

Testing Different Ensemble Configurations for Feature Selection

In recent years, ensemble learning has become a prolific area of study in pattern recognition, based on the assumption that using and combining different learning models in the same problem could lead to better performance results than using a single model. This idea of ensemble learning has traditionally been used for classification tasks, but has more recently been adapted to other machine learning tasks such as clustering and feature selection. We propose several feature selection ensemble configurations based on combining rankings of features from individual rankers according to the combination method and threshold value used. The performance of each proposed ensemble configuration was tested for synthetic datasets (to assess the adequacy of the selection), real classical datasets (with more samples than features), and DNA microarray datasets (with more features than samples). Five different classifiers were studied in order to test the suitability of the proposed ensemble configurations and assess the results.