Dynamic multi-objective evolution of classifier ensembles for video face recognition

Due to a limited control over changing operational conditions and personal physiology, systems used for video-based face recognition are confronted with complex and changing pattern recognition environments. Although a limited amount of reference data is initially available during enrollment, new samples often become available over time, through re-enrollment, post analysis and labeling of operational data, etc. Adaptive multi-classifier systems (AMCSs) are therefore desirable for the design and incremental update of facial models. For real time recognition of individuals appearing in video sequences, facial regions are captured with one or more cameras, and an AMCS must perform fast and efficient matching against the facial model of individual enrolled to the system. In this paper, an incremental learning strategy based on particle swarm optimization (PSO) is proposed to efficiently evolve heterogeneous classifier ensembles in response to new reference data. This strategy is applied to an AMCS where all parameters of a pool of fuzzy ARTMAP (FAM) neural network classifiers (i.e., a swarm of classifiers), each one corresponding to a particle, are co-optimized such that both error rate and network size are minimized. To provide a high level of accuracy over time while minimizing the computational complexity, the AMCS integrates information from multiple diverse classifiers, where learning is guided by an aggregated dynamical niching PSO (ADNPSO) algorithm that optimizes networks according both these objectives. Moreover, pools of FAM networks are evolved to maintain (1) genotype diversity of solutions around local optima in the optimization search space and (2) phenotype diversity in the objective space. Accurate and low cost ensembles are thereby designed by selecting classifiers on the basis of accuracy, and both genotype and phenotype diversity. For proof-of-concept validation, the proposed strategy is compared to AMCSs where incremental learning of FAM networks is guided through mono- and multi-objective optimization. Performance is assessed in terms of video-based error rate and resource requirements under different incremental learning scenarios, where new data is extracted from real-world video streams (IIT-NRC and MoBo). Simulation results indicate that the proposed strategy provides a level of accuracy that is comparable to that of using mono-objective optimization and reference face recognition systems, yet requires a fraction of the computational cost (between 16% and 20% of a mono-objective strategy depending on the data base and scenario).     

Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity

This paper has three main goals: (i) to employ two classes of algorithms: bio-inspired and gradient-based to train multi-layer perceptron (MLP) neural networks for pattern classification; (ii) to combine the trained neural networks into ensembles of classifiers; and (iii) to investigate the influence of diversity in the classification performance of individual and ensembles of classifiers. The optimization version of an artificial immune network, named opt-aiNet, particle swarm optimization (PSO) and an evolutionary algorithm (EA) are used as bio-inspired methods to train MLP networks. Besides, the standard backpropagation with momentum (BPM), a quasi-Newton method called DFP and a modified scaled-conjugate gradient (SCGM) are the gradient-based algorithms used to train MLP networks in this work. Comparisons among all the training methods are presented in terms of classification accuracy and diversity of the solutions found. The results obtained suggest that most bio-inspired algorithms deteriorate the diversity of solutions during the search, while immune-based methods, like opt-aiNet, and multiple initializations of standard gradient-based algorithms provide diverse solutions that result in good classification accuracy for the ensembles.     

基于离散微粒群算法求解背包问题研究

微粒群算法(PSO)是一种新的演化算法,主要用于求解数值优化问题.基于离散微粒群算法(DPSO)分别与处理约束问题的罚函数法和贪心变换方法相结合,提出了求解背包问题的两个算法:基于罚函数策略的离散微粒群算法(PFDPSO)和基于贪心变换策略的离散微粒群算法(GDPSO).通过将这两个算法与文献[7]中的混合微粒群算法(Hybrid_PSO)进行数值计算比较发现:对于求解大规模的背包问题,GDPSO非常优秀,其求解能力优于Hybrid_PSO和PFDPSO,是求解背包问题的一种非常有效的方法.     

A dynamic optimization approach for adaptive incremental learning

A fundamental problem when performing incremental learning is that the best set of a classification system's parameters can change with the evolution of the data. Consequently, unless the system self‐adapts to such changes, it will become obsolete, even if the application environment seems to be static. To address this problem, we propose a dynamic optimization approach in this paper that performs incremental learning in an adaptive fashion by tracking, evolving, and combining optimum hypotheses overtime. The approach incorporates various theories, such as dynamic particle swarm optimization, incremental support vector machine classifiers, change detection, and dynamic ensemble selection based on classifiers' confidence levels. Experiments carried out on synthetic and real‐world databases demonstrate that the proposed approach actually outperforms the classification methods often used in incremental learning scenarios. © 2011 Wiley Periodicals, Inc.     

Ensemble feature selection with the simple Bayesian classification

A popular method for creating an accurate classifier from a set of training data is to build several classifiers, and then to combine their predictions. The ensembles of simple Bayesian classifiers have traditionally not been a focus of research. One way to generate an ensemble of accurate and diverse simple Bayesian classifiers is to use different feature subsets generated with the random subspace method. In this case, the ensemble consists of multiple classifiers constructed by randomly selecting feature subsets, that is, classifiers constructed in randomly chosen subspaces. In this paper, we present an algorithm for building ensembles of simple Bayesian classifiers in random subspaces. The EFS_SBC algorithm includes a hill-climbing-based refinement cycle, which tries to improve the accuracy and diversity of the base classifiers built on random feature subsets. We conduct a number of experiments on a collection of 21 real-world and synthetic data sets, comparing the EFS_SBC ensembles with the single simple Bayes, and with the boosted simple Bayes. In many cases the EFS_SBC ensembles have higher accuracy than the single simple Bayesian classifier, and than the boosted Bayesian ensemble. We find that the ensembles produced focusing on diversity have lower generalization error, and that the degree of importance of diversity in building the ensembles is different for different data sets. We propose several methods for the integration of simple Bayesian classifiers in the ensembles. In a number of cases the techniques for dynamic integration of classifiers have significantly better classification accuracy than their simple static analogues. We suggest that a reason for that is that the dynamic integration better utilizes the ensemble coverage than the static integration.     

具有重组学习和混合变异的动态多种群粒子群优化算法

为解决粒子群优化算法中种群多样性与收敛性间的矛盾,提出一种具有重组学习和混合变异的动态多种群粒子群优化算法.该算法动态划分多种群并融入重构粒子作为引导因子,在增加种群多样性的同时保留优秀粒子的空间信息;在算法执行阶段对最优个体施加混合变异,基于时变概率实施反向学习策略或者邻域扰动操作,帮助粒子快速跳出局部困境,加强对附近区域内的精细搜索.基于14个多类型标准测试函数,并与其他的改进粒子群算法进行对比,验证了几种改进措施的有效性和叠加影响.为进一步探究概率性混合变异策略的敏感性,对变异方式及参数设置进行仿真实验,结果表明,所采用的极值扰动策略具有显著的优势,合理地控制学习强度可以充分发挥反向学习的作用,并给出影响参数的建议取值范围.实验结果还表明,所提出的算法能够更好地平衡种群的开发与勘探能力,提高求解精度和收敛性能.     

A discrete particle swarm optimization algorithm with self-adaptive diversity control for the permutation flowshop problem with blocking

This paper proposes a discrete particle swarm optimization (DPSO) algorithm for the m-machine permutation flowshop scheduling problem with blocking to minimize the makespan, which has a strong industrial background, e.g., many production processes of chemicals and pharmaceuticals in chemical industry can be reduced to this problem. To prevent the DPSO from premature convergence, a self-adaptive diversity control strategy is adopted to diversify the population when necessary by adding a random perturbation to the velocity of each particle according to a probability controlled by the diversity of the current population. In addition, a stochastic variable neighborhood search is used as the local search to improve the search intensification. Computational results using benchmark problems show that the proposed DPSO algorithm outperforms previous algorithms proposed in the literature and that it can obtain 111 new best known upper bounds for the 120 benchmark problems.     

Developing new fitness functions in genetic programming for classification with unbalanced data

Machine learning algorithms such as genetic programming (GP) can evolve biased classifiers when data sets are unbalanced. Data sets are unbalanced when at least one class is represented by only a small number of training examples (called the minority class) while other classes make up the majority. In this scenario, classifiers can have good accuracy on the majority class but very poor accuracy on the minority class(es) due to the influence that the larger majority class has on traditional training criteria in the fitness function. This paper aims to both highlight the limitations of the current GP approaches in this area and develop several new fitness functions for binary classification with unbalanced data. Using a range of real-world classification problems with class imbalance, we empirically show that these new fitness functions evolve classifiers with good performance on both the minority and majority classes. Our approaches use the original unbalanced training data in the GP learning process, without the need to artificially balance the training examples from the two classes (e.g., via sampling).     

Incremental learning for online tool condition monitoring using Ellipsoid ARTMAP network model

In this paper, an Ellipsoid ARTMAP (EAM) network model based on incremental learning algorithm is proposed to realize online learning and tool condition monitoring. The main characteristic of EAM model is that hyper-ellipsoid is used for geometric representation of categories which can depict the sample distribution robustly and accurately. Meanwhile, adaptive resonance based strategy can realize the update of the hyper-ellipsoid node locally and monotonically. Therefore, the model has strong incremental learning ability, which guarantees the constructed classifier can learn new knowledge without forgetting the original information. Based on incremental EAM model, a tool condition monitoring system is realized. In this system, features are firstly extracted from the force and vibration signals to depict dynamic features of tool wear process. Then, fast correlation based filter (FCBF) method is introduced to select the minimum redundant features adaptively so as to decrease the feature redundancy and improve classifier robustness. Based on the selected features, EAM based incremental classifier is constructed to realize recognition of the tool wear states. To show the effectiveness of the proposed method, multi-teeth milling experiments of Ti-6Al-4V alloy were carried out. Moreover, to estimate the generation error of the classifiers accurately, a five-fold cross validation method is utilized. By comparison with the commonly used Fuzzy ARTMAP (FAM) classifier, it can be shown that the averaging recognition rate of EAM initial classifier can reach 98.67%, which is higher than FAM. Moreover, the incremental learning ability of EAM is also analyzed and compared with FAM using the new data coming from different cutting passes and tool wear category. The results show that the updated EAM classifier can get higher classification accuracy on the original knowledge while realizing effective online learning of the new knowledge.     

A comparative study of classifier ensembles for bankruptcy prediction

The aim of bankruptcy prediction in the areas of data mining and machine learning is to develop an effective model which can provide the higher prediction accuracy. In the prior literature, various classification techniques have been developed and studied, in/with which classifier ensembles by combining multiple classifiers approach have shown their outperformance over many single classifiers. However, in terms of constructing classifier ensembles, there are three critical issues which can affect their performance. The first one is the classification technique actually used/adopted, and the other two are the combination method to combine multiple classifiers and the number of classifiers to be combined, respectively. Since there are limited, relevant studies examining these aforementioned disuses, this paper conducts a comprehensive study of comparing classifier ensembles by three widely used classification techniques including multilayer perceptron (MLP) neural networks, support vector machines (SVM), and decision trees (DT) based on two well-known combination methods including bagging and boosting and different numbers of combined classifiers. Our experimental results by three public datasets show that DT ensembles composed of 80–100 classifiers using the boosting method perform best. The Wilcoxon signed ranked test also demonstrates that DT ensembles by boosting perform significantly different from the other classifier ensembles. Moreover, a further study over a real-world case by a Taiwan bankruptcy dataset was conducted, which also demonstrates the superiority of DT ensembles by boosting over the others.     

基于粒子群优化算法的支持向量机参数选择

由于支持向量机的主要参数的选择能够在很大程度上影响分类性能和效果,并且目前参数优化缺乏理论指导,提出一种粒子群优化算法以优化支持向量机参数的方法.该方法通过引入非线性递减惯性权值和异步线性变化的学习因子策略来改善标准粒子群算法的后期收敛速度慢、易陷入局部最优的缺陷.实验结果表明,相对于标准粒子群算法,本方法在参数优化方面具有良好的鲁棒性、快速收敛和全局搜索能力,具有更高的分类精确度和效率.     

层次学习骨干粒子群优化算法

对骨干粒子群优化(BPSO) 种群多样性迅速丧失的原因进行分析, 提出层次学习骨干粒子群优化算法以克 服早熟现象. 该算法中粒子依不同的学习概率向粒子自身的最优粒子、优胜粒子和群体最优粒子学习, 该机制使群 体实现不同层次的搜索并有效维持群体的多样性. 此外, 群体最优粒子依概率采用跳跃策略以增强逃逸能力或采用 扰动策略以提高解的质量. 将所提出的算法与多种改进的粒子群优化算法进行对比, 仿真结果表明, 所提出算法的综 合表现优于其他算法.     

The effects of combining classifiers with the same training statistics using Bayesian decision rules

Multiple classifier systems (MCS) are attracting increasing interest in the field of pattern recognition and machine learning. Recently, MCS are also being introduced in the remote sensing field where the importance of classifier diversity for image classification problems has not been examined. In this article, Satellite Pour l'Observation de la Terre (SPOT) IV panchromatic and multispectral satellite images are classified into six land cover classes using five base classifiers: contextual classifier, k-nearest neighbour classifier, Mahalanobis classifier, maximum likelihood classifier and minimum distance classifier. The five base classifiers are trained with the same feature sets throughout the experiments and a posteriori probability, derived from the confusion matrix of these base classifiers, is applied to five Bayesian decision rules (product rule, sum rule, maximum rule, minimum rule and median rule) for constructing different combinations of classifier ensembles. The performance of these classifier ensembles is evaluated for overall accuracy and kappa statistics. Three statistical tests, the McNemar's test, the Cochran's Q test and the Looney's F-test, are used to examine the diversity of the classification results of the base classifiers compared to the results of the classifier ensembles. The experimental comparison reveals that (a) significant diversity amongst the base classifiers cannot enhance the performance of classifier ensembles; (b) accuracy improvement of classifier ensembles can only be found by using base classifiers with similar and low accuracy; (c) increasing the number of base classifiers cannot improve the overall accuracy of the MCS and (d) none of the Bayesian decision rules outperforms the others.     

基于扩散机制的双种群粒子群优化算法*

为了避免标准粒子群优化算法（PSO）过早收敛的缺点,把热力学中的扩散现象引入到PSO算法的改进当中,提出了基于扩散机制的双种群粒子群优化算法(DPSO)。DPSO算法中定义了粒子的扩散能、种群的温度和粒子的扩散概率三个概念,两个群体中的粒子在进化过程中根据粒子的扩散概率被选入到各自种群的扩散池中,从而实现两个种群之间信息的交换和共享。通过解决典型的多峰、高维函数优化问题来证实DPSO算法的有效性,实验结果表明DPSO比标准PSO具有更高的性能。     

一种限制输出模型规模的集成进化分类算法

AdaBoost算法是一种典型的集成学习框架,通过线性组合若干个弱分类器来构造成强学习器,其分类精度远高于单个弱分类器,具有很好的泛化误差和训练误差。然而AdaBoost 算法不能精简输出模型的弱分类器,因而不具备良好的可解释性。本文将遗传算法引入AdaBoost算法模型,提出了一种限制输出模型规模的集成进化分类算法(Ensemble evolve classification algorithm for controlling the size of final model,ECSM)。通过基因操作和评价函数能够在AdaBoost迭代框架下强制保留物种样本的多样性,并留下更好的分类器。实验结果表明,本文提出的算法与经典的AdaBoost算法相比,在基本保持分类精度的前提下,大大减少了分类器数量。     

解决约束优化问题的改进粒子群算法

针对约束优化问题的求解,提出一种改进的粒子群算法（CMPSO）。在CMPSO算法中,为了增加种群多样性,提升种群跳出局部最优解的能力,引入种群多样性阈值,当种群多样性低于给定阈值时,对全局最优粒子位置和粒子自身最优位置进行多项式变异;并根据粒子违背约束条件的程度,提出一种新的粒子间比较准则来比较粒子间的优劣,该准则可以保留一部分性能较优的不可行解;为提升种群向全局最优解飞行的概率,采取一种广义学习策略。对经典测试函数的仿真结果表明,所提出的算法是一种可行的约束优化问题的求解方法。     

基于多因子粒子群的高维数据特征选择算法

特征选择是机器学习和数据挖掘领域中一项重要的数据预处理技术,它旨在最大化分类任务的精度和最小化最优子集特征个数。运用粒子群算法在高维数据集中寻找最优子集面临着陷入局部最优和计算代价昂贵的问题,导致分类精度下降。针对此问题,提出了基于多因子粒子群算法的高维数据特征选择算法。引入了进化多任务的算法框架,提出了一种两任务模型生成的策略,通过任务间的知识迁移加强种群交流,提高种群多样性以改善易陷入局部最优的缺陷;设计了基于稀疏表示的初始化策略,在算法初始阶段设计具有稀疏表示的初始解,降低了种群在趋向最优解集时的计算开销。在6个公开医学高维数据集上的实验结果表明,所提算法能够有效实现分类任务且得到较好的精度。     

A Particle Swarm Optimization Based Deep Learning Model for Vehicle Classification

Image classification is a core field in the research area of image processing and computer vision in which vehicle classification is a critical domain. The purpose of vehicle categorization is to formulate a compact system to assist in real-world problems and applications such as security, traffic analysis, and self-driving and autonomous vehicles. The recent revolution in the field of machine learning and artificial intelligence has provided an immense amount of support for image processing related problems and has overtaken the conventional, and handcrafted means of solving image analysis problems. In this paper, a combination of pre-trained CNN GoogleNet and a nature-inspired problem optimization scheme, particle swarm optimization (PSO), was employed for autonomous vehicle classification. The model was trained on a vehicle image dataset obtained from Kaggle that has been suitably augmented. The trained model was classified using several classifiers; however, the Cubic SVM (CSVM) classifier was found to outperform the others in both time consumption and accuracy (94.8%). The results obtained from empirical evaluations and statistical tests reveal that the model itself has shown to outperform the other related models not only in terms of accuracy (94.8%) but also in terms of training time (82.7 s) and speed prediction (380 obs/sec).     

基于粒子群蚁群算法的供应链合作伙伴选择研究

针对供应链合作伙伴选择的准确性和效率问题,提出一种基于粒子群和蚁群优化的合作伙伴选择算法。建立基于供应链链节体和连接弧的有向图路径模型,构造多目标规划模型。利用改进的离散型粒子群算法,求取伙伴选择问题的初始解集,构建初始信息素矩阵,通过改进蚁群算法的寻径规则,求取供应链合作伙伴选择问题的最优解。实验结果表明,所提算法有效提高了供应链合作伙伴选择的精度和效率,具有较好的性能。     

求解TSP问题的自逃逸混合离散粒子群算法研究

通过对旅行商问题（TSP）局部最优解与个体最优解、群体最优解之间的关系分析,针对DPSO算法易早熟和收敛慢的缺点,重新定义了离散粒子群DPSO的速度、位置公式,结合生物界中物种在生存密度过大时个体会自动分散迁徙的特性和局部搜索算法（SEC）后,提出了一种新的自逃逸混合离散粒子群算法（SEHDPSO）.自逃逸思想是一种确定性变异操作,能使算法中陷入局部极小区域的粒子通过自逃逸行为进行全局寻优,从而克服算法易早熟的缺陷.仿真结果表明,SEHDPSO算法比混合蚁群算法（ACS＋2-OPT）具有更好的收敛性和搜索效率.