Multi-objective rule mining using a chaotic particle swarm optimization algorithm

In this paper, classification rule mining which is one of the most studied tasks in data mining community has been modeled as a multi-objective optimization problem with predictive accuracy and comprehensibility objectives. A multi-objective chaotic particle swarm optimization (PSO) method has been introduced as a search strategy to mine classification rules within datasets. The used extension to PSO uses similarity measure for neighborhood and far-neighborhood search to store the global best particles found in multi-objective manner. For the bi-objective problem of rule mining of high accuracy/comprehensibility, the multi-objective approach is intended to allow the PSO algorithm to return an approximation to the upper accuracy/comprehensibility border, containing solutions that are spread across the border. The experimental results show the efficiency of the algorithm.     

Predictive and comprehensible rule discovery using a multi-objective genetic algorithm

We present a multi-objective genetic algorithm for mining highly predictive and comprehensible classification rules from large databases. We emphasize predictive accuracy and comprehensibility of the rules. However, accuracy and comprehensibility of the rules often conflict with each other. This makes it an optimization problem that is very difficult to solve efficiently. We have proposed a multi-objective evolutionary algorithm called improved niched Pareto genetic algorithm (INPGA) for this purpose. We have compared the rule generation by INPGA with that by simple genetic algorithm (SGA) and basic niched Pareto genetic algorithm (NPGA). The experimental result confirms that our rule generation has a clear edge over SGA and NPGA.     

Multiple Instance Learning with Multiple Objective Genetic Programming for Web Mining

This paper introduces a multi-objective grammar based genetic programming algorithm, MOG3P-MI, to solve a Web Mining problem from the perspective of multiple instance learning. This algorithm is evaluated and compared to other algorithms that were previously used to solve this problem. Computational experiments show that the MOG3P-MI algorithm obtains the best results, adds comprehensibility and clarity to the knowledge discovery process and overcomes the main drawbacks of previous techniques obtaining solutions which maintain a balance between conflicting measurements like sensitivity and specificity.     

New mechanism for archive maintenance in PSO-based multi-objective feature selection

In classification problems, a large number of features are typically used to describe the problem’s instances. However, not all of these features are useful for classification. Feature selection is usually an important pre-processing step to overcome the problem of “curse of dimensionality”. Feature selection aims to choose a small number of features to achieve similar or better classification performance than using all features. This paper presents a particle swarm Optimization (PSO)-based multi-objective feature selection approach to evolving a set of non-dominated feature subsets which achieve high classification performance. The proposed algorithm uses local search techniques to improve a Pareto front and is compared with a pure multi-objective PSO algorithm, three well-known evolutionary multi-objective algorithms and a current state-of-the-art PSO-based multi-objective feature selection approach. Their performances are examined on 12 benchmark datasets. The experimental results show that in most cases, the proposed multi-objective algorithm generates better Pareto fronts than all other methods.     

Artificial bee colony-based support vector machines with feature selection and parameter optimization for rule extraction

Support vector machine (SVM) is a state-of-art classification tool with good accuracy due to its ability to generate nonlinear model. However, the nonlinear models generated are typically regarded as incomprehensible black-box models. This lack of explanatory ability is a serious problem for practical SVM applications which require comprehensibility. Therefore, this study applies a C5 decision tree (DT) to extract rules from SVM result. In addition, a metaheuristic algorithm is employed for the feature selection. Both SVM and C5 DT require expensive computation. Applying these two algorithms simultaneously for high-dimensional data will increase the computational cost. This study applies artificial bee colony optimization (ABC) algorithm to select the important features. The proposed algorithm ABC–SVM–DT is applied to extract comprehensible rules from SVMs. The ABC algorithm is applied to implement feature selection and parameter optimization before SVM–DT. The proposed algorithm is evaluated using eight datasets to demonstrate the effectiveness of the proposed algorithm. The result shows that the classification accuracy and complexity of the final decision tree can be improved simultaneously by the proposed ABC–SVM–DT algorithm, compared with genetic algorithm and particle swarm optimization algorithm.     

The role of algorithm and result comprehensibility of automated scheduling on complacency

Several studies have stressed that even expert operators who are aware of a machine's limits could adopt its proposals without questioning them (i.e., the complacency phenomenon). In production scheduling for manufacturing, this is a significant problem, as it is often suggested that the machine be allowed to build the production schedule, confining the human role to that of rescheduling. This article evaluates the characteristics of scheduling algorithms on human rescheduling performance, the quality of which was related to complacency. It is suggested that scheduling algorithms be characterized as having result comprehensibility (the result respects the scheduler's expectations in terms of the discourse rules of the information display) or algorithm comprehensibility (the complexity of the algorithm hides some important constraints). The findings stress, on the one hand, that result comprehensibility is necessary to achieve good production performance and to limit complacency. On the other hand, algorithm comprehensibility leads to poor performance due to the very high cost of understanding the algorithm. © 2008 Wiley Periodicals, Inc.     

An investigation into the application of neural networks,fuzzy logic,genetic algorithms,and rough sets to automated knowledge acquisition for classification problems

This paper presents some highlights in the application of neural networks, fuzzy logic, genetic algorithms, and rough sets to automated knowledge acquisition. These techniques are capable of dealing with inexact and imprecise problem domains and have been demonstrated to be useful in the solution of classification problems. It addresses the issue of the application of appropriate evaluation criteria such as rule base accuracy and comprehensibility for new knowledge acquisition techniques. An empirical study is then described in which three approaches to knowledge acquisition are investigated. The first approach combines neural networks and fuzzy logic, the second, genetic algorithms and fuzzy logic, and in the third a rough sets approach has been examined, and compared. In this study neural network and genetic algorithm fuzzy rule induction systems have been developed and applied to three classification problems. Rule induction software based on rough sets theory was also used to generate and test rule bases for the same data. A comparison of these approaches with the C4.5 inductive algorithm was also carried out. Our research to date indicates that, based on the evaluation criteria used, the genetic/fuzzy approach compares more than favourably with the neuro/fuzzy and rough set approaches. On the data sets used the genetic algorithm system displays a higher accuracy of classification and rule base comprehensibility than the C4.5 inductive algorithm.     

基于双标签集的标签匹配集成学习算法

当标识示例的两个标签分别来源于两个标签集时,这种多标签分类问题称之为标签匹配问题,目前还没有针对标签匹配问题的学习算法。 尽管可以用传统的多标签分类学习算法来解决标签匹配问题,但显然标签匹配问题有其自身特殊性。 通过对标签匹配问题进行深入的研究,在连续AdaBoost(real Adaptive Boosting)算法的基础上,基于整体优化的思想,采用算法适应的方法,提出了基于双标签集的标签匹配集成学习算法,该算法能够较好地学习到标签匹配规律从而完成标签匹配。 实验结果表明,与传统的多标签学习算法用于解决标签匹配问题相比,提出的新算法不仅缩小了搜索的标签空间的范围,而且最小化学习误差可以随着分类器个数的增加而降低,进而使得标签匹配分类更加快速、准确。     

A multi-objective model for discovering high-quality knowledge based on data quality and prior knowledge

Discovering knowledge from data means finding useful patterns in data, this process has increased the opportunity and challenge for businesses in the big data era. Meanwhile, improving the quality of the discovered knowledge is important for making correct decisions in an unpredictable environment. Various models have been developed in the past; however, few used both data quality and prior knowledge to control the quality of the discovery processes and results. In this paper, a multi-objective model of knowledge discovery in databases is developed, which aids the discovery process by utilizing prior process knowledge and different measures of data quality. To illustrate the model, association rule mining is considered and formulated as a multi-objective problem that takes into account data quality measures and prior process knowledge instead of a single objective problem. Measures such as confidence, support, comprehensibility and interestingness are used. A Pareto-based integrated multi-objective Artificial Bee Colony (IMOABC) algorithm is developed to solve the problem. Using well-known and publicly available databases, experiments are carried out to compare the performance of IMOABC with NSGA-II, MOPSO and Apriori algorithms, respectively. The computational results show that IMOABC outperforms NSGA-II, MOPSO and Apriori on different measures and it could be easily customized or tailored to be in line with user requirements and still generates high-quality association rules.     

An effective hybrid particle swarm optimization algorithm for multi-objective flexible job-shop scheduling problem

Flexible job-shop scheduling problem (FJSP) is an extension of the classical job-shop scheduling problem. Although the traditional optimization algorithms could obtain preferable results in solving the mono-objective FJSP. However, they are very difficult to solve multi-objective FJSP very well. In this paper, a particle swarm optimization (PSO) algorithm and a tabu search (TS) algorithm are combined to solve the multi-objective FJSP with several conflicting and incommensurable objectives. PSO which integrates local search and global search scheme possesses high search efficiency. And, TS is a meta-heuristic which is designed for finding a near optimal solution of combinatorial optimization problems. Through reasonably hybridizing the two optimization algorithms, an effective hybrid approach for the multi-objective FJSP has been proposed. The computational results have proved that the proposed hybrid algorithm is an efficient and effective approach to solve the multi-objective FJSP, especially for the problems on a large scale.     

修正免疫克隆约束多目标优化算法

针对约束多目标优化问题,提出修正免疫克隆约束多目标优化算法.该算法通过引进一个约束处理策略,用一个修正算法对个体的目标函数值进行修正,并对修正后的目标函数值采用免疫克隆算法进行优化,用一个精英种群对可行非支配解进行存储.该算法在优化过程中,既保留了非支配可行解,也充分利用了约束偏离值小的非可行解,同时引进整体克隆策略来提高解分布的多样性.通过对约束多目标问题的各项性能指标的测试以及和对比算法的比较可以看出:该算法在处理约束多目标优化测试问题时,所得解的多样性得到了一定的提高.同时,解的收敛性和均匀性也得到了一定的改进.     

Seru系统调度优化的知识引导协同进化算法

作为一种新型的生产模式, Seru系统能够兼顾柔性和效率且快速响应市场, 已在装配企业得到广泛应用.为了实现实际生产过程生产效率和劳动效率的协同优化, 本文研究以最小化最大完工时间和工人总劳动时间为目标的Seru系统多目标调度问题, 提出一种知识引导的协同进化算法. 首先, 将问题分解为Seru构造和Seru调度, 构造两个种群分别优化子问题. 同时, 设计种群规模的调整策略, 通过为有潜力的种群分配更多个体来提高协同搜索的效率. 进而, 通过分析问题的性质, 提炼规则性知识用于设计有效的搜索算子和重生成规则, 指导精英个体执行知识驱动的增强搜索, 从而进一步提升算法的局部开发能力. 通过数值仿真和统计性能对比, 验证了算法各设计环节的有效性, 并取得了显著优于现有最新算法的多目标调度优化性能.     

Multi-objective genetic algorithm and its applications to flowshop scheduling

In this paper, we propose a multi-objective genetic algorithm and apply it to flowshop scheduling. The characteristic features of our algorithm are its selection procedure and elite preserve strategy. The selection procedure in our multi-objective genetic algorithm selects individuals for a crossover operation based on a weighted sum of multiple objective functions with variable weights. The elite preserve strategy in our algorithm uses multiple elite solutions instead of a single elite solution. That is, a certain number of individuals are selected from a tentative set of Pareto optimal solutions and inherited to the next generation as elite individuals. In order to show that our approach can handle multi-objective optimization problems with concave Pareto fronts, we apply the proposed genetic algorithm to a two-objective function optimization problem with a concave Pareto front. Last, the performance of our multi-objective genetic algorithm is examined by applying it to the flowshop scheduling problem with two objectives: to minimize the makespan and to minimize the total tardiness. We also apply our algorithm to the flowshop scheduling problem with three objectives: to minimize the makespan, to minimize the total tardiness, and to minimize the total flowtime.     

Hybrid algorithms for the twin–screw extrusion configuration problem

The twin-screw configuration problem (TSCP) arises in the context of polymer processing, where twin-screw extruders are used to prepare polymer blends, compounds or composites. The goal of the TSCP is to define the configuration of a screw from a given set of screw elements. The TSCP can be seen as a sequencing problem as the order of the screw elements on the screw axis has to be defined. It is also inherently a multi-objective problem since processing has to optimize various conflicting parameters related to the degree of mixing, shear rate, or mechanical energy input among others. In this article, we develop hybrid algorithms to tackle the bi-objective TSCP. The hybrid algorithms combine different local search procedures, including Pareto local search and two phase local search algorithms, with two different population-based algorithms, namely a multi-objective evolutionary algorithm and a multi-objective ant colony optimization algorithm. The experimental evaluation of these approaches shows that the best hybrid designs, combining Pareto local search with a multi-objective ant colony optimization approach, outperform the best algorithms that have been previously proposed for the TSCP.     

基于分解的预测型动态多目标粒子群优化算法

针对动态多目标问题求解,提出一种基于分解的预测型动态多目标粒子群优化算法.首先借助分解思想,将目标问题划分为多个不同的子问题,当问题动态变化时,选择对应于不同子问题的优化个体检测环境变化程度,以提高算法对不同动态问题的适应与响应能力;然后,设计一种群体预测策略,通过将目标空间中相同收敛方向上不同时刻的个体位置转换为时间序列,引入时间序列预测方法预测下一刻位置,从而提高预测种群的多样性和有效性,进而有效减少算法在问题变化后的收敛时间;最后,为避免问题发生变化后个体与子问题不匹配,设计一种再匹配策略,以提高预测策略的准确性.实验结果表明,在6个标准动态多目标测试问题上,与2个动态多目标优化算法进行比较,所提出算法在收敛性、分布性与稳定性上均具有显著优势.     

基于项集归减的高维频繁高效用项集挖掘多目标优化方法

频繁高效用项集挖掘是数据挖掘的一项重要任务,挖掘到的项集由支持度和效用这2个指标衡量.在一系列用于解决这类问题的方法中,进化多目标方法能够提供1组高质量解以满足不同用户的需求,避免传统算法中支持度和效用的阈值难以确定的问题.但是已有多目标算法多采用0-1编码,使得决策空间的维度与数据集中项数成正比,因此,面对高维数据集会出现维度灾难问题.鉴于此,设计一种项集归减策略,通过在进化过程中不断对不重要项进行归减以减小搜索空间.基于此策略,进而提出一种基于项集归减的高维频繁高效用项集挖掘多目标优化算法(IR-MOEA),并针对可能存在的归减过度或未归减到位的个体提出基于学习的种群修复策略用以调整进化方向.此外还提出一种基于项集适应度的初始化策略,使得算法在进化初期生成利于后期进化的稀疏解.多个数据集上的实验结果表明,所提出算法优于现有的多目标优化算法,特别是在高维数据集上.     

Hypervolume-based multi-objective local search

This paper presents a multi-objective local search, where the selection is realized according to the hypervolume contribution of solutions. The HBMOLS algorithm proposed is inspired from the IBEA algorithm, an indicator-based multi-objective evolutionary algorithm proposed by Zitzler and Künzli in 2004, where the optimization goal is defined in terms of a binary indicator defining the selection operator. In this paper, we use the indicator optimization principle, and we apply it to an iterated local search algorithm, using hypervolume contribution indicator as selection mechanism. The methodology proposed here has been defined in order to be easily adaptable and to be as parameter-independent as possible. We carry out a range of experiments on the multi-objective flow shop problem and the multi-objective quadratic assignment problem, using the hypervolume contribution selection as well as two different binary indicators which were initially proposed in the IBEA algorithm. Experimental results indicate that the HBMOLS algorithm is highly effective in comparison with the algorithms based on binary indicators.     

Consolidated trees versus bagging when explanation is required

In some real-world problems solved by machine learning it is compulsory for the solution provided to be comprehensible so that the correct decision can be made. It is in this context that this paper compares bagging (one of the most widely used multiple classifier systems) with the consolidated trees construction (CTC) algorithm, when the learning problem to be solved requires the classification made to be provided with an explanation. Bearing in mind the comprehensibility shortcomings of bagging, the Domingos’ proposal, called combining multiple models, has been used to address this problem. The two algorithms have been compared from three main points of view: accuracy, quality of the explanation the classification is provided with, and computational cost. The results obtained show that it is beneficial to use CTC in situations where an explanation is required, because: CTC has a greater discriminating capacity than the explanation extraction algorithm added to bagging; the explanation provided is of a greater quality; it is simpler and more reliable; and CTC is computationally more efficient.     

Conception of a dominance-based multi-objective local search in the context of classification rule mining in large and imbalanced data sets

Classification on medical data raises several problems such as class imbalance, double meaning of missing data, volumetry or need of highly interpretable results. In this paper a new algorithm is proposed: MOCA-I (Multi-Objective Classification Algorithm for Imbalanced data), a multi-objective local search algorithm that is conceived to deal with these issues all together. It is based on a new modelization as a Pittsburgh multi-objective partial classification rule mining problem, which is described in the first part of this paper. An existing dominance-based multi-objective local search (DMLS) is modified to deal with this modelization. After experimentally tuning the parameters of MOCA-I and determining which version of DMLS algorithm is the most effective, the obtained MOCA-I version is compared to several state-of-the-art classification algorithms. This comparison is realized on 10 small and middle-sized data sets of literature and 2 real data sets; MOCA-I obtains the best results on the 10 data sets and is statistically better than other approaches on the real data sets.     

Adaptive multi-objective genetic algorithms for scheduling of drilling operation in printed circuit board industry

In this paper, a scheduling problem for drilling operation in a real-world printed circuit board factory is considered. Two derivatives of multi-objective genetic algorithms are proposed under two objectives, i.e. makespan and total tardiness time. The proposed algorithms possess a rare characteristic from traditional multi-objective genetic algorithms. The crossover and mutation rates of the proposed algorithms can be variables or adjusted according to the searching performance while the rates of traditional algorithm are fixed. Production data retrieved from the shop floor are used as the test instances. The numerical result indicates that both two proposed multi-objective genetic algorithms have satisfactory performance and the adaptive multi-objective genetic algorithm performs better. The result shows the algorithms are effective and efficiency to the current system used in the shop floor. Thus, the result may be of interest to practical applications.