Data mining methods for knowledge discovery in multi-objective optimization: Part A - Survey

Real-world optimization problems typically involve multiple objectives to be optimized simultaneously under multiple constraints and with respect to several variables. While multi-objective optimization itself can be a challenging task, equally difficult is the ability to make sense of the obtained solutions. In this two-part paper, we deal with data mining methods that can be applied to extract knowledge about multi-objective optimization problems from the solutions generated during optimization. This knowledge is expected to provide deeper insights about the problem to the decision maker, in addition to assisting the optimization process in future design iterations through an expert system. The current paper surveys several existing data mining methods and classifies them by methodology and type of knowledge discovered. Most of these methods come from the domain of exploratory data analysis and can be applied to any multivariate data. We specifically look at methods that can generate explicit knowledge in a machine-usable form. A framework for knowledge-driven optimization is proposed, which involves both online and offline elements of knowledge discovery. One of the conclusions of this survey is that while there are a number of data mining methods that can deal with data involving continuous variables, only a few ad hoc methods exist that can provide explicit knowledge when the variables involved are of a discrete nature. Part B of this paper proposes new techniques that can be used with such datasets and applies them to discrete variable multi-objective problems related to production systems.     

基于决策空间变换最近邻方法的Pareto支配性预测

为提高在决策空间运用最近邻方法预测多目标优化Pareto支配性的精度，提出一种基于决策空间变换的最近邻预测方法.在分析目标函数与决策分量相关性的基础上，提出属性变化趋势模型的构造方法，建立低计算成本的属性趋势代理模型.通过属性趋势模型引入决策空间到目标空间的映射知识，对多目标问题的决策空间进行变换，使决策空间的最近邻更有效反映目标空间的最近邻.选取具有不同相关系数特征的典型多目标优化问题，进行Pareto支配性预测的可对比实验，结果表明在新空间中运用最近邻方法可显著提高分类准确性.     

核属性蚁群算法的规则获取

蚁群算法是一种新型的模拟进化算法,研究已经表明该算法具有许多优良的性质,并且在优化计算中已得到了很多应用.粗糙集理论作为一种智能数据分析和数据挖掘的新的数学工具,其主要优点在于它不需要任何关于被处理数据的先验或额外知识.本文从规则获取和优化两方面研究基于粗糙集理论和蚁群算法的分类规则挖掘方法.通过研究决策表和决策规则系数,建立基于粗糙集表示和度量的知识理论,将粗糙集理论与蚁群算法融合,采用粗糙集理论进行属性约简,利用蚁群算法获取最优分类规则,优势互补.实验结果比较表明,算法获取的分类规则,具有良好的预测能力和更为简洁的表示形式.     

Multi-objective optimization problem under fuzzy rule constraints using particle swarm optimization

In this paper, a fuzzy multi-objective programming problem is considered where functional relationships between decision variables and objective functions are not completely known to us. Due to uncertainty in real decision situations sometimes it is difficult to find the exact functional relationship between objectives and decision variables. It is assumed that information source from where some knowledge may be obtained about the objective functions consists of a block of fuzzy if-then rules. In such situations, the decision making is difficult and the presence of multiple objectives gives rise to multi-objective optimization problem under fuzzy rule constraints. In order to tackle the problem, appropriate fuzzy reasoning schemes are used to determine crisp functional relationship between the objective functions and the decision variables. Thus a multi-objective optimization problem is formulated from the original fuzzy rule-based multi-objective optimization model. In order to solve the resultant problem, a deterministic single-objective non-linear optimization problem is reformulated with the help of fuzzy optimization technique. Finally, PSO (Particle Swarm Optimization) algorithm is employed to solve the resultant single-objective non-linear optimization model and the computation procedure is illustrated by means of numerical examples.     

基于项集归减的高维频繁高效用项集挖掘多目标优化方法

频繁高效用项集挖掘是数据挖掘的一项重要任务,挖掘到的项集由支持度和效用这2个指标衡量.在一系列用于解决这类问题的方法中,进化多目标方法能够提供1组高质量解以满足不同用户的需求,避免传统算法中支持度和效用的阈值难以确定的问题.但是已有多目标算法多采用0-1编码,使得决策空间的维度与数据集中项数成正比,因此,面对高维数据集会出现维度灾难问题.鉴于此,设计一种项集归减策略,通过在进化过程中不断对不重要项进行归减以减小搜索空间.基于此策略,进而提出一种基于项集归减的高维频繁高效用项集挖掘多目标优化算法(IR-MOEA),并针对可能存在的归减过度或未归减到位的个体提出基于学习的种群修复策略用以调整进化方向.此外还提出一种基于项集适应度的初始化策略,使得算法在进化初期生成利于后期进化的稀疏解.多个数据集上的实验结果表明,所提出算法优于现有的多目标优化算法,特别是在高维数据集上.     

Data mining based job dispatching using hybrid simulation-optimization approach for shop scheduling problem

A data mining based approach to discover previously unknown priority dispatching rules for job shop scheduling problem is presented. This approach is based on seeking the knowledge that is assumed to be embedded in the efficient solutions provided by the optimization module built using tabu search. The objective is to discover the scheduling concepts using data mining and hence to obtain a set of rules capable of approximating the efficient solutions for a job shop scheduling problem (JSSP). A data mining based scheduling framework is presented and implemented for a job shop problem with maximum lateness as the scheduling objective.     

Pattern filtering and classification for market basket analysis with profit‐based measures

Market basket analysis is one of the typical applications in mining association rules. The valuable information discovered from data mining can be used to support decision making. Generally, support and confidence (objective) measures are used to evaluate the interestingness of association rules. However, in some cases, by using these two measures, the discovered rules may be not profitable and not actionable (not interesting) to enterprises. Therefore, how to discover the patterns by considering both objective measures (e.g. probability) and subjective measures (e.g. profit) is a challenge in data mining, particularly in marketing applications. This paper focuses on pattern evaluation in the process of knowledge discovery by using the concept of profit mining. Data Envelopment Analysis is utilized to calculate the efficiency of discovered association rules with multiple objective and subjective measures. After evaluating the efficiency of association rules, they are categorized into two classes, relatively efficient (interesting) and relatively inefficient (uninteresting). To classify these two classes, Decision Tree (DT)‐based classifier is built by using the attributes of association rules. The DT classifier can be used to find out the characteristics of interesting association rules, and to classify the unknown (new) association rules.     

A multi-objective genetic algorithm approach to rule mining for affective product design

A novel multi-objective genetic algorithm (GA)-based rule-mining method for affective product design is proposed to discover a set of rules relating design attributes with customer evaluation based on survey data. The proposed method can generate approximate rules to consider the ambiguity of customer assessments. The generated rules can be used to determine the lower and upper limits of the affective effect of design patterns. For a rule-mining problem, the proposed multi-objective GA approach could simultaneously consider the accuracy, comprehensibility, and definability of approximate rules. In addition, the proposed approach can deal with categorical attributes and quantitative attributes, and determine the interval of quantitative attributes. Categorical and quantitative attributes in affective product design should be considered because they are commonly used to define the design profile of a product. In this paper, a two-stage rule-mining approach is proposed to generate rules with a simple chromosome design in the first stage of rule mining. In the second stage of rule mining, entire rule sets are refined to determine solutions considering rule interaction. A case study on mobile phones is used to demonstrate and validate the performance of the proposed rule-mining method. The method can discover rule sets with good support and coverage rates from the survey data.     

基于值约简和决策树的最简规则提取算法

粗糙集理论中的值约简和数据挖掘领域中的决策树都是有效的分类方法,但二者都有其局限性。将这两种方法结合起来,生成一种新的基于值核的极小化方法对决策树进行修剪,提出了约简规则的判定准则,缩小了约简的范围,最后再对生成的规则进行极大化处理,以保证规则覆盖信息的一致性,实验验证了该算法的有效性。     

基于多目标优化的网络社区发现方法

社区发现是复杂网络挖掘中的重要任务之一,在恐怖组织识别、蛋白质功能预测、舆情分析等方面具有重要的理论和应用价值.但是,现有的社区质量评判指标具有数据依赖性与耦合关联性,而且基于单一评判指标优化的网络社区发现算法有很大的局限性.针对这些问题,将网络社区发现问题形式化为多目标优化问题,提出了一种基于多目标粒子群优化的网络社区发现算法MOCD-PSO,它选取模块度Q、最小最大割MinMaxCut 与轮廓(silhouette)这3 个指标进行综合寻优.实验结果表明,MOCD-PSO 算法具有较好的收敛性,能够发现分布均匀且分散度较高的Pareto 最优网络社区结构集,并且无论与单目标优化方法(GN 与GA-Net)相比较,还是与多目标优化算法(MOGANet与SCAH-MOHSA)相比较,MOCD-PSO 算法都能在无先验信息的条件下挖掘出更高质量的网络社区.     

区间多目标优化中决策空间约束、支配及同序解筛选策略

针对优化函数未知的昂贵区间多目标优化, 根据决策空间数据挖掘, 提出了一种基于最近邻法和主成分分析法(Principal component analysis, PCA)的NSGA-II算法. 该算法首先通过约束条件将待测解集分为可行解和非可行解, 利用最近邻法对待测解和样本解进行相似性计算, 判断待测解是否满足约束. 然后对于两个解的Pareto支配性同样利用最近邻法来区分解之间的被支配和非被支配关系. 由于目标空间拥挤距离无法求出, 为此在决策空间利用主成分分析法将K-均值聚类后的解集降维, 找出待测解的前、后近距离解, 通过决策空间拥挤距离对同序值解进行筛选. 实现NSGA-II算法的改进.     

Mining fuzzy association rules for classification problems

The effective development of data mining techniques for the discovery of knowledge from training samples for classification problems in industrial engineering is necessary in applications, such as group technology. This paper proposes a learning algorithm, which can be viewed as a knowledge acquisition tool, to effectively discover fuzzy association rules for classification problems. The consequence part of each rule is one class label. The proposed learning algorithm consists of two phases: one to generate large fuzzy grids from training samples by fuzzy partitioning in each attribute, and the other to generate fuzzy association rules for classification problems by large fuzzy grids. The proposed learning algorithm is implemented by scanning training samples stored in a database only once and applying a sequence of Boolean operations to generate fuzzy grids and fuzzy rules; therefore, it can be easily extended to discover other types of fuzzy association rules. The simulation results from the iris data demonstrate that the proposed learning algorithm can effectively derive fuzzy association rules for classification problems.     

基于等价分量交叉相似性的Pareto支配性预测

研究用最近邻分类预测多目标优化问题Pareto支配性的相似性测度方法. 在分析决策分量对各目标分量贡献率的基础上定义决策向量的等价子向量,等价子向量由贡献率相同的决策分量所组成.提出基于等价子向量的最小交叉距离加 权和相似性测度方法.对每个目标分量,独立评价待测数据与N个已知样本的相似度,每个样本按其相似度值的升序赋予[0:N-1]之间的序号,按各目标上的序号之和最小准则确定最近邻样本.等价子向量最小交叉距离加权和相似性测度以及多目标最近邻搜索方法在确定决策向量相似性时,引入了决策空间到目标向量空间的映射知识,使决策变量相似性测度更真实地反映目标向量相似性.对典型多目标优化问题的Pareto支配性最近邻分类实验结果表明,提出的方法可显著地提高分类准确性.     

基于多目标PSO算法的信息工程监理

为了解决信息化工程监理面临的综合管理的复杂性, 通过综合考虑信息工程监理过程中对质量、投资、进度的控制, 采用以资源作为决策变量, 以整体工期优化为目标, 建立一种信息工程监理过程多目标优化的数学模型. 针对该数学模型, 构建了信息工程监理控制优化的多目标决策问题的目标函数, 结合一种多目标离散粒子群进化算法, 根据具体问题的特点, 重新定义和设计新的粒子进化方程, 从而较好地解决多目标优化信息工程监理控制目标的问题.     

交互式多目标文化算法优化多模态混合指标

多模态混合指标优化是一类难以求解的多目标优化问题。针对该问题,借鉴文化算法的双层结构,构建了一种能融合历史知识、标准化知识和领域知识的交互式文化算法。该算法以指标均衡性构建信度空间样本库。知识提取函数根据样本库内个体在决策空间和目标空间的特殊拥挤距离选取多模态解。将选取的多模态解作为聚类中心推荐给用户评价。根据种群的指标均衡性,知识引导自适应交叉和变异概率,扩大种群多样性。采用指标均衡性引导形势知识更新。基于个体表现型相似性估计大规模种群隐式性能指标。提出新的多模态解评价测度。将算法应用于室内布局优化问题,与代表性方法比较,验证所提算法的有效性和可用性。     

基于粗糙集的多维关联规则挖掘方法

海量的数据使得关联规则挖掘非常耗时,而并非所有的规则都是用户感兴趣的,应用传统的挖掘方法会挖掘出许多无关信息。此外,目前大部分算法是针对单维规则的。因此,定义了一种挖掘语言使得用户可以指定感兴趣的项以及关联规则的参数（如支持度,置信度等）,并提出一种基于粗糙集理论的多维关联规则挖掘方法,动态生成频繁集和多维关联规则,减少频繁项集的生成搜索空间。实例分析验证该算法的可行性与有效性。     

Multiobjective optimisation on a budget: Exploring surrogate modelling for robust multi-reservoir rules generation under hydrological uncertainty

Developing long term operation rules for multi-reservoir systems is complicated due to the number of decision variables, the non-linearity of system dynamics and the hydrological uncertainty. This uncertainty can be addressed by coupling simulation models with multi-objective optimisation algorithms driven by stochastically generated hydrological timeseries but the computational effort required imposes barriers to the exploration of the solution space. The paper addresses this by (a) employing a parsimonious multi-objective parameterization-simulation-optimization (PSO) framework, which incorporates hydrological uncertainty through stochastic simulation and allows the use of probabilistic objective functions and (b) by investigating the potential of multi-objective surrogate based optimisation (MOSBO) to significantly reduce the resulting computational effort. Three MOSBO algorithms are compared against two multi-objective evolutionary algorithms. Results suggest that MOSBOs are indeed able to provide robust, uncertainty-aware operation rules much faster, without significant loss of neither the generality of evolutionary algorithms nor of the knowledge embedded in domain-specific models.     

Knowledge Discovery with Genetic Programming for Providing Feedback to Courseware Authors

We introduce a methodology to improve Adaptive Systems for Web-Based Education. This methodology uses evolutionary algorithms as a data mining method for discovering interesting relationships in students usage data. Such knowledge may be very useful for teachers and course authors to select the most appropriate modifications to improve the effectiveness of the course. We use Grammar-Based Genetic Programming (GBGP) with multi-objective optimization techniques to discover prediction rules. We present a specific data mining tool that can help non-experts in data mining carry out the complete rule discovery process, and demonstrate its utility by applying it to an adaptive Linux course that we developed.