Data mining methods for knowledge discovery in multi-objective optimization: Part B - New developments and applications

The first part of this paper served as a comprehensive survey of data mining methods that have been used to extract knowledge from solutions generated during multi-objective optimization. The current paper addresses three major shortcomings of existing methods, namely, lack of interactiveness in the objective space, inability to handle discrete variables and inability to generate explicit knowledge. Four data mining methods are developed that can discover knowledge in the decision space and visualize it in the objective space. These methods are (i) sequential pattern mining, (ii) clustering-based classification trees, (iii) hybrid learning, and (iv) flexible pattern mining. Each method uses a unique learning strategy to generate explicit knowledge in the form of patterns, decision rules and unsupervised rules. The methods are also capable of taking the decision maker’s preferences into account to generate knowledge unique to preferred regions of the objective space. Three realistic production systems involving different types of discrete variables are chosen as application studies. A multi-objective optimization problem is formulated for each system and solved using NSGA-II to generate the optimization datasets. Next, all four methods are applied to each dataset. In each application, the methods discover similar knowledge for specified regions of the objective space. Overall, the unsupervised rules generated by flexible pattern mining are found to be the most consistent, whereas the supervised rules from classification trees are the most sensitive to user-preferences.     

Survey of multi-objective optimization methods for engineering

A survey of current continuous nonlinear multi-objective optimization (MOO) concepts and methods is presented. It consolidates and relates seemingly different terminology and methods. The methods are divided into three major categories: methods with a priori articulation of preferences, methods with a posteriori articulation of preferences, and methods with no articulation of preferences. Genetic algorithms are surveyed as well. Commentary is provided on three fronts, concerning the advantages and pitfalls of individual methods, the different classes of methods, and the field of MOO as a whole. The Characteristics of the most significant methods are summarized. Conclusions are drawn that reflect often-neglected ideas and applicability to engineering problems. It is found that no single approach is superior. Rather, the selection of a specific method depends on the type of information that is provided in the problem, the users preferences, the solution requirements, and the availability of software.     

Data mining with various optimization methods

Road traffic represents the main source of noise in urban environments that is proven to significantly affect human mental and physical health and labour productivity. Thus, in order to control noise sound level in urban areas, it is very important to develop methods for modelling the road traffic noise. As observed in the literature, the models that deal with this issue are mainly based on regression analysis, while other approaches are very rare. In this paper a novel approach for modelling traffic noise that is based on optimization is presented. Four optimization techniques were used in simulation in this work: genetic algorithms, Hooke and Jeeves algorithm, simulated annealing and particle swarm optimization. Two different scenarios are presented in this paper. In the first scenario the optimization methods use the whole measurement dataset to find the most suitable parameters, whereas in the second scenario optimized parameters were found using only some of the measurement data, while the rest of the data was used to evaluate the predictive capabilities of the model. The goodness of the model is evaluated by the coefficient of determination and other statistical parameters, and results show agreement of high extent between measured data and calculated values in both scenarios. In addition, the model was compared with classical statistical model, and superior capabilities of proposed model were demonstrated. The simulations were done using the originally developed user friendly software package.     

Data mining and knowledge discovery in materials science and engineering: A polymer nanocomposites case study

In this study, data mining and knowledge discovery techniques were employed to validate their efficacy in acquiring information about the viscoelastic properties of vapor-grown carbon nanofiber (VGCNF)/vinyl ester (VE) nanocomposites solely from data derived from a designed experimental study. Formulation and processing factors (VGCNF type, use of a dispersing agent, mixing method, and VGCNF weight fraction) and testing temperature were utilized as inputs and the storage modulus, loss modulus, and tan delta were selected as outputs. The data mining and knowledge discovery algorithms and techniques included self-organizing maps (SOMs) and clustering techniques. SOMs demonstrated that temperature had the most significant effect on the output responses followed by VGCNF weight fraction. SOMs also showed how to prepare different VGCNF/VE nanocomposites with the same storage and loss modulus responses. A clustering technique, i.e., fuzzy C-means algorithm, was also applied to discover certain patterns in nanocomposite behavior after using principal component analysis as a dimensionality reduction technique. Particularly, these techniques were able to separate the nanocomposite specimens into different clusters based on temperature and tan delta features as well as to place the neat VE specimens (i.e., specimens containing no VGCNFs) in separate clusters. Most importantly, the results from data mining are consistent with previous response surface characterizations of this nanocomposite system. This work highlights the significance and utility of data mining and knowledge discovery techniques in the context of materials informatics.     

Injecting domain knowledge in multi-objective optimization problems: A semantic approach

In the field of complex problem optimization with metaheuristics, semantics has been used for modeling different aspects, such as: problem characterization, parameters, decision-maker’s preferences, or algorithms. However, there is a lack of approaches where ontologies are applied in a direct way into the optimization process, with the aim of enhancing it by allowing the systematic incorporation of additional domain knowledge. This is due to the high level of abstraction of ontologies, which makes them difficult to be mapped into the code implementing the problems and/or the specific operators of metaheuristics. In this paper, we present a strategy to inject domain knowledge (by reusing existing ontologies or creating a new one) into a problem implementation that will be optimized using a metaheuristic. Thus, this approach based on accepted ontologies enables building and exploiting complex computing systems in optimization problems. We describe a methodology to automatically induce user choices (taken from the ontology) into the problem implementations provided by the jMetal optimization framework. With the aim of illustrating our proposal, we focus on the urban domain. Concretely, we start from defining an ontology representing the domain semantics for a city (e.g., building, bridges, point of interest, routes, etc.) that allows defining a-priori preferences by a decision maker in a standard, reusable, and formal (logic-based) way. We validate our proposal with several instances of two use cases, consisting in bi-objective formulations of the Traveling Salesman Problem (TSP) and the Radio Network Design problem (RND), both in the context of an urban scenario. The results of the experiments conducted show how the semantic specification of domain constraints are effectively mapped into feasible solutions of the tackled TSP and RND scenarios. This proposal aims at representing a step forward towards the automatic modeling and adaptation of optimization problems guided by semantics, where the annotation of a human expert can be now considered during the optimization process.     

昂贵区间多目标优化空间数据挖掘求解策略

针对优化函数未知的昂贵区间多目标优化问题,提出一种基于主曲线建模的NSGA-II算法.该算法首先根据决策空间流形分布的种群数据构建K主曲线;然后利用所构建的K主曲线模型,通过插值和延展的方法生成子代.与遗传算法的随机生成子代策略相比,通过所提出方法生成有效子代效率会更高.由于目标空间拥挤距离无法求出,为此利用K主曲线找出待测解的前、后近距离解,按照决策空间拥挤距离对同序值解进行筛选,从而实现NSGA-II算法的改进.     

Ontology-based data mining model management for self-service knowledge discovery

Data mining (DM) models are knowledge-intensive information products that enable knowledge creation and discovery. As large volume of data is generated with high velocity from a variety of sources, there is a pressing need to place DM model selection and self-service knowledge discovery in the hands of the business users. However, existing knowledge discovery and data mining (KDDM) approaches do not sufficiently address key elements of data mining model management (DMMM) such as model sharing, selection and reuse. Furthermore, they are mainly from a knowledge engineer’s perspective, while the business requirements from business users are often lost. To bridge these semantic gaps, we propose an ontology-based DMMM approach for self-service model selection and knowledge discovery. We develop a DM³ ontology to translate the business requirements into model selection criteria and measurements, provide a detailed deployment architecture for its integration within an organization’s KDDM application, and use the example of a student loan company to demonstrate the utility of the DM^3.     

An information fusion approach to integrate image annotation and text mining methods for geographic knowledge discovery

Due to the steady increase in the number of heterogeneous types of location information on the internet, it is hard to organize a complete overview of the geospatial information for the tasks of knowledge acquisition related to specific geographic locations. The text- and photo-types of geographical dataset contain numerous location data, such as location-based tourism information, therefore defining high dimensional spaces of attributes that are highly correlated. In this work, we utilized text- and photo-types of location information with a novel approach of information fusion that exploits effective image annotation and location based text-mining approaches to enhance identification of geographic location and spatial cognition. In this paper, we describe our feature extraction methods to annotating images, and utilizing text mining approach to analyze images and texts simultaneously, in order to carry out geospatial text mining and image classification tasks. Subsequently, photo-images and textual documents are projected to a unified feature space, in order to generate a co-constructed semantic space for information fusion. Also, we employed text mining approaches to classify documents into various categories based upon their geospatial features, with the aims to discovering relationships between documents and geographical zones. The experimental results show that the proposed method can effectively enhance the tasks of location based knowledge discovery.     

A domain-specific decision support system for knowledge discovery using association and text mining

We propose a novel association and text mining system for knowledge discovery (ASTEK) from the warranty and service data in the automotive domain. The complex architecture of modern vehicles makes fault diagnosis and isolation a non-trivial task. The association mining isolates anomaly cases from the millions of service and claims records. ASTEK has shown 86% accuracy in correctly identifying the anomaly cases. The text mining subscribes to the diagnosis and prognosis (D&P) ontology, which provides the necessary domain-specific knowledge. The root causes associated with the anomaly cases are identified by discovering frequent symptoms associated with the part failures along with the repair actions used to fix the part failures. The best-practice knowledge is disseminated to the dealers involved in the anomaly cases. ASTEK has been implemented as a prototype in the service and quality department of GM and its performance has been validated in the real life set up. On an average, the analysis time is reduced from few weeks to few minutes, which in real life industry are significant improvements.     

Dimensionality optimization by heuristic greedy learning vs. genetic algorithms in knowledge discovery and data mining

Dimensionality optimization involves optimizing the size of data sets from both dimensions, variable and observation selections. The ultimate objective of dimensionality optimization is to obtain the induced data space, by reducing both dimensionalities in such a way that the reduced subset could retain sufficient information. In most real-world applications, it is not known what the best subset is and what should be contained in such a subset. Selecting the appropriate subset is extremely important in effectively mining over large data sets in the sense that it is the only source for any data mining and knowledge discovery algorithm to work with the data of interest reliably.The statistical as well as artificial intelligence community has provided good methods in this domain, but still a lot of improvements need to be made, especially for data mining applications. This paper introduces a heuristic methodology that integrates heuristic greedy search methods and tree-structured SampleC4.5 to efficiently find the optimal subset of very large data sets from both dimensions simultaneously. A GA-based optimization approach is also proposed in the paper. Experimental results are presented which illustrate the effectiveness of our approaches in digging out the important underlying patterns, and indicate the potential advantages of the proposed techniques to improve the optimizing process while staying out of misleading dilemma. The results of our experiments also show the robustness of our approaches and complementary characteristics for knowledge discovery and data mining tasks.     

A survey of temporal knowledge discovery paradigms and methods

With the increase in the size of data sets, data mining has recently become an important research topic and is receiving substantial interest from both academia and industry. At the same time, interest in temporal databases has been increasing and a growing number of both prototype and implemented systems are using an enhanced temporal understanding to explain aspects of behavior associated with the implicit time-varying nature of the universe. This paper investigates the confluence of these two areas, surveys the work to date, and explores the issues involved and the outstanding problems in temporal data mining.     

jMetal: A Java framework for multi-objective optimization

This paper describes jMetal, an object-oriented Java-based framework aimed at the development, experimentation, and study of metaheuristics for solving multi-objective optimization problems. jMetal includes a number of classic and modern state-of-the-art optimizers, a wide set of benchmark problems, and a set of well-known quality indicators to assess the performance of the algorithms. The framework also provides support to carry out full experimental studies, which can be configured and executed by using jMetal’s graphical interface. Other features include the automatic generation of statistical information of the obtained results, and taking advantage of the current availability of multi-core processors to speed-up the running time of the experiments. In this work, we include two case studies to illustrate the use of jMetal in both solving a problem with a metaheuristic and designing and performing an experimental study.     

知识发现中的因果关联规则挖掘研究

因果关联规则是知识库中一类重要的知识类型,具有重要的应用价值。首先对因果关系的特殊性质进行了分析,然后基于语言场和广义归纳逻辑因果模型,从表示、挖掘、评价和应用几方面,对因果关联规则的研究进行了详细论述。并在此基础上提出了隐含因果关联规则的概念。通过语言场和推理机制的运用,使因果关联规则这一重要知识形式的挖掘和评价过程具有良好的逻辑性和扩张性。     

Perception-based hidden Markov models: a theoretical framework for data mining and knowledge discovery

 The combination of objective measurements and human perceptions using hidden Markov models with particular reference to sequential data mining and knowledge discovery is presented in this paper. Both human preferences and statistical analysis are utilized for verification and identification of hypotheses as well as detection of hidden patterns. As another theoretical view, this work attempts to formalize the complementarity of the computational theories of hidden Markov models and perceptions for providing solutions associated with the manipulation of the internet.     

轻烃回收装置数据挖掘及生产优化

以控制制冷量提高液烃收率为目标,将膨胀机进口温度、出口温度及膨胀机进出口温差作为响应值,膨胀机转速、喷嘴压力、膨胀比、膨胀机进出口压差、原料气流量、原料气压力等8个工艺参数则构成影响响应值的主要变量。通过对某轻烃回收装置进行生产数据挖掘,提出了优化该装置制冷系统的生产指导方案。生产试验表明,按该方案生产,可提高液烃收率5％以上。     

A synergy of multi-objective optimization and data mining for the analysis of a flexible flow shop

A method for analyzing production systems by applying multi-objective optimization and data mining techniques on discrete-event simulation models, the so-called Simulation-based Innovization (SBI) is presented in this paper. The aim of the SBI analysis is to reveal insight on the parameters that affect the performance measures as well as to gain deeper understanding of the problem, through post-optimality analysis of the solutions acquired from multi-objective optimization. This paper provides empirical results from an industrial case study, carried out on an automotive machining line, in order to explain the SBI procedure. The SBI method has been found to be particularly suitable in this case study as the three objectives under study, namely total tardiness, makespan and average work-in-process, are in conflict with each other. Depending on the system load of the line, different decision variables have been found to be influencing. How the SBI method is used to find important patterns in the explored solution set and how it can be valuable to support decision making in order to improve the scheduling under different system loadings in the machining line are addressed.     

基于数据融合的知识发现方法在网络管理中的应用

提出用于网络管理的基于数据融合的知识发现系统框架，研究数据融合技术在知识发现的数据准备和预处理阶段的应用，研究关联规则在表达网络管理知识方面的适用性并针对网络管理数据时序性的特点，引入情景规则来表示期望发掘的知识，指出网络故障管理中关联规则和情景规则的挖掘算法以及知识增量式更新的算法，并简介了原型系统的实现方法。