ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems

This paper concerns multiobjective optimization in scenarios where each solution evaluation is financially and/or temporally expensive. We make use of nine relatively low-dimensional, nonpathological, real-valued functions, such as arise in many applications, and assess the performance of two algorithms after just 100 and 250 (or 260) function evaluations. The results show that NSGA-II, a popular multiobjective evolutionary algorithm, performs well compared with random search, even within the restricted number of evaluations used. A significantly better performance (particularly, in the worst case) is, however, achieved on our test set by an algorithm proposed herein-ParEGO-which is an extension of the single-objective efficient global optimization (EGO) algorithm of Jones et al. ParEGO uses a design-of-experiments inspired initialization procedure and learns a Gaussian processes model of the search landscape, which is updated after every function evaluation. Overall, ParEGO exhibits a promising performance for multiobjective optimization problems where evaluations are expensive or otherwise restricted in number.     

异构集成代理辅助多目标粒子群优化算法

针对代理辅助进化算法在减少昂贵适应度评估时难以通过少量样本点构造高质量代理模型的问题,提出异构集成代理辅助多目标粒子群优化算法。该方法通过使用加权平均法将Kriging模型和径向基函数网络模型组合成高精度的异构集成模型,达到增强算法处理不确定性信息能力的目的。基于集成学习的两种代理模型分别应用于全局搜索和局部搜索,在多目标粒子群优化算法框架基础上,新提出的方法为每个目标函数自适应地构造了异构集成模型,利用其模型的非支配解来指导粒子群的更新,得出目标函数的最优解集。实验结果表明,所提方法提高了代理模型的搜索能力,减少了评估次数,并且随着搜索维度的增加,其计算复杂性也具有更好的可扩展性。     

Multiobjective optimization and hybrid evolutionary algorithm to solve constrained optimization problems.

This paper presents a novel evolutionary algorithm (EA) for constrained optimization problems, i.e., the hybrid constrained optimization EA (HCOEA). This algorithm effectively combines multiobjective optimization with global and local search models. In performing the global search, a niching genetic algorithm based on tournament selection is proposed. Also, HCOEA has adopted a parallel local search operator that implements a clustering partition of the population and multiparent crossover to generate the offspring population. Then, nondominated individuals in the offspring population are used to replace the dominated individuals in the parent population. Meanwhile, the best infeasible individual replacement scheme is devised for the purpose of rapidly guiding the population toward the feasible region of the search space. During the evolutionary process, the global search model effectively promotes high population diversity, and the local search model remarkably accelerates the convergence speed. HCOEA is tested on 13 well-known benchmark functions, and the experimental results suggest that it is more robust and efficient than other state-of-the-art algorithms from the literature in terms of the selected performance metrics, such as the best, median, mean, and worst objective function values and the standard deviations.     

Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions

This paper presents a new algorithm for derivative-free optimization of expensive black-box objective functions subject to expensive black-box inequality constraints. The proposed algorithm, called ConstrLMSRBF, uses radial basis function (RBF) surrogate models and is an extension of the Local Metric Stochastic RBF (LMSRBF) algorithm by Regis and Shoemaker (2007a) [1] that can handle black-box inequality constraints. Previous algorithms for the optimization of expensive functions using surrogate models have mostly dealt with bound constrained problems where only the objective function is expensive, and so, the surrogate models are used to approximate the objective function only. In contrast, ConstrLMSRBF builds RBF surrogate models for the objective function and also for all the constraint functions in each iteration, and uses these RBF models to guide the selection of the next point where the objective and constraint functions will be evaluated. Computational results indicate that ConstrLMSRBF is better than alternative methods on 9 out of 14 test problems and on the MOPTA08 problem from the automotive industry (Jones, 2008 [2]). The MOPTA08 problem has 124 decision variables and 68 inequality constraints and is considered a large-scale problem in the area of expensive black-box optimization. The alternative methods include a Mesh Adaptive Direct Search (MADS) algorithm (Abramson and Audet, 2006 [3]; Audet and Dennis, 2006 [4]) that uses a kriging-based surrogate model, the Multistart LMSRBF algorithm by Regis and Shoemaker (2007a) [1] modified to handle black-box constraints via a penalty approach, a genetic algorithm, a pattern search algorithm, a sequential quadratic programming algorithm, and COBYLA (Powell, 1994 [5]), which is a derivative-free trust-region algorithm. Based on the results of this study, the results in Jones (2008) [2] and other approaches presented at the ISMP 2009 conference, ConstrLMSRBF appears to be among the best, if not the best, known algorithm for the MOPTA08 problem in the sense of providing the most improvement from an initial feasible solution within a very limited number of objective and constraint function evaluations.     

A Bi-population Cooperative Optimization Algorithm Assisted by an Autoencoder for Medium-scale Expensive Problems

This study presents an autoencoder-embedded optimization (AEO) algorithm which involves a bi-population cooperative strategy for medium-scale expensive problems (MEPs). A huge search space can be compressed to an informative low-dimensional space by using an autoencoder as a dimension reduction tool. The search operation conducted in this low space facilitates the population with fast convergence towards the optima. To strike the balance between exploration and exploitation during optimization, two phases of a tailored teaching-learning-based optimization (TTLBO) are adopted to coevolve solutions in a distributed fashion, wherein one is assisted by an autoencoder and the other undergoes a regular evolutionary process. Also, a dynamic size adjustment scheme according to problem dimension and evolutionary progress is proposed to promote information exchange between these two phases and accelerate evolutionary convergence speed. The proposed algorithm is validated by testing benchmark functions with dimensions varying from 50 to 200. As indicated in our experiments, TTLBO is suitable for dealing with medium-scale problems and thus incorporated into the AEO framework as a base optimizer. Compared with the state-of-the-art algorithms for MEPs, AEO shows extraordinarily high efficiency for these challenging problems, thus opening new directions for various evolutionary algorithms under AEO to tackle MEPs and greatly advancing the field of medium-scale computationally expensive optimization.      

Improving convergence of evolutionary multi-objective optimization with local search: a concurrent-hybrid algorithm

A local search method is often introduced in an evolutionary optimization algorithm, to enhance its speed and accuracy of convergence to optimal solutions. In multi-objective optimization problems, the implementation of local search is a non-trivial task, as determining a goal for local search in presence of multiple conflicting objectives becomes a difficult task. In this paper, we borrow a multiple criteria decision making concept of employing a reference point based approach of minimizing an achievement scalarizing function and integrate it as a search operator with a concurrent approach in an evolutionary multi-objective algorithm. Simulation results of the new concurrent-hybrid algorithm on several two to four-objective problems compared to a serial approach, clearly show the importance of local search in aiding a computationally faster and accurate convergence to the Pareto optimal front.     

Memetic algorithm using multi-surrogates for computationally expensive optimization problems

In this paper, we present a multi-surrogates assisted memetic algorithm for solving optimization problems with computationally expensive fitness functions. The essential backbone of our framework is an evolutionary algorithm coupled with a local search solver that employs multi-surrogate in the spirit of Lamarckian learning. Inspired by the notion of ‘blessing and curse of uncertainty’ in approximation models, we combine regression and exact interpolating surrogate models in the evolutionary search. Empirical results are presented for a series of commonly used benchmark problems to demonstrate that the proposed framework converges to good solution quality more efficiently than the standard genetic algorithm, memetic algorithm and surrogate-assisted memetic algorithms.     

Solution of Jiles-Atherton vector hysteresis parameters estimation by modified Differential Evolution approaches

Differential Evolution (DE) is a simple and efficient stochastic global optimization algorithm of evolutionary computation field, which involves the evolution of a population of solutions using operators such as mutation, crossover, and selection. The basic idea of DE is to adapt the search during the evolutionary process. At the start of the evolution, the perturbations are large since parent populations are far away from each other. As the evolutionary process matures, the population converges to a small region and the perturbations adaptively become small. DE approaches have been successfully applied to solve a wide range of optimization problems. In this paper, the parameters set of the Jiles-Atherton vector hysteresis model is obtained with an approach based on modified Differential Evolution (MDE) approaches using generation-varying control parameters based on generation of random numbers with uniform distribution. Several evaluated MDE approaches perform better than the classical DE methods and a genetic algorithm approach in terms of the quality and stability of the final solutions in optimization of vector Jiles-Atherton vector hysteresis model from a workbench containing a rotational single sheet tester.     

Managing approximation models in multiobjective optimization

In engineering problems, computationally intensive high-fidelity models or expensive computer simulations hinder the use of standard optimization techniques because they should be invoked repeatedly during optimization, despite the tremendous growth of computer capability. Therefore, these expensive analyses are often replaced with approximation models that can be evaluated at considerably less effort. However, due to their limited accuracy, it is practically impossible to exactly find an actual optimum (or a set of actual noninferior solutions) of the original single (or multi) objective optimization problem. Significant efforts have been made to overcome this limitation. The model management framework is one of such endeavours. The approximation models are sequentially updated during the iterative optimization process in such a way that their capability to accurately model the original functions especially in the region of our interests can be improved. The models are modified and improved by using one or several sample points generated by making a good use of the predictive ability of the approximation models. However, these approaches have been restricted to a single objective optimization problem. It seems that there is no reported management framework that can handle a multi-objective optimization problem. This paper will suggest strategies that can successfully treat not only a single objective but also multiple objectives by extending the concept of sequentially managing approximation models and combining this extended concept with a genetic algorithm which can treat multiple objectives (MOGA). Consequently, the number of exact analyses required to converge to an actual optimum or to generate a sufficiently accurate Pareto set can be reduced considerably. Especially, the approach for multiple objectives will lead to a surprising reduction in number. We will confirm these effects through several illustrative examples.     

Discovering driving strategies with a multiobjective optimization algorithm

When driving a vehicle along a given route, several objectives such as the traveling time and the fuel consumption have to be considered. This can be viewed as an optimization problem and solved with the appropriate optimization algorithms. The existing optimization algorithms mostly combine objectives into a weighted-sum cost function and solve the corresponding single-objective problem. Using a multiobjective approach should be, in principle, advantageous, since it enables better exploration of the multiobjective search space, however, no results about the optimization of driving with this approach have been reported yet. To test the multiobjective approach, we designed a two-level Multiobjective Optimization algorithm for discovering Driving Strategies (MODS). It finds a set of nondominated driving strategies with respect to two conflicting objectives: the traveling time and the fuel consumption. The lower-level algorithm is based on a deterministic breadth-first search and nondominated sorting, and searches for nondominated driving strategies. The upper-level algorithm is an evolutionary algorithm that optimizes the input parameters for the lower-level algorithm. The MODS algorithm was tested on data from real-world routes and compared with the existing single-objective algorithms for discovering driving strategies. The results show that the presented algorithm, on average, significantly outperforms the existing algorithms.     

Finite element based hybrid evolutionary optimization approach to solving rigid pavement inversion problem

This paper focuses on the development of a new backcalculation method for concrete road structures based on a hybrid evolutionary global optimization algorithm, namely shuffled complex evolution (SCE). Evolutionary optimization algorithms are ideally suited for intrinsically multi-modal, non-convex, and discontinuous real-world problems such as pavement backcalculation because of their ability to explore very large and complex search spaces and locate the globally optimal solution using a parallel search mechanism as opposed to a point-by-point search mechanism employed by traditional optimization algorithms. SCE, a type of evolutionary optimization algorithms based on the tradeoff of exploration and exploitation, has proved to be an efficient method for many global optimization problems and in some cases it does not suffer the difficulties encountered by other evolutionary computation techniques. The SCE optimization approach is hybridized with a neural networks surrogate finite-element based forward pavement response model to enable rapid computation of global or near-global pavement layer moduli solutions. The proposed rigid pavement backcalculation model is evaluated using field non-destructive test data acquired from a full-scale airport pavement test facility.     

Multi-modal function optimization with the ZEDS (zoomed evolutionary dual strategy) algorithmzoomed evolutionary algorithm

This article introduces a new evolutionary algorithm for multi-modal function optimization called ZEDS (zoomed evolutionary dual strategy). ZEDS employs a two-step, zoomed (global to local), evolutionary approach. In the first (global) step, an improved ‘GT algorithm’ is employed to perform a global recombinatory search that divides the search space into niches according to the positions of its approximate solutions. In the second (local) step, a ‘niche evolutionary strategy’ performs a local search in the niches obtained from the first step, which is repeated until acceptable solutions are found. The ZEDS algorithm was applied to some challenging problems with good results, as shown in this article.     

Supervisory nonlinear MPC for emergency voltage control using pattern search

The design of a higher-layer controller using model predictive control (MPC) is considered. The higher-layer controller uses MPC to determine set-points for controllers in a lower control layer. In this paper the use of an object-oriented model of the system for making predictions is proposed. When employing such an object-oriented prediction model the MPC problem is a nonlinear, non-smooth optimization problem, with an objective function that is expensive to evaluate. Multi-start pattern search is proposed as approach to solving this problem, since it deals effectively with the local minima and the non-smoothness of the problem, and does not require expensive estimation of derivatives. Experiments in an emergency voltage control problem on a 9-bus dynamic power network show the superior performance of the proposed multi-start pattern-search approach when compared to a gradient-based approach.     

基于多点加点准则的代理模型辅助社会学习微粒群算法

代理模型辅助的进化算法目前已广泛用于解决计算代价高的复杂优化问题.然而,大多数现有的代理辅助进化算法只适用于低维问题且仍然需要数千次昂贵的真实适应值评价来获得较优解.为此,提出一种基于多点加点准则的代理模型辅助的社会学习微粒群算法,用于解决高维问题并使用更少的评价次数.该算法选用高斯过程构造代理模型,以社会学习微粒群算法(SLPSO)作为优化器,提出一种基于相似度的多点加点规则(SMIC),用于选取需要使用原函数进行实际计算的候选解.在仿真实验中将该方法与现有研究成果进行比较,通过对50维sim100维的基准函数的测试,验证了所提出算法在有限的适应值计算次数下拥有更好的寻优性能,尤其是在高维优化问题上拥有更显著的优势.     

A graph-based framework for model-driven optimization facilitating impact analysis of mutation operator properties

Optimization problems in software engineering typically deal with structures as they occur in the design and maintenance of software systems. In model-driven optimization (MDO), domain-specific models are used to represent these structures while evolutionary algorithms are often used to solve optimization problems. However, designing appropriate models and evolutionary algorithms to represent and evolve structures is not always straightforward. Domain experts often need deep knowledge of how to configure an evolutionary algorithm. This makes the use of model-driven meta-heuristic search difficult and expensive. We present a graph-based framework for MDO that identifies and clarifies core concepts and relies on mutation operators to specify evolutionary change. This framework is intended to help domain experts develop and study evolutionary algorithms based on domain-specific models and operators. In addition, it can help in clarifying the critical factors for conducting reproducible experiments in MDO. Based on the framework, we are able to take a first step toward identifying and studying important properties of evolutionary operators in the context of MDO. As a showcase, we investigate the impact of soundness and completeness at the level of mutation operator sets on the effectiveness and efficiency of evolutionary algorithms.

     

Hybridation of Bayesian networks and evolutionary algorithms for multi-objective optimization in an integrated product design and project management context

A better integration of preliminary product design and project management processes at early steps of system design is nowadays a key industrial issue. Therefore, the aim is to make firms evolve from classical sequential approach (first product design the project design and management) to new integrated approaches. In this paper, a model for integrated product/project optimization is first proposed which allows taking into account simultaneously decisions coming from the product and project managers. However, the resulting model has an important underlying complexity, and a multi-objective optimization technique is required to provide managers with appropriate scenarios in a reasonable amount of time. The proposed approach is based on an original evolutionary algorithm called evolutionary algorithm oriented by knowledge (EAOK). This algorithm is based on the interaction between an adapted evolutionary algorithm and a model of knowledge (MoK) used for giving relevant orientations during the search process. The evolutionary operators of the EA are modified in order to take into account these orientations. The MoK is based on the Bayesian Network formalism and is built both from expert knowledge and from individuals generated by the EA. A learning process permits to update probabilities of the BN from a set of selected individuals. At each cycle of the EA, probabilities contained into the MoK are used to give some bias to the new evolutionary operators. This method ensures both a faster and effective optimization, but it also provides the decision maker with a graphic and interactive model of knowledge linked to the studied project. An experimental platform has been developed to experiment the algorithm and a large campaign of tests permits to compare different strategies as well as the benefits of this novel approach in comparison with a classical EA.     

Combinations of estimation of distribution algorithms and other techniques

This paper summaries our recent work on combining estimation of distribution algorithms (EDA) and other techniques for solving hard search and optimization problems:a) guided mutation,an offspring generator in which the ideas from EDAs and genetic algorithms are combined together,we have shown that an evolutionary algorithm with guided mutation outperforms the best GA for the maximum clique problem,b)evolutionary algorithms refining a heuristic,we advocate a strategy for solving a hard optimization problem with complicated data structure,and c) combination of two different local search techniques and EDA for numerical global optimization problems,its basic idea is that not all the new generated points are needed to be improved by an expensive local search.     

A swarm optimization algorithm inspired in the behavior of the social-spider

Swarm intelligence is a research field that models the collective behavior in swarms of insects or animals. Several algorithms arising from such models have been proposed to solve a wide range of complex optimization problems. In this paper, a novel swarm algorithm called the Social Spider Optimization (SSO) is proposed for solving optimization tasks. The SSO algorithm is based on the simulation of cooperative behavior of social-spiders. In the proposed algorithm, individuals emulate a group of spiders which interact to each other based on the biological laws of the cooperative colony. The algorithm considers two different search agents (spiders): males and females. Depending on gender, each individual is conducted by a set of different evolutionary operators which mimic different cooperative behaviors that are typically found in the colony. In order to illustrate the proficiency and robustness of the proposed approach, it is compared to other well-known evolutionary methods. The comparison examines several standard benchmark functions that are commonly considered within the literature of evolutionary algorithms. The outcome shows a high performance of the proposed method for searching a global optimum with several benchmark functions.     

An Adaptive Tradeoff Model for Constrained Evolutionary Optimization

In this paper, an adaptive tradeoff model (ATM) is proposed for constrained evolutionary optimization. In this model, three main issues are considered: (1) the evaluation of infeasible solutions when the population contains only infeasible individuals; (2) balancing feasible and infeasible solutions when the population consists of a combination of feasible and infeasible individuals; and (3) the selection of feasible solutions when the population is composed of feasible individuals only. These issues are addressed in this paper by designing different tradeoff schemes during different stages of a search process to obtain an appropriate tradeoff between objective function and constraint violations. In addition, a simple evolutionary strategy (ES) is used as the search engine. By integrating ATM with ES, a generic constrained optimization evolutionary algorithm (ATMES) is derived. The new method is tested on 13 well-known benchmark test functions, and the empirical results suggest that it outperforms or performs similarly to other state-of-the-art techniques referred to in this paper in terms of the quality of the resulting solutions.     

A surrogate-assisted evolutionary algorithm based on the genetic diversity objective

In this work, a novel surrogate-assisted memetic algorithm is proposed which is based on the preservation of genetic diversity within the population. The aim of the algorithm is to solve multi-objective optimization problems featuring computationally expensive fitness functions in an efficient manner. The main novelty is the use of an evolutionary algorithm as global searcher that treats the genetic diversity as an objective during the evolution and uses it, together with a non-dominated sorting approach, to assign the ranks. This algorithm, coupled with a gradient-based algorithm as local searcher and a back-propagation neural network as global surrogate model, demonstrates to provide a reliable and effective balance between exploration and exploitation. A detailed performance analysis has been conducted on five commonly used multi-objective problems, each one involving distinct features that can make the convergence difficult toward the Pareto-optimal front. In most cases, the proposed algorithm outperformed the other state-of-the-art evolutionary algorithms considered in the comparison, assuring higher repeatability on the final non-dominated set, deeper convergence level and higher convergence rate. It also demonstrates a clear ability to widely cover the Pareto-optimal front with larger percentage of non-dominated solutions if compared to the total number of function evaluations.