Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions

The integration of optimization methodologies with computational analyses/simulations has a profound impact on the product design. Such integration, however, faces multiple challenges. The most eminent challenges arise from high-dimensionality of problems, computationally-expensive analysis/simulation, and unknown function properties (i.e., black-box functions). The merger of these three challenges severely aggravates the difficulty and becomes a major hurdle for design optimization. This paper provides a survey on related modeling and optimization strategies that may help to solve High-dimensional, Expensive (computationally), Black-box (HEB) problems. The survey screens out 207 references including multiple historical reviews on relevant subjects from more than 1,000 papers in a variety of disciplines. This survey has been performed in three areas: strategies tackling high-dimensionality of problems, model approximation techniques, and direct optimization strategies for computationally-expensive black-box functions and promising ideas behind non-gradient optimization algorithms. Major contributions in each area are discussed and presented in an organized manner. The survey exposes that direct modeling and optimization strategies to address HEB problems are scarce and sporadic, partially due to the difficulty of the problem itself. Moreover, it is revealed that current modeling research tends to focus on sampling and modeling techniques themselves and neglect studying and taking the advantages of characteristics of the underlying expensive functions. Based on the survey results, two promising approaches are identified to solve HEB problems. Directions for future research are also discussed.     

A Survey of Evolutionary Algorithms for Multi-Objective Optimization Problems With Irregular Pareto Fronts

Evolutionary algorithms have been shown to be very successful in solving multi-objective optimization problems(MOPs).However,their performance often deteriorates when solving MOPs with irregular Pareto fronts.To remedy this issue,a large body of research has been performed in recent years and many new algorithms have been proposed.This paper provides a comprehensive survey of the research on MOPs with irregular Pareto fronts.We start with a brief introduction to the basic concepts,followed by a summary of the benchmark test problems with irregular problems,an analysis of the causes of the irregularity,and real-world optimization problems with irregular Pareto fronts.Then,a taxonomy of the existing methodologies for handling irregular problems is given and representative algorithms are reviewed with a discussion of their strengths and weaknesses.Finally,open challenges are pointed out and a few promising future directions are suggested.     

Multiobjective evolutionary algorithm based on multimethod with dynamic resources allocation

In the last two decades, multiobjective optimization has become main stream and various multiobjective evolutionary algorithms (MOEAs) have been suggested in the field of evolutionary computing (EC) for solving hard combinatorial and continuous multiobjective optimization problems. Most MOEAs employ single evolutionary operators such as crossover, mutation and selection for population evolution. In this paper, we suggest a multiobjective evolutionary algorithm based on multimethods (MMTD) with dynamic resource allocation for coping with continuous multi-objective optimization problems (MOPs). The suggested algorithm employs two well known population based stochastic algorithms namely MOEA/D and NSGA-II as constituent algorithms for population evolution with a dynamic resource allocation scheme. We have examined the performance of the proposed MMTD on two different MOPs test suites: the widely used ZDT problems and the recently formulated test instances for the special session on MOEAs competition of the 2009 IEEE congress on evolutionary computation (CEC’09). Experimental results obtained by the suggested MMTD are more promising than those of some state-of-the-art MOEAs in terms of the inverted generational distance (IGD)-metric on most test problems.     

面向复杂约束优化问题的进化算法综述

约束优化是多数实际工程应用优化问题的呈现方式.进化算法由于其高效的表现,近年来被广泛应用于约束优化问题求解.但约束条件使得问题解空间离散、缩小、改变,给进化算法求解约束优化问题带来极大挑战.在此背景下,融合约束处理技术的进化算法成为研究热点.此外,随着研究的深入,近年来约束处理技术在复杂工程应用问题优化中得到了广泛发展,例如多目标、高维、等式优化等.根据复杂性的缘由,将面向复杂约束优化问题的进化优化分为面向复杂目标的进化约束优化算法和面向复杂约束场景的进化算法两种类别进行综述,其中,重点探讨了实际工程应用的复杂性对约束处理技术的挑战和目前研究的最新进展,并最后总结了未来的研究趋势与挑战.     

High-Level Parallel Ant Colony Optimization with Algorithmic Skeletons

Parallel implementations of swarm intelligence algorithms such as the ant colony optimization (ACO) have been widely used to shorten the execution time when solving complex optimization problems. When aiming for a GPU environment, developing efficient parallel versions of such algorithms using CUDA can be a difficult and error-prone task even for experienced programmers. To overcome this issue, the parallel programming model of Algorithmic Skeletons simplifies parallel programs by abstracting from low-level features. This is realized by defining common programming patterns (e.g. map, fold and zip) that later on will be converted to efficient parallel code. In this paper, we show how algorithmic skeletons formulated in the domain specific language Musket can cope with the development of a parallel implementation of ACO and how that compares to a low-level implementation. Our experimental results show that Musket suits the development of ACO. Besides making it easier for the programmer to deal with the parallelization aspects, Musket generates high performance code with similar execution times when compared to low-level implementations.

     

DisABC: A new artificial bee colony algorithm for binary optimization

Artificial bee colony (ABC) algorithm is one of the recently proposed swarm intelligence based algorithms for continuous optimization. Therefore it is not possible to use the original ABC algorithm directly to optimize binary structured problems. In this paper we introduce a new version of ABC, called DisABC, which is particularly designed for binary optimization. DisABC uses a new differential expression, which employs a measure of dissimilarity between binary vectors in place of the vector subtraction operator typically used in the original ABC algorithm. Such an expression helps to maintain the major characteristics of the original one and is respondent to the structure of binary optimization problems, too. Similar to original ABC algorithm, DisABC's differential expression works in continuous space while its consequence is used in a two-phase heuristic to construct a complete solution in binary space. Effectiveness of DisABC algorithm is tested on solving the uncapacitated facility location problem (UFLP). A set of 15 benchmark test problem instances of UFLP are adopted from OR-Library and solved by the proposed algorithm. Results are compared with two other state of the art binary optimization algorithms, i.e., binDE and PSO algorithms, in terms of three quality indices. Comparisons indicate that DisABC performs very well and can be regarded as a promising method for solving wide class of binary optimization problems.     

Set-based discrete particle swarm optimization and its applications: a survey

Particle swarm optimization (PSO) is one of the most popular population-based stochastic algorithms for solving complex optimization problems. While PSO is simple and effective, it is originally defined in continuous space. In order to take advantage of PSO to solve combinatorial optimization problems in discrete space, the set-based PSO (S-PSO) framework extends PSO for discrete optimization by redefining the operations in PSO utilizing the set operations. Since its proposal, S-PSO has attracted increasing research attention and has become a promising approach for discrete optimization problems. In this paper, we intend to provide a comprehensive survey on the concepts, development and applications of S-PSO. First, the classification of discrete PSO algorithms is presented. Then the S-PSO framework is given. In particular, we will give an insight into the solution construction strategies, constraint handling strategies, and alternative reinforcement strategies in S-PSO together with its different variants. Furthermore, the extensions and applications of S-PSO are also discussed systemically. Some potential directions for the research of S-PSO are also discussed in this paper.     

Corona virus optimization (CVO): a novel optimization algorithm inspired from the Corona virus pandemic

This research introduces a new probabilistic and meta-heuristic optimization approach inspired by the Corona virus pandemic. Corona is an infection that originates from an unknown animal virus, which is of three known types and COVID-19 has been rapidly spreading since late 2019. Based on the SIR model, the virus can easily transmit from one person to several, causing an epidemic over time. Considering the characteristics and behavior of this virus, the current paper presents an optimization algorithm called Corona virus optimization (CVO) which is feasible, effective, and applicable. A set of benchmark functions evaluates the performance of this algorithm for discrete and continuous problems by comparing the results with those of other well-known optimization algorithms. The CVO algorithm aims to find suitable solutions to application problems by solving several continuous mathematical functions as well as three continuous and discrete applications. Experimental results denote that the proposed optimization method has a credible, reasonable, and acceptable performance.

     

A survey of symbiotic organisms search algorithms and applications

Nature-inspired algorithms take inspiration from living things and imitate their behaviours to accomplish robust systems in engineering and computer science discipline. Symbiotic organisms search (SOS) algorithm is a recent metaheuristic algorithm inspired by symbiotic interaction between organisms in an ecosystem. Organisms develop symbiotic relationships such as mutualism, commensalism, and parasitism for their survival in ecosystem. SOS was introduced to solve continuous benchmark and engineering problems. The SOS has been shown to be robust and has faster convergence speed when compared with genetic algorithm, particle swarm optimization, differential evolution, and artificial bee colony which are the traditional metaheuristic algorithms. The interests of researchers in using SOS for handling optimization problems are increasing day by day, due to its successful application in solving optimization problems in science and engineering fields. Therefore, this paper presents a comprehensive survey of SOS advances and its applications, and this will be of benefit to the researchers engaged in the study of SOS algorithm.

     

Overbreak prediction and optimization in tunnel using neural network and bee colony techniques

Overbreak is an undesirable phenomenon in blasting operations. The causing factors of overbreak can be generally divided as blasting and geological parameters. Due to multiplicity of effective parameters and complexity of interactions among these parameters, empirical methods may not be fully appropriated for blasting pattern design. In this research, artificial neural network (ANN) as a powerful tool for solving such complicated problems is developed to predict overbreak induced by blasting operations in the Gardaneh Rokh tunnel, Iran. To develop an ANN model, an established database comprising of 255 datasets has been utilized. A three-layer ANN was found as an optimum model for prediction of overbreak. The coefficient of determination (R²) and root mean square error (RMSE) values of the selected model were obtained as 0.921, 0.4820, 0.923 and 0.4277 for training and testing, respectively, which demonstrate a high capability of ANN in predicting overbreak. After selecting the best model, the selected model was used for optimization purpose using artificial bee colony (ABC) algorithm as one of the most powerful optimization algorithms. Considering this point that overbreak is one of the main problems in tunneling, reducing its amount causes to have a good tunneling operation. After making several models of optimization and variations in its weights, the optimum amount for the extra drilling was 1.63 m², which is 47% lower than the lowest value (3.055 m²). It can be concluded that ABC algorithm can be introduced as a new optimizing algorithm to minimize overbreak induced by tunneling.

     

Robustness,generality and efficiency of optimization algorithms for practical applications

The design of most engineering systems is a complex and time-consuming process. In addition, the need to optimize such systems where multidisciplinary analysis and design procedures are required can cost additional human and computational resources if proper software and numerical algorithms are not used. Several computational aspects of optimization algorithms and the associated software must be considered while making comparative studies and selecting a suitable algorithm for practical applications. Several parameters, such asaccuracy, generality, robustness, efficiency and ease of use, must be considered while deciding the superiority of an optimization approach. Approximate algorithms without sound mathematical basis can be sometimes more efficient for a specific problem, but fail to satisfy other requirements. They are, therefore, not suitable for general applications. An objective of the paper is to emphasize the critical importance of the above-mentioned parameters in large scalestructural optimization and other applications. Theoretical foundations of two promising approaches, thesequential quadratic programming (SQP) andoptimality criteria (OC), are presented and analysed. Recent numerical experiments and experiences with the SQP algorithm satisfying these requirements are described by solving a variety of structural design problems. An important conclusion of the paper is that the SQP method with a potential constraint strategy is a better choice as compared to the currently prevalent mathematical programming (MP) and OC approaches.     

Evolution of neural networks for classification and regression

Although Artificial Neural Networks (ANNs) are importantdata mining techniques, the search for the optimal ANN is a challenging task: the ANN should learn the input–output mapping without overfitting the data and training algorithms may get trapped in local minima. The use of Evolutionary Computation (EC) is a promising alternative for ANN optimization. This work presents two hybrid EC/ANN algorithms: the first evolves neural topologies while the latter performs simultaneous optimization of architectures and weights. Sixteen real-world tasks were used to test these strategies. Competitive results were achieved when compared with a heuristic model selection and other Data Mining algorithms.     

Multi-objective modified differential evolution algorithm with archive-base mutation for solving multi-objective $$$$-xylene oxidation rocess

Maximizing the diversity of the obtained objective vectors and increasing the convergence speed to the true Pareto front are two important issues in the design of multi-objective evolutionary algorithms (MOEAs). To solve complex multi-objective optimization problems (MOPs), a multi-objective modified differential evolution algorithm with archive-base mutation (MOMDE-AM) is proposed. In MOMDE-AM, with the purpose of reducing the loss of population evolution information, a modified mutation strategy with archive is introduced, which could utilize several useful inferior solutions and provide promising direction information toward the true Pareto front. The performance of MOMDE-AM is compared with five other MOEAs on five bi-objective and five tri-objective optimization problems. The simulation and statistical analysis results indicate that the overall performance of MOMDE-AM is better than those of the compared algorithms on these test functions. Finally, MOMDE-AM is used to optimize ten operation conditions of the \(p\)-xylene oxidation reaction process; the results show that MOMDE-AM is an effective and efficient optimization tool for solving actual MOPs.     

New measures for comparing optimization algorithms on dynamic optimization problems

Dynamic optimization problems have emerged as an important field of research during the last two decades, since many real-world optimization problems are changing over time. These problems need fast and accurate algorithms, not only to locate the optimum in a limited amount of time but also track its trajectories as close as possible. Although lots of research efforts have been given in developing dynamic benchmark generator/problems and proposing algorithms to solve these problems, the role of numerical performance measurements have been barely considered in the literature. Several performance criteria have been already proposed to evaluate the performance of algorithms. However, because they only take confined aspects of the algorithms into consideration, they do not provide enough information about the effectiveness of each algorithm. In this paper, at first we review the existing performance measures and then we present a set of two measures as a framework for comparing algorithms in dynamic environments, named fitness adaptation speed and alpha–accuracy. A comparative study is then conducted among different state-of-the-art algorithms on moving peaks benchmark via proposed metrics, along with several other performance measures, to demonstrate the relative advantages of the introduced measures. We hope that the collected knowledge in this paper opens a door toward a more comprehensive comparison among algorithms for dynamic optimization problems.

     

求解多维背包问题的蚁群-拉格朗日松弛混合优化算法

针对多维背包问题(MKP) NP-hard、约束强的特点, 提出一种高效的蚁群-拉格朗日松弛(LR) 混合优化算法. 该算法以蚁群优化(ACO) 为基本框架, 并基于LR 对偶信息定义了一种MKP效用指标. ACO使得整体算法具有全局搜索能力, 所设计的效用指标将MKP的优化目标与约束条件有机地融合在一起. 该指标一方面可以用来定 义MKP核问题, 降低问题规模; 另一方面, 可以用作ACO的启发因子, 引导算法在有希望的解区域中强化搜索. 在大量标准算例上的测试结果表明, 所提出算法的鲁棒性较好; 与其他已有算法相比, 在求解质量和求解效率方面均具有很强的竞争力.

     

Human behavior-based optimization: a novel metaheuristic approach to solve complex optimization problems

Optimization techniques, specially evolutionary algorithms, have been widely used for solving various scientific and engineering optimization problems because of their flexibility and simplicity. In this paper, a novel metaheuristic optimization method, namely human behavior-based optimization (HBBO), is presented. Despite many of the optimization algorithms that use nature as the principal source of inspiration, HBBO uses the human behavior as the main source of inspiration. In this paper, first some human behaviors that are needed to understand the algorithm are discussed and after that it is shown that how it can be used for solving the practical optimization problems. HBBO is capable of solving many types of optimization problems such as high-dimensional multimodal functions, which have multiple local minima, and unimodal functions. In order to demonstrate the performance of HBBO, the proposed algorithm has been tested on a set of well-known benchmark functions and compared with other optimization algorithms. The results have been shown that this algorithm outperforms other optimization algorithms in terms of algorithm reliability, result accuracy and convergence speed.

     

Data Mining based mutation function for engineering problems with mixed continuous-discrete design variables

Genetic algorithms are well established for solving engineering optimization problems having both continuous and discrete design variables. In this paper, a mutation function for discrete design variables based on Data Mining is introduced. The M5P Data Mining algorithm is used to build rules for the prediction of the optimization objectives with respect to the discrete design variables. The most promising combinations of discrete design variables are then selected in the mutation function of the genetic algorithm GAME to create children. This approach results in faster convergence and better results for both single and multi-objective problems when compared with a standard mutation scheme of discrete design variables. The optimization of a vehicle space frame showed that a mutation probability between 40% and 60% for the discrete design variables results in the fastest convergence. A multi-objective aerospace conceptual design example showed a substantial improvement in the number of pareto-optimal solutions found after 100 generations.     

Optimal BSR Solutions to Several Convex Polygon Problems

This paper focuses on BSR (Broadcasting with Selective Reduction) implementation of algorithms solving basic convex polygon problems. More precisely, constant time solutions on a linear number, max(N, M) (where N and M are the number of edges of the two considered polygons), of processors for computing the maximum distance between two convex polygons, finding critical support lines of two convex polygons, computing the diameter, the width of a convex polygon and the vector sum of two convex polygons are described. These solutions are based on the merging slopes technique using one criterion BSR operations.     

Integrating domain and constraint privacy reasoning in the distributed stochastic algorithm with breakouts

Privacy has traditionally been a major motivation of distributed problem solving. One popular approach to enable privacy in distributed environments is to implement complex cryptographic protocols. In this paper, we propose a different, orthogonal approach, which is to control the quality and the quantity of publicized data. We consider the Open Constraint Programming model and focus on algorithms that solve Distributed Constraint Optimization Problems (DCOPs) using a local search approach. Two such popular algorithms exist to find good solutions to DCOP: DSA and GDBA. In this paper, we propose DSAB, a new algorithm that merges ideas from both algorithms to allow extensive handling of constraint privacy. We also study how algorithms behave when solving Utilitarian DCOPs, where utilitarian agents want to reach an agreement while reducing the privacy loss. We experimentally study how the utilitarian approach impacts the quality of the solution and of publicized data.

     

A systems approach to evolutionary multiobjective structural optimization and beyond

Multiobjective evolutionary algorithms (MOEAs) have shown to be effective in solving a wide range of test problems. However, it is not straightforward to apply MOEAs to complex real-world problems. This paper discusses the major challenges we face in applying MOEAs to complex structural optimization, including the involvement of time-consuming and multi-disciplinary quality evaluation processes, changing environments, vagueness in formulating criteria formulation, and the involvement of multiple sub-systems. We propose that the successful tackling of all these aspects give birth to a systems approach to evolutionary design optimization characterized by considerations at four levels, namely, the system property level, temporal level, spatial level and process level. Finally, we suggest a few promising future research topics in evolutionary structural design that consist in the necessary steps towards a life-like design approach, where design principles found in biological systems such as self-organization, self-repair and scalability play a central role.