A surrogate-assisted particle swarm optimization algorithm based on efficient global optimization for expensive black-box problems

Evolutionary algorithms cannot effectively handle computationally expensive problems because of the unaffordable computational cost brought by a large number of fitness evaluations. Therefore, surrogates are widely used to assist evolutionary algorithms in solving these problems. This article proposes an improved surrogate-assisted particle swarm optimization (ISAPSO) algorithm, in which a hybrid particle swarm optimization (PSO) is combined with global and local surrogates. The global surrogate is not only used to predict fitness values for reducing computational burden but also regarded as a global searcher to speed up the global search process of PSO by using an efficient global optimization algorithm, while the local one is constructed for a local search in the neighbourhood of the current optimal solution by finding the predicted optimal solution of the local surrogate. Empirical studies on 10 widely used benchmark problems and a real-world structural design optimization problem of a driving axle show that the ISAPSO algorithm is effective and highly competitive.     

A global optimization algorithm for simulation-based problems via the extended DIRECT scheme

This article presents a global optimization algorithm via the extension of the DIviding RECTangles (DIRECT) scheme to handle problems with computationally expensive simulations efficiently. The new optimization strategy improves the regular partition scheme of DIRECT to a flexible irregular partition scheme in order to utilize information from irregular points. The metamodelling technique is introduced to work with the flexible partition scheme to speed up the convergence, which is meaningful for simulation-based problems. Comparative results on eight representative benchmark problems and an engineering application with some existing global optimization algorithms indicate that the proposed global optimization strategy is promising for simulation-based problems in terms of efficiency and accuracy.     

Hybrid metamodel-based design space management method for expensive problems

Metamodel-based global optimization methods have been extensively studied for their great potential in solving expensive problems. In this work, a design space management strategy is proposed to improve the accuracy and efficiency of metamodel-based optimization methods. In this strategy, the whole design space is divided into two parts: the important region constructed using several expensive points and the other region. Combined with a previously developed hybrid metamodel strategy, a hybrid metamodel-based design space management method (HMDSM) is developed. In this method, three representative metamodels are used simultaneously in the search of the global optimum in both the important region and the other region. In the search process, the important region is iteratively reduced and the global optimum is soon captured. Tests using a series of benchmark mathematical functions and a practical expensive problem demonstrate the excellent performance of the proposed method.     

Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points

This article develops two new algorithms for constrained expensive black-box optimization that use radial basis function surrogates for the objective and constraint functions. These algorithms are called COBRA and Extended ConstrLMSRBF and, unlike previous surrogate-based approaches, they can be used for high-dimensional problems where all initial points are infeasible. They both follow a two-phase approach where the first phase finds a feasible point while the second phase improves this feasible point. COBRA and Extended ConstrLMSRBF are compared with alternative methods on 20 test problems and on the MOPTA08 benchmark automotive problem (D.R. Jones, Presented at MOPTA 2008), which has 124 decision variables and 68 black-box inequality constraints. The alternatives include a sequential penalty derivative-free algorithm, a direct search method with kriging surrogates, and two multistart methods. Numerical results show that COBRA algorithms are competitive with Extended ConstrLMSRBF and they generally outperform the alternatives on the MOPTA08 problem and most of the test problems.     

Trust regions in Kriging-based optimization with expected improvement

The Kriging-based Efficient Global Optimization (EGO) method works well on many expensive black-box optimization problems. However, it does not seem to perform well on problems with steep and narrow global minimum basins and on high-dimensional problems. This article develops a new Kriging-based optimization method called TRIKE (Trust Region Implementation in Kriging-based optimization with Expected improvement) that implements a trust-region-like approach where each iterate is obtained by maximizing an Expected Improvement (EI) function within some trust region. This trust region is adjusted depending on the ratio of the actual improvement to the EI. This article also develops the Kriging-based CYCLONE (CYClic Local search in OptimizatioN using Expected improvement) method that uses a cyclic pattern to determine the search regions where the EI is maximized. TRIKE and CYCLONE are compared with EGO on 28 test problems with up to 32 dimensions and on a 36-dimensional groundwater bioremediation application in appendices supplied as an online supplement available at http://dx.doi.org/10.1080/0305215X.2015.1082350. The results show that both algorithms yield substantial improvements over EGO and they are competitive with a radial basis function method.     

A surrogate-model based multi-modal optimization algorithm

In this article a new algorithm for optimization of multi-modal, nonlinear, black-box objective functions is introduced. It extends the recently-introduced adaptive multi-modal optimization by incorporating surrogate modelling features similar to response surface methods. The resulting algorithm has reduced computational intensity and is well-suited for optimization of expensive objective functions. It relies on an adaptive, multi-resolution mesh to obtain an initial estimation of the objective function surface. Local surrogate models are then constructed and used to generate additional trial points around the local minima discovered. The steps of mesh refinement and surrogate modelling continue until convergence is achieved. The algorithm produces progressively accurate surrogate models, which can be used for post-optimization studies such as sensitivity and tolerance analyses with minimal computational effort. This article demonstrates the effectiveness of the algorithm using comparative optimization of several multi-modal objective functions, and shows an engineering application of the design of a power electronic converter.     

An evolutionary algorithm for global optimization based on self-organizing maps

In this article, a new population-based algorithm for real-parameter global optimization is presented, which is denoted as self-organizing centroids optimization (SOC-opt). The proposed method uses a stochastic approach which is based on the sequential learning paradigm for self-organizing maps (SOMs). A modified version of the SOM is proposed where each cell contains an individual, which performs a search for a locally optimal solution and it is affected by the search for a global optimum. The movement of the individuals in the search space is based on a discrete-time dynamic filter, and various choices of this filter are possible to obtain different dynamics of the centroids. In this way, a general framework is defined where well-known algorithms represent a particular case. The proposed algorithm is validated through a set of problems, which include non-separable problems, and compared with state-of-the-art algorithms for global optimization.     

Mode-pursuing sampling method for global optimization on expensive black-box functions

The presence of black-box functions in engineering design, which are usually computation-intensive, demands efficient global optimization methods. This article proposes a new global optimization method for black-box functions. The global optimization method is based on a novel mode-pursuing sampling method that systematically generates more sample points in the neighborhood of the function mode while statistically covering the entire search space. Quadratic regression is performed to detect the region containing the global optimum. The sampling and detection process iterates until the global optimum is obtained. Through intensive testing, this method is found to be effective, efficient, robust, and applicable to both continuous and discontinuous functions. It supports simultaneous computation and applies to both unconstrained and constrained optimization problems. Because it does not call any existing global optimization tool, it can be used as a standalone global optimization method for inexpensive problems as well. Limitations of the method are also identified and discussed.     

Interval algorithm for global numerical optimization

The interval optimization algorithm shows great advantages in bound constrained global optimization. An interval algorithm is presented in this article based on a new selection criterion. The selection criterion is proposed based on numerical experiments and the parameter pf* designed by Casado, Garcia and Csendes in 2000. The proposed criterion at each iteration selects some intervals of which the number is not greater than a constant so that the possible memory problem during the implementation of the algorithm is avoided and the running time of the algorithm is decreased, when the dimension of the problem is increasing. Based on the selection criterion, the proposed algorithm is implemented for a wide set of tested functions which includes easy and hard problems. Numerical experiments show that the proposed algorithm is efficient.     

An adaptive species conservation genetic algorithm for multimodal optimization

This paper introduces an adaptive species conservation genetic algorithm (ASCGA) by defining a species with three parameters: species seed, species radius and species boundary fitness. A species is defined as a group of individuals that have similar characteristics and that are dominated by the best individual in the species, called the species seed. Species radius defines the species' upper boundary and the species boundary fitness is the lowest value of fitness in the boundary. Some heuristic algorithms have been developed to adjust these parameters and an ASCGA has been proposed to solve multimodal optimization problems. With heuristic techniques, ASCGA can automatically adjust species parameters and allow the species to adapt to an optimization problem. Experimental results presented demonstrate that the proposed algorithm is capable of finding the global and local optima of test multimodal optimization problems with a higher efficiency than the methods from the literature. ASCGA has also successfully found a significantly different solution of a 25‐bar space truss design and identified 761 local solutions of the 2‐D Shubert function. Copyright © 2009 John Wiley & Sons, Ltd.     

A novel metaheuristic for continuous optimization problems: Virus optimization algorithm

A novel metaheuristic for continuous optimization problems, named the virus optimization algorithm (VOA), is introduced and investigated. VOA is an iteratively population-based method that imitates the behaviour of viruses attacking a living cell. The number of viruses grows at each replication and is controlled by an immune system (a so-called ‘antivirus’) to prevent the explosive growth of the virus population. The viruses are divided into two classes (strong and common) to balance the exploitation and exploration effects. The performance of the VOA is validated through a set of eight benchmark functions, which are also subject to rotation and shifting effects to test its robustness. Extensive comparisons were conducted with over 40 well-known metaheuristic algorithms and their variations, such as artificial bee colony, artificial immune system, differential evolution, evolutionary programming, evolutionary strategy, genetic algorithm, harmony search, invasive weed optimization, memetic algorithm, particle swarm optimization and simulated annealing. The results showed that the VOA is a viable solution for continuous optimization.     

Trends,features, and tests of common and recently introduced global optimization methods

Global optimization techniques have been used extensively due to their capability in handling complex engineering problems. In addition to a number of well known global optimization techniques, many new methods have been introduced recently for various optimal design applications. In this work, a number of representative, well known and recently introduced global optimization techniques are closely examined and compared. The historical development, special features and trends on the development of global optimization algorithms are reviewed. Special attention is devoted to the recent developments of multidisciplinary design optimization algorithms based on effective metamodelling techniques. Commonly used benchmark optimization problems are used as test examples to reveal the pros and cons of these global optimization methods. A new meta-model based global optimization search method, introduced and improved recently by the authors, is also included in the tests and comparison.     

An adaptive direct multisearch method for black-box multi-objective optimization

Optimization and Engineering - At present, black-box and simulation-based optimization problems with multiple objective functions are becoming increasingly common in the engineering context. In...     

Using support vector machine and dynamic parameter encoding to enhance global optimization

This study presents an approach which combines support vector machine (SVM) and dynamic parameter encoding (DPE) to enhance the run-time performance of global optimization with time-consuming fitness function evaluations. SVMs are used as surrogate models to partly substitute for fitness evaluations. To reduce the computation time and guarantee correct convergence, this work proposes a novel strategy to adaptively adjust the number of fitness evaluations needed according to the approximate error of the surrogate model. Meanwhile, DPE is employed to compress the solution space, so that it not only accelerates the convergence but also decreases the approximate error. Numerical results of optimizing a few benchmark functions and an antenna in a practical application are presented, which verify the feasibility, efficiency and robustness of the proposed approach.     

求解矩形布局问题的自适应算法

矩形布局问题属于NP-Hard问题,其求解算法多为启发式算法。该文侧重于构造布局求解算法中定位函数(规则)的优化,将模拟退火算法的思想融入到遗传算法中,提出了求解矩形布局问题的自适应算法,其利用自适应交叉、变异及接收劣质解的概率等方法对定位函数中各参数进行优化。算法通过两种方式确定初始种群的数目,具有较强的适应性。在算法搜索的后期,利用差异性较大的个体进行交叉操作,从而保持种群的多样性。最后通过实例证明了该算法能够很好的应用于矩形布局问题的求解。     

Parallel global optimization with the particle swarm algorithm

Present day engineering optimization problems often impose large computational demands, resulting in long solution times even on a modern high-end processor. To obtain enhanced computational throughput and global search capability, we detail the coarse-grained parallelization of an increasingly popular global search method, the particle swarm optimization (PSO) algorithm. Parallel PSO performance was evaluated using two categories of optimization problems possessing multiple local minima-large-scale analytical test problems with computationally cheap function evaluations and medium-scale biomechanical system identification problems with computationally expensive function evaluations. For load-balanced analytical test problems formulated using 128 design variables, speedup was close to ideal and parallel efficiency above 95% for up to 32 nodes on a Beowulf cluster. In contrast, for load-imbalanced biomechanical system identification problems with 12 design variables, speedup plateaued and parallel efficiency decreased almost linearly with increasing number of nodes. The primary factor affecting parallel performance was the synchronization requirement of the parallel algorithm, which dictated that each iteration must wait for completion of the slowest fitness evaluation. When the analytical problems were solved using a fixed number of swarm iterations, a single population of 128 particles produced a better convergence rate than did multiple independent runs performed using sub-populations (8 runs with 16 particles, 4 runs with 32 particles, or 2 runs with 64 particles). These results suggest that (1) parallel PSO exhibits excellent parallel performance under load-balanced conditions, (2) an asynchronous implementation would be valuable for real-life problems subject to load imbalance, and (3) larger population sizes should be considered when multiple processors are available.     

Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems

Particle swarm optimization (PSO) is a population-based, heuristic technique based on social behaviour that performs well on a variety of problems including those with non-convex, non-smooth objective functions with multiple minima. However, the method can be computationally expensive in that a large number of function calls is required. This is a drawback when evaluations depend on an off-the-shelf simulation program, which is often the case in engineering applications. An algorithm is proposed which incorporates surrogates as a stand-in for the expensive objective function, within the PSO framework. Numerical results are presented on standard benchmarking problems and a simulation-based hydrology application to show that this hybrid can improve efficiency. A comparison is made between the application of a global PSO and a standard PSO to the same formulations with surrogates. Finally, data profiles, probability of success, and a measure of the signal-to-noise ratio of the the objective function are used to assess the use of a surrogate.     

A multiple surrogate assisted evolutionary algorithm for optimization involving iterative solvers

In many real-world optimization problems, the underlying objective and constraint function(s) are evaluated using computationally expensive iterative simulations such as the solvers for computational electro-magnetics, computational fluid dynamics, the finite element method, etc. The default practice is to run such simulations until convergence using termination criteria, such as maximum number of iterations, residual error thresholds or limits on computational time, to estimate the performance of a given design. This information is used to build computationally cheap approximations/surrogates which are subsequently used during the course of optimization in lieu of the actual simulations. However, it is possible to exploit information on pre-converged solutions if one has the control to abort simulations at various stages of convergence. This would mean access to various performance estimates in lower fidelities. Surrogate assisted optimization methods have rarely been used to deal with such classes of problem, where estimates at various levels of fidelity are available. In this article, a multiple surrogate assisted optimization approach is presented, where solutions are evaluated at various levels of fidelity during the course of the search. For any solution under consideration, the choice to evaluate it at an appropriate fidelity level is derived from neighbourhood information, i.e. rank correlations between performance at different fidelity levels and the highest fidelity level of the neighbouring solutions. Moreover, multiple types of surrogates are used to gain a competitive edge. The performance of the approach is illustrated using a simple 1D unconstrained analytical test function. Thereafter, the performance is further assessed using three 10D and three 20D test problems, and finally a practical design problem involving drag minimization of an unmanned underwater vehicle. The numerical experiments clearly demonstrate the benefits of the proposed approach for such classes of problem.     

A new deterministic global optimization method for general twice-differentiable constrained nonlinear programming problems

A deterministic global optimization method that is applicable to general nonlinear programming problems composed of twice-differentiable objective and constraint functions is proposed. The method hybridizes the branch-and-bound algorithm and a convex cut function (CCF). For a given subregion, the difference of a convex underestimator that does not need an iterative local optimizer to determine the lower bound of the objective function is generated. If the obtained lower bound is located in an infeasible region, then the CCF is generated for constraints to cut this region. The cutting region generated by the CCF forms a hyperellipsoid and serves as the basis of a discarding rule for the selected subregion. However, the convergence rate decreases as the number of cutting regions increases. To accelerate the convergence rate, an inclusion relation between two hyperellipsoids should be applied in order to reduce the number of cutting regions. It is shown that the two-hyperellipsoid inclusion relation is determined by maximizing a quadratic function over a sphere, which is a special case of a trust region subproblem. The proposed method is applied to twelve nonlinear programming test problems and five engineering design problems. Numerical results show that the proposed method converges in a finite calculation time and produces accurate solutions.     

一种自适应抽样的代理模型构建及其在复材结构优化中的应用

提出了基于一种自适应抽样和增强径向基插值的自适应代理模型方法,这种自适应抽样方法以确定适量的样本点数量和提高代理模型自适应能力为目的,使新增样本点位于设计空间的稀疏区域并确保所有的样本点均匀分布于设计空间以提高代理模型精度。标准误差用来判断代理模型的精度大小并决定是否对代理模型进行更新。一种条件随机抽样被用来对比本文的自适应抽样方法。经过对比验证发现,采用自适应抽样方法的代理模型精度比条件随机抽样方法的代理模型精度高。这种自适应代理模型结合多岛遗传算法被用来优化旋翼臂的碳纤维增强环氧树脂复合材料铺层角度使得旋翼臂的一阶模态频率最大。优化结果表明,不同的碳纤维增强环氧树脂复合材料铺层角度对旋翼臂的一阶模态频率值影响较大,优化结果获取了最优铺层角度,旋翼臂的一阶模态频率值被提高以远离激励频率而避免旋翼飞机的共振。