A framework for design optimization using surrogates

Design optimization is a computationally expensive process as it requires the assessment of numerous designs and each of such assessments may be based on expensive analyses (e.g. computational fluid dynamics method or finite element based method). One way to contain the computational time within affordable limits is to use computationally cheaper approximations (surrogates) in lieu of the actual analyses during the course of optimization. This article introduces a framework for design optimization using surrogates. The framework is built upon a stochastic, zero-order, population-based optimization algorithm, which is embedded with a modified elitism scheme, to ensure convergence in the actual function space. The accuracy of the surrogate model is maintained via periodic retraining and the number of data points required to create the surrogate model is identified by a k-means clustering algorithm. A comparison is provided between different surrogate models (Kriging, radial basis functions (Exact and Fixed) and Cokriging) using a number of mathematical test functions and engineering design optimization problems. The results clearly indicate that for a given fixed number of actual function evaluations, the surrogate assisted optimization model consistently performs better than a pure optimization model using actual function evaluations.     

A new Kriging–Bat Algorithm for solving computationally expensive black-box global optimization problems

Many global optimization (GO) algorithms have been introduced in recent decades to deal with the Computationally Expensive Black-Box (CEBB) optimization problems. The high number of objective function evaluations, required by conventional GO methods, is prohibitive or at least inconvenient for practical design applications. In this work, a new Kriging–Bat algorithm (K–BA) is introduced for solving CEBB problems with further improved search efficiency and robustness. A Kriging surrogate model (SM) is integrated with the Bat Algorithm (BA) to find the global optimum using substantially reduced number of evaluations of the computationally expensive objective function. The new K–BA algorithm is tested and compared with other well-known GO algorithms, using a set of standard benchmark problems with 2 to 16 design variables, as well as a real-life engineering optimization application, to determine its search capability, efficiency and robustness. Results of the comprehensive tests demonstrated the suitability and superior capability of the new K–BA.     

MISO: mixed-integer surrogate optimization framework

We introduce MISO, the mixed-integer surrogate optimization framework. MISO aims at solving computationally expensive black-box optimization problems with mixed-integer variables. This type of optimization problem is encountered in many applications for which time consuming simulation codes must be run in order to obtain an objective function value. Examples include optimal reliability design and structural optimization. A single objective function evaluation may take from several minutes to hours or even days. Thus, only very few objective function evaluations are allowable during the optimization. The development of algorithms for this type of optimization problems has, however, rarely been addressed in the literature. Because the objective function is black-box, derivatives are not available and numerically approximating the derivatives requires a prohibitively large number of function evaluations. Therefore, we use computationally cheap surrogate models to approximate the expensive objective function and to decide at which points in the variable domain the expensive objective function should be evaluated. We develop a general surrogate model framework and show how sampling strategies of well-known surrogate model algorithms for continuous optimization can be modified for mixed-integer variables. We introduce two new algorithms that combine different sampling strategies and local search to obtain high-accuracy solutions. We compare MISO in numerical experiments to a genetic algorithm, NOMAD version 3.6.2, and SO-MI. The results show that MISO is in general more efficient than NOMAD and the genetic algorithm with respect to finding improved solutions within a limited budget of allowable evaluations. The performance of MISO depends on the chosen sampling strategy. The MISO algorithm that combines a coordinate perturbation search with a target value strategy and a local search performs best among all algorithms.     

Reliability-based design optimization with progressive surrogate models

Reliability-based design optimization (RBDO) has traditionally been solved as a nested (bilevel) optimization problem, which is a computationally expensive approach. Unilevel and decoupled approaches for solving the RBDO problem have also been suggested in the past to improve the computational efficiency. However, these approaches also require a large number of response evaluations during optimization. To alleviate the computational burden, surrogate models have been used for reliability evaluation. These approaches involve construction of surrogate models for the reliability computation at each point visited by the optimizer in the design variable space. In this article, a novel approach to solving the RBDO problem is proposed based on a progressive sensitivity surrogate model. The sensitivity surrogate models are built in the design variable space outside the optimization loop using the kriging method or the moving least squares (MLS) method based on sample points generated from low-discrepancy sampling (LDS) to estimate the most probable point of failure (MPP). During the iterative deterministic optimization, the MPP is estimated from the surrogate model for each design point visited by the optimizer. The surrogate sensitivity model is also progressively updated for each new iteration of deterministic optimization by adding new points and their responses. Four example problems are presented showing the relative merits of the kriging and MLS approaches and the overall accuracy and improved efficiency of the proposed approach.     

Efficient global optimization for high-dimensional constrained problems by using the Kriging models combined with the partial least squares method

In many engineering optimization problems, the number of function evaluations is often very limited because of the computational cost to run one high-fidelity numerical simulation. Using a classic optimization algorithm, such as a derivative-based algorithm or an evolutionary algorithm, directly on a computational model is not suitable in this case. A common approach to addressing this challenge is to use black-box surrogate modelling techniques. The most popular surrogate-based optimization algorithm is the efficient global optimization (EGO) algorithm, which is an iterative sampling algorithm that adds one (or many) point(s) per iteration. This algorithm is often based on an infill sampling criterion, called expected improvement, which represents a trade-off between promising and uncertain areas. Many studies have shown the efficiency of EGO, particularly when the number of input variables is relatively low. However, its performance on high-dimensional problems is still poor since the Kriging models used are time-consuming to build. To deal with this issue, this article introduces a surrogate-based optimization method that is suited to high-dimensional problems. The method first uses the ‘locating the regional extreme’ criterion, which incorporates minimizing the surrogate model while also maximizing the expected improvement criterion. Then, it replaces the Kriging models by the KPLS(+K) models (Kriging combined with the partial least squares method), which are more suitable for high-dimensional problems. Finally, the proposed approach is validated by a comparison with alternative methods existing in the literature on some analytical functions and on 12-dimensional and 50-dimensional instances of the benchmark automotive problem ‘MOPTA08’.     

Efficient infill sampling for unconstrained robust optimization problems

A novel infill sampling criterion is proposed for efficient estimation of the global robust optimum of expensive computer simulation based problems. The algorithm is especially geared towards addressing problems that are affected by uncertainties in design variables and problem parameters. The method is based on constructing metamodels using Kriging and adaptively sampling the response surface via a principle of expected improvement adapted for robust optimization. Several numerical examples and an engineering case study are used to demonstrate the ability of the algorithm to estimate the global robust optimum using a limited number of expensive function evaluations.     

A method for simulation based optimization using radial basis functions

We propose an algorithm for the global optimization of expensive and noisy black box functions using a surrogate model based on radial basis functions (RBFs). A method for RBF-based approximation is introduced in order to handle noise. New points are selected to minimize the total model uncertainty weighted against the surrogate function value. The algorithm is extended to multiple objective functions by instead weighting against the distance to the surrogate Pareto front; it therefore constitutes the first algorithm for expensive, noisy and multiobjective problems in the literature. Numerical results on analytical test functions show promise in comparison to other (commercial) algorithms, as well as results from a simulation based optimization problem.     

基于统计提升准则的注塑工艺参数优化

为了同时改善生产平板型注塑制品时的总体收缩度和收缩均匀度,提出基于统计提升准则的注塑成型工艺参数的多目标优化方法,寻找平衡两个质量指标的优化设计.首先利用小规模的实验设计方法获得建模数据集,针对应用中存在的建模数据奇异点问题提出一种数据预处理方法,并依此分别建立两个指标的初始替代模型,用于代替优化过程中代价高昂的计算分析;随后依据Pareto统计提升准则寻找新的采样点加入建模数据集来重新建模,使寻优结果不断趋近真实的Pareto前沿.仿真结果表明,较常规的建模优化方法,本文提出的方法能使用较少的采样数据,显著地改善平板制品的收缩质量.对于HDPE材质的矩形制品,保压曲线先恒定后线性递减可以获得好的收缩均匀度,使用压力上限值恒定保压可以获得好的平均收缩度.     

A surrogate assisted parallel multiobjective evolutionary algorithm for robust engineering design

A number of multi-objective evolutionary algorithms have been proposed in recent years and many of them have been used to solve engineering design optimization problems. However, designs need to be robust for real-life implementation, i.e. performance should not degrade substantially under expected variations in the variable values or operating conditions. Solutions of constrained robust design optimization problems should not be too close to the constraint boundaries so that they remain feasible under expected variations. A robust design optimization problem is far more computationally expensive than a design optimization problem as neighbourhood assessments of every solution are required to compute the performance variance and to ensure neighbourhood feasibility. A framework for robust design optimization using a surrogate model for neighbourhood assessments is introduced in this article. The robust design optimization problem is modelled as a multi-objective optimization problem with the aim of simultaneously maximizing performance and minimizing performance variance. A modified constraint-handling scheme is implemented to deal with neighbourhood feasibility. A radial basis function (RBF) network is used as a surrogate model and the accuracy of this model is maintained via periodic retraining. In addition to using surrogates to reduce computational time, the algorithm has been implemented on multiple processors using a master–slave topology. The preliminary results of two constrained robust design optimization problems indicate that substantial savings in the actual number of function evaluations are possible while maintaining an acceptable level of solution quality.     

Stochastic free vibration analyses of composite shallow doubly curved shells – A Kriging model approach

This paper presents the Kriging model approach for stochastic free vibration analysis of composite shallow doubly curved shells. The finite element formulation is carried out considering rotary inertia and transverse shear deformation based on Mindlin’s theory. The stochastic natural frequencies are expressed in terms of Kriging surrogate models. The influence of random variation of different input parameters on the output natural frequencies is addressed. The sampling size and computational cost is reduced by employing the present method compared to direct Monte Carlo simulation. The convergence studies and error analysis are carried out to ensure the accuracy of present approach. The stochastic mode shapes and frequency response function are also depicted for a typical laminate configuration. Statistical analysis is presented to illustrate the results using Kriging model and its performance.     

Infill sampling criteria for surrogate-based optimization with constraint handling

This article discusses the benefits of different infill sampling criteria used in surrogate-based constrained global optimization. A new method which selects multiple updates based on Pareto optimal solutions is introduced showing improvements over a number of existing methods. The construction of surrogates (also known as meta-models or response surface models) involves the selection of a limited number of designs which are analysed using the original expensive functions. A typical approach involves two stages. First the surrogate is built using an initial sampling plan; the second stage updates the model using an infill sampling criterion to select further designs that offer improvement. Selecting multiple update points at each iteration, allowing distribution of the expensive function evaluations on several processors offers large potential for accelerating the overall optimization process. This article provides a comparison between different infill sampling criteria suitable for selecting multiple update points in the presence of constraints.     

Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems

Particle swarm optimization (PSO) is a population-based, heuristic technique based on social behaviour that performs well on a variety of problems including those with non-convex, non-smooth objective functions with multiple minima. However, the method can be computationally expensive in that a large number of function calls is required. This is a drawback when evaluations depend on an off-the-shelf simulation program, which is often the case in engineering applications. An algorithm is proposed which incorporates surrogates as a stand-in for the expensive objective function, within the PSO framework. Numerical results are presented on standard benchmarking problems and a simulation-based hydrology application to show that this hybrid can improve efficiency. A comparison is made between the application of a global PSO and a standard PSO to the same formulations with surrogates. Finally, data profiles, probability of success, and a measure of the signal-to-noise ratio of the the objective function are used to assess the use of a surrogate.     

基于全局灵敏度分析的某自动装填机构轻量化设计

针对某自动装填机构轻量化设计中出现参数多、模型计算量大等问题,提出将全局灵敏度分析与代理模型技术相结合的优化策略。通过基于Morris轨迹的全局灵敏度分析从32个系统参数中确定14个关键参数,基于拉丁超立方采样技术及径向基函数神经网络技术(Radial basis function neural networks, RBF NN)建立系统响应关于关键参数的代理模型,用多岛遗传算法对系统参数进行优化求解,致机构重量下降21.8%。数值检验结果表明仅含关键参数的代理模型预测精度较高,证明该方法在多参数复杂系统结构轻量化设计中的有效性。     

Assessment of the efficiency of Kriging surrogate models for structural reliability analysis

This paper presents an assessment of the efficiency of the Kriging interpolation models as surrogate models for structural reliability problems involving time-consuming numerical models such as nonlinear finite element analysis structural models. The efficiency assessment is performed through a systematic comparison of the accuracy of the failure probability predictions based on the first-order reliability method using the most common first- and second-order polynomial regression models and the Kriging interpolation models as surrogates for the true limit state function. An application problem of practical importance in the field of marine structures that requires the evaluation of a nonlinear finite element structural model is adopted as numerical example. The accuracy of the failure probability predictions is characterised as a function of the number of support points, dispersion of the support points in relation to the so-called design point and order of the Kriging basis functions. It is shown with the application problem considered that the Kriging interpolation models are efficient surrogate models for structural reliability problems and can provide significantly more accurate failure probability predictions as compared with the most common polynomial regression models.     

A non‐stationary covariance‐based Kriging method for metamodelling in engineering design

Metamodels are widely used to facilitate the analysis and optimization of engineering systems that involve computationally expensive simulations. Kriging is a metamodelling technique that is well known for its ability to build surrogate models of responses with non‐linear behaviour. However, the assumption of a stationary covariance structure underlying Kriging does not hold in situations where the level of smoothness of a response varies significantly. Although non‐stationary Gaussian process models have been studied for years in statistics and geostatistics communities, this has largely been for physical experimental data in relatively low dimensions. In this paper, the non‐stationary covariance structure is incorporated into Kriging modelling for computer simulations. To represent the non‐stationary covariance structure, we adopt a non‐linear mapping approach based on parameterized density functions. To avoid over‐parameterizing for the high dimension problems typical of engineering design, we propose a modified version of the non‐linear map approach, with a sparser, yet flexible, parameterization. The effectiveness of the proposed method is demonstrated through both mathematical and engineering examples. The robustness of the method is verified by testing multiple functions under various sampling settings. We also demonstrate that our method is effective in quantifying prediction uncertainty associated with the use of metamodels. Copyright © 2006 John Wiley & Sons, Ltd.     

A Constraint Mapping Approach to the Structural Optimization of an Expensive Model using Surrogates

The use of response surface methods are well established in the global optimization of expensive functions, the response surface acting as a surrogate to the expensive function objective.In structural design however, the change in objective may vary little between the two models: it is more often the constraints that change with models of varying fidelity. Here approaches are described whereby the coarse model constraints are mapped so that the mapped constraints more faithfully approximate the fine model constraints. The shape optimization of a simple structure demonstrates the approach.     

连续体结构裂纹识别的Kriging代理模型求解策略

本文提出了一种基于代理模型的裂纹识别方法,利用初始样本构造Kriging代理模型,建立裂纹模型参数与结构响应的关系,来代替结构的原有的结构参数与动力响应关系,最大程度地减少了反演优化迭代过程中反复网格剖分和冗繁的有限元计算次数。使用最优设计加点准则进行代理模型修正,以改进初始代理模型的准确性。为了识别连续体结构上的裂纹模型参数,采用随机粒子群优化方法搜索代理模型多极值域下的全局最优解。数值算例对具有裂纹的悬臂梁和板结构进行了裂纹识别。结果表明,该方法能有效地识别裂纹参数,并且具有良好的抗噪性能。此外,初始样本数量对裂纹识别效率及识别结果的影响也进行了讨论。     

Locally weighted regression models for surrogate-assisted design optimization

We consider engineering design optimization problems where the objective and/or constraint functions are evaluated by means of computationally expensive blackboxes. Our practical optimization strategy consists of solving surrogate optimization problems in the search step of the mesh adaptive direct search algorithm. In this paper, we consider locally weighted regression models to build the necessary surrogates, and present three ideas for appropriate and effective use of locally weighted scatterplot smoothing (LOWESS) models for surrogate optimization. First, a method is proposed to reduce the computational cost of LOWESS models. Second, a local scaling coefficient is introduced to adapt LOWESS models to the density of neighboring points while retaining smoothness. Finally, an appropriate order error metric is used to select the optimal shape coefficient of the LOWESS model. Our surrogate-assisted optimization approach utilizes LOWESS models to both generate and rank promising candidates found in the search and poll steps. The “real” blackbox functions that govern the original optimization problem are then evaluated at these ranked candidates with an opportunistic strategy, reducing CPU time significantly. Computational results are reported for four engineering design problems with up to six variables and six constraints. The results demonstrate the effectiveness of the LOWESS models as well as the order error metric for surrogate optimization.     

Evolutionary reliable regional Kriging surrogate for expensive optimization

This work proposes a sequential optimization algorithm, EORKS, combining a Kriging surrogate from an adaptive sampling and an iterative constrained search in the dynamic reliable regions to reduce the sampling size in expensive optimization. A surrogate established from small samples is liable to limited generality, which leads to a false prediction of optimum. EORKS applies Kriging variance to establish the reliable region neighbouring the learning samples to constrain the evolutionary searches of the surrogate. The verified quasi-optimum is used as an additional sample to dynamically update the regional model according to the prediction accuracy. A hybrid infilling strategy switches between the iterative quasi-optima and the maximum expected improvement from Kriging to prevent early convergence of local optimum. EORKS provides superior optima in several benchmark functions and an engineering design problem, using much smaller samples compared with the literature results, which demonstrates the sampling efficiency and searching robustness.     

A general optimality criteria algorithm for a class of engineering optimization problems

An optimality criteria (OC)-based algorithm for optimization of a general class of nonlinear programming (NLP) problems is presented. The algorithm is only applicable to problems where the objective and constraint functions satisfy certain monotonicity properties. For multiply constrained problems which satisfy these assumptions, the algorithm is attractive compared with existing NLP methods as well as prevalent OC methods, as the latter involve computationally expensive active set and step-size control strategies. The fixed point algorithm presented here is applicable not only to structural optimization problems but also to certain problems as occur in resource allocation and inventory models. Convergence aspects are discussed. The fixed point update or resizing formula is given physical significance, which brings out a strength and trim feature. The number of function evaluations remains independent of the number of variables, allowing the efficient solution of problems with large number of variables.