A surrogate-assisted evolution strategy for constrained multi-objective optimization

In many real-world optimization problems, several conflicting objectives must be achieved and optimized simultaneously and the solutions are often required to satisfy certain restrictions or constraints. Moreover, in some applications, the numerical values of the objectives and constraints are obtained from computationally expensive simulations. Many multi-objective optimization algorithms for continuous optimization have been proposed in the literature and some have been incorporated or used in conjunction with expert and intelligent systems. However, relatively few of these multi-objective algorithms handle constraints, and even fewer, use surrogates to approximate the objective or constraint functions when these functions are computationally expensive. This paper proposes a surrogate-assisted evolution strategy (ES) that can be used for constrained multi-objective optimization of expensive black-box objective functions subject to expensive black-box inequality constraints. Such an algorithm can be incorporated into an intelligent system that finds approximate Pareto optimal solutions to simulation-based constrained multi-objective optimization problems in various applications including engineering design optimization, production management and manufacturing. The main idea in the proposed algorithm is to generate a large number of trial offspring in each generation and use the surrogates to predict the objective and constraint function values of these trial offspring. Then the algorithm performs an approximate non-dominated sort of the trial offspring based on the predicted objective and constraint function values, and then it selects the most promising offspring (those with the smallest predicted ranks from the non-dominated sort) to become the actual offspring for the current generation that will be evaluated using the expensive objective and constraint functions. The proposed method is implemented using cubic radial basis function (RBF) surrogate models to assist the ES. The resulting RBF-assisted ES is compared with the original ES and to NSGA-II on 20 test problems involving 2–15 decision variables, 2–5 objectives and up to 13 inequality constraints. These problems include well-known benchmark problems and application problems in manufacturing and robotics. The numerical results showed that the RBF-assisted ES generally outperformed the original ES and NSGA-II on the problems used when the computational budget is relatively limited. These results suggest that the proposed surrogate-assisted ES is promising for computationally expensive constrained multi-objective optimization.     

Adaptive MLS-HDMR metamodeling techniques for high dimensional problems

Metamodeling technique is to represent the approximation of input variables and output variables. With the exponential increase of dimension of assigned problems, accurate and robust model is difficult to achieve by popular regression methodologies. High-dimensional model representation (HDMR) is a general set of metamodel assessment and analysis tools to improve the efficiency of deducing high dimensional underlying system behavior. In this paper, a new HDMR, based on moving least square (MLS), termed as MLS-HDMR, is introduced. The MLS-HDMR naturally explores and exploits the linearity/nonlinearity and correlation relationships among variables of the underlying function, which is unknown or computationally expensive. Furthermore, to improve the efficiency of the MLS-HDMR, an intelligent sampling strategy, DIviding RECTangles (DIRECT) method is used to sample points. Multiple mathematical test functions are given to illustrate the modeling principles, procedures, and the efficiency and accuracy of the MLS-HDMR models with problems of a wide scope of dimensionalities.     

Comparative study of HDMRs and other popular metamodeling techniques for high dimensional problems

The efficiency of optimization for the high dimensional problem has been improved by the metamodeling techniques in multidisciplinary in the past decades. In this study, comparative studies are implemented for high dimensional problems on the accuracy of four popular metamodeling methods, Kriging (KRG), radial basis function (RBF), least square support vector regression (LSSVR) and cut-high dimensional model representation (cut-HDMR) methods. Besides, HDMR methods with different basis functions are considered, including KRG-HDMR, RBF-HDMR and SVR-HDMR. Four factors that might influence the quality of metamodeling methods involving parameter interaction of problems, sample sizes, noise level and sampling strategies are considered. The results show that the LSSVR with Gaussian kernel, using Latin hypercube sampling (LHS) strategy, constructs more accurate metamodels than the KRG. The RBF with Gaussian basis function performs poor in the group. Generally, cut-HDMR methods perform much better than the other metamodeling methods when handling the function with weak parameter interaction, but not better when handling the function with strong parameter interaction.

     

High dimensional model representation (HDMR) coupled intelligent sampling strategy for nonlinear problems

High-dimensional model representation (HDMR) is a general set of metamodel assessment and analysis tools to improve the efficiency of high dimensional underlying system behavior. Compared with the current popular modeling methods, such as Kriging (KG), radial basis function (RBF), and the moving least square approximation method (MLS), the distinctive characteristic of the HDMR is to decouple the input variables. Therefore, a high dimensional problem can be transformed as a low, middle or combination of middle dimensional function. Although the HDMR is a feasible method for high dimensional problems, the computational cost is still a bottleneck for complex engineering problems. To improve the efficiency of the HDMR method further, the purpose of this study is to use an intelligent sampling method for the HDMR. Because the HDMR cannot be integrated with the sampling method directly, a projection-based intelligent method is suggested. Compared with the popular HDMR methods, the construction procedure for the HDMR-based model is optimized. To validate the performance of the suggested method, multiple mathematical test functions are given to illustrate the modeling principles, procedures, and the efficiency and accuracy of HDMR models with problems of a wide scope of dimensionalities.     

Performance study of gradient-enhanced Kriging

The use of surrogate models for approximating computationally expensive simulations has been on the rise for the last two decades. Kriging-based surrogate models are popular for approximating deterministic computer models. In this work, the performance of Kriging is investigated when gradient information is introduced for the approximation of computationally expensive black-box simulations. This approach, known as gradient-enhanced Kriging, is applied to various benchmark functions of varying dimensionality (2D-20D). As expected, results from the benchmark problems show that additional gradient information can significantly enhance the accuracy of Kriging. Gradient-enhanced Kriging provides a better approximation even when gradient information is only partially available. Further comparison between gradient-enhanced Kriging and an alternative formulation of gradient-enhanced Kriging, called indirect gradient-enhanced Kriging, highlights various advantages of directly employing gradient information, such as improved surrogate model accuracy, better conditioning of the correlation matrix, etc. Finally, gradient-enhanced Kriging is used to model 6- and 10-variable fluid–structure interaction problems from bio-mechanics to identify the arterial wall’s stiffness.     

Concurrent treatment of parametric uncertainty and metamodeling uncertainty in robust design

Robust design is an effective approach to design under uncertainty. Many works exist on mitigating the influence of parametric uncertainty associated with design or noise variables. However, simulation models are often computationally expensive and need to be replaced by metamodels created using limited samples. This introduces the so-called metamodeling uncertainty. Previous metamodel-based robust designs often treat a metamodel as the real model and ignore the influence of metamodeling uncertainty. In this study, we introduce a new uncertainty quantification method to evaluate the compound effect of both parametric uncertainty and metamodeling uncertainty. Then the new uncertainty quantification method is used for robust design. Simplified expressions of the response mean and variance is derived for a Kriging metamodel. Furthermore, the concept of robust design is extended for metamodel-based robust design accounting for both sources of uncertainty. To validate the benefits of our method, two mathematical examples without constraints are first illustrated. Results show that a robust design solution can be misleading without considering the metamodeling uncertainty. The proposed uncertainty quantification method for robust design is shown to be effective in mitigating the effect of metamodeling uncertainty, and the obtained solution is found to be more “robust” compared to the conventional approach. An automotive crashworthiness example, a highly expensive and non-linear problem, is used to illustrate the benefits of considering both sources of uncertainty in robust design with constraints. Results indicate that the proposed method can reduce the risk of constraint violation due to metamodel uncertainty and results in a “safer” robust solution.     

Efficient surrogate construction by combining response surface methodology and reduced order modeling

Response surface methodology is an efficient method for approximating the output of complex, computationally expensive codes. Challenges remain however in decreasing their construction cost as well as in approximating high dimensional output instead of scalar values. We propose a novel approach addressing both these challenges simultaneously for cases where the expensive code solves partial differential equations involving the resolution of a large system of equations, such as by finite element. Our method is based on the combination of response surface methodology and reduced order modeling by projection, also known as reduced basis modeling. The novel idea is to carry out the full resolution of the system only at a small, appropriately chosen, number of points. At the other points only the inexpensive reduced basis solution is computed while controlling the quality of the approximation being sought. A first application of the proposed surrogate modeling approach is presented for the problem of identification of orthotropic elastic constants from full field displacement measurements based on a tensile test on a plate with a hole. A surrogate of the entire displacement field was constructed using the proposed method. A second application involves the construction of a surrogate for the temperature field in a rocket engine combustion chamber wall. Compared to traditional response surface methodology a reduction by about an order of magnitude in the total system resolution time was achieved using the proposed sequential surrogate construction strategy.     

A RBF-based constrained global optimization algorithm for problems with computationally expensive objective and constraints

This paper presents a metamodel-based constrained optimization method, called Radial basis function-based Constrained Global Optimization (RCGO), to solve optimization problems involving computationally expensive objective function and inequality constraints. RCGO is an extension of the adaptive metamodel-based global optimization (AMGO) algorithm which can handle unconstrained black-box optimization problems. Firstly, a sequential sampling method is implemented to obtain the initial points for building the radial basis function (RBF) approximations to all computational expensive functions while enforcing a feasible solution. Then, an auxiliary objective function subject to the approximate constraints is constructed to determine the next iterative point and further improve the solution. During the process, a distance function with a group of exponents is introduced in the auxiliary function to balance the local exploitation and the global exploration. The RCGO method is tested on a series of benchmark problems, and the results demonstrate that RCGO needs fewer costly evaluations and can be applied for costly constrained problems with all infeasible start points. And the test results on the 30D problems demonstrate that RCGO has advantages in solving the problems. The proposed method is then applied to the design of a cycloid gear pump and desirable results are obtained.     

结合Kriging和物理规划的多目标代理优化算法

遗传算法处理高耗时且具有黑箱性的工程优化问题效率不足。为了提高工程优化效率，结合Kriging代理优化和物理规划，提出了基于Kriging和物理规划的多目标代理优化算法。在处理多目标问题时，使用物理规划将多目标问题转换成单目标问题，再使用Kriging代理优化对单目标问题进行求解。通过两个多目标数值算例和一个工程实例对提出的算法进行验证。结果表明，提出的算法能够求出符合偏好设置的Pareto最优解，且算法的效率更高。     

一种新颖的混合响应面优化方法

为减少黑箱优化过程中的评估次数,提出了一种新颖的混合响应面优化方法(HRSO),利用混合响应面建立高精度的近似模型作为代理模型,通过迭代更新响应面不断接近真实模型,从而完成优化。以Dixon-Szego函数类作为测试函数,以评估次数为方法性能优劣的评价指标,实验结果表明,与Gutmann-RBF、CORS-RBF两种方法相比,HRSO能够在较少的评估次数内满足相同的收敛条件,且向全局快速收敛,是一种适合求解黑箱优化问题的方法。     

SVM-HDMR高维非线性近似模型构造法

为了构造高维下的近似模型,将最小二乘支持向量机（LS-SVM）引入切割高维模型表示（Cut-HDMR）,提出了SVM-HDMR高维非线性近似模型构造法,给出了相应的自适应采样和模型构造算法。该方法利用Cut-HDMR将高维问题转化为一系列低维问题,用LS-SVM求解这些低维问题。数值算例的测试结果表明该方法具有较好的近似精度,且与传统近似方法相比极大地降低了计算成本,从而更适用于高维工程问题的求解。     

Ensemble of surrogate based global optimization methods using hierarchical design space reduction

For computationally expensive black-box problems, surrogate models are widely employed to reduce the needed computation time and efforts during the search of the global optimum. However, the construction of an effective surrogate model over a large design space remains a challenge in many cases. In this work, a new global optimization method using an ensemble of surrogates and hierarchical design space reduction is proposed to deal with the optimization problems with computation-intensive, black-box objective functions. During the search, an ensemble of three representative surrogate techniques with optimized weight factors is used for selecting promising sample points, narrowing down space exploration and identifying the global optimum. The design space is classified into: Original Global Space (OGS), Promising Joint Space (PJS), Important Local Space (ILS), using the newly proposed hierarchical design space reduction (HSR). Tested using eighteen representative benchmark and two engineering design optimization problems, the newly proposed global optimization method shows improved capability in identifying promising search area and reducing design space, and superior search efficiency and robustness in identifying the global optimum.     

Sequential optimization using multi-level cokriging and extended expected improvement criterion

To reduce the computational cost of metamodel based design optimization that directly relies on the computationally expensive simulation, the multi-fidelity cokriging method has gained increasing attention by fusing data from two or more models with different levels of fidelity. In this paper, an enhanced cokriging based sequential optimization method is proposed. Firstly, the impact of considering full correlation of data among all models on the hyper-parameter estimation during cokriging modeling is investigated by setting up a unified maximum likelihood function. Then, to reduce the computational cost, an extended expected improvement function is established to more reasonably identify the location and fidelity level of the next response evaluation based on the original expected improvement criterion. The results from comparative studies and one airfoil aerodynamic optimization application show that the proposed cokriging based sequential optimization method is more accurate in modeling and efficient in model evaluation than some existing popular approaches, demonstrating its effectiveness and relative merits.     

An efficient multi-objective optimization method for black-box functions using sequential approximate technique

Multi-objective optimization problems in practical engineering usually involve expensive black-box functions. How to reduce the number of function evaluations at a good approximation of Pareto frontier has been a crucial issue. To this aim, an efficient multi-objective optimization method based on a sequential approximate technique is suggested in this paper. In each iteration, according to the prediction of radial basis function with a micro multi-objective genetic algorithm, an extended trust region updating strategy is adopted to adjust the design region, a sample inheriting strategy is presented to reduce the number of new function evaluations, and then a local-densifying strategy is proposed to improve the accuracy of approximations in concerned regions. At the end of each iteration, the obtained actual Pareto optimal points are stored in an external archive and are updated as the iteration process. The effect of the present method is demonstrated by eight test functions. Finally, it is employed to perform the structure optimization of a vehicle door.     

基于PRS元模型的粒子群优化算法及其应用*

针对传统粒子群算法优化黑箱模型过程中存在巨大计算开销的问题,提出一种基于PRS元模型的改进粒子群优化算法—PPSO算法。在该算法迭代过程中,构建PRS元模型,利用其最优值点辅助粒子种群的更新,此外仅选择元模型预估集中优值集的粒子进行目标函数的计算仿真。将PPSO算法与基本粒子群算法、混沌粒子群算法进行数值测试对比,并应用于模糊控制器的优化设计,仿真结果表明该算法可减少真实估值次数,提高优化搜索能力。     

Metamodeling development for reliability-based design optimization of automotive body structure

Metamodels are commonly used in reliability-based design optimization (RBDO) due to the enormously expensive computation cost of numerical simulations. However, for large-scale design optimization of automotive body structure, with the increasing number of design variable and enhanced nonlinearity degree of structural performance, polynomial response surface which is commonly used for vehicle design optimization often suffers exponentially increased computation burden and serious loss of approximation accuracy. In this paper, support vector regression, along with other four complex metamodeling techniques including moving least square, artificial neural network, radial basis function and Kriging, is investigated for approximating frontal crashworthiness performance which is one of the most highly nonlinear performances. It aims at testing support vector regression and providing advanced metamodeling technique for RBDO of automotive body structure. Approximation results are compared in both accuracy and computational efficiency. Based on the frontal crashworthiness example, it is found that support vector regression and moving least square are preferable techniques to approximate structural performances with good accuracy. But support vector regression is recommended for its computational efficiency and better approximation potential. Moreover, the ensemble of support vector regression, moving least square, Kriging and artificial neural network is an effective alternative and is proved, in the RBDO example for the lightweight design of front body structure, to outperform any other single metamodel. The remarkable predominance indicates that the ensemble of support vector regression, moving least square, Kriging and artificial neural network holds great potential in approximating highly nonlinear performances for RBDO of automotive body structure.     

Fast Generation of Approximate Blue Noise Point Sets

Poisson‐disk sampling is a popular sampling method because of its blue noise power spectrum, but generation of these samples is computationally very expensive. In this paper, we propose an efficient method for fast generation of a large number of blue noise samples using a small initial patch of Poisson‐disk samples that can be generated with any existing approach. Our main idea is to convolve this set of samples with another to generate our final set of samples. We use the convolution theorem from signal processing to show that the spectrum of the resulting sample set preserves the blue noise properties. Since our method is approximate, we have error with respect to the true Poisson‐disk samples, but we show both mathematically and practically that this error is only a function of the number of samples in the small initial patch and is therefore bounded. Our method is parallelizable and we demonstrate an implementation of it on a GPU, running more than 10 times faster than any previous method and generating more than 49 million 2D samples per second. We can also use the proposed approach to generate multidimensional blue noise samples.     

Accelerating the kernel-method-based feature extraction procedure from the viewpoint of numerical approximation

The kernel method suffers from the following problem: the computational efficiency of the feature extraction procedure is inversely proportional to the size of the training sample set. In this paper, from a novel viewpoint, we propose a very simple and mathematically tractable method to produce the computationally efficient kernel-method-based feature extraction procedure. We first address the issue that how to make the feature extraction result of the reformulated kernel method well approximate that of the naïve kernel method. We identify these training samples that statistically contribute much to the feature extraction results and exploit them to reformulate the kernel method to produce the computationally efficient kernel-method-based feature extraction procedure. Indeed, the proposed method has the following basic idea: when one training sample has little effect on the feature extraction result and statistically has the high correlation with regard to all the training samples, the feature extraction term associated with this training sample can be removed from the feature extraction procedure. The proposed method has the following advantages: First, it proposes, for the first time, to improve the kernel method through formal and reasonable evaluation on the feature extraction term. Second, the proposed method improves the kernel method at a low extra cost and thus has a much more computationally efficient training phase than most of the previous improvements to the kernel method. The experimental comparison shows that the proposed method performs well in classification problems. This paper also intuitively shows the geometrical relation between the identified training samples and other training samples.     

Multi-fidelity information fusion based on prediction of kriging

In this paper, a novel kriging-based multi-fidelity method is proposed. Firstly, the model uncertainty of low-fidelity and high-fidelity models is quantified. On the other hand, the prediction uncertainty of kriging-based surrogate models(SM) is confirmed by its mean square error. After that, the integral uncertainty is acquired by math modeling. Meanwhile, the SMs are constructed through data from low-fidelity and high-fidelity models. Eventually, the low-fidelity (LF) and high-fidelity (HF) SMs with integral uncertainty are obtained and a proposed fusion algorithm is implemented. The fusion algorithm refers to the Kalman filter’s idea of optimal estimation to utilize the independent information from different models synthetically. Through several mathematical examples implemented, the fused SM is certified that its variance is decreased and the fused results tend to the true value. In addition, an engineering example about autonomous underwater vehicles’ hull design is provided to prove the feasibility of this proposed multi-fidelity method in practice. In the future, it will be a helpful tool to deal with reliability optimization of black-box problems and potentially applied in multidisciplinary design optimization.     

Incomplete series expansion for function approximation

We present an incomplete series expansion (ISE) as a basis for function approximation. The ISE is expressed in terms of an approximate Hessian matrix, which may contain second, third, and even higher order “main” or diagonal terms, but which excludes “interaction” or off-diagonal terms. From the ISE, a family of approximation functions may be derived. The approximation functions may be based on an arbitrary number of previously sampled points, and any of the function and gradient values at suitable previously sampled points may be enforced when deriving the approximation functions. When function values only are enforced, the storage requirements are minimal. However, irrespective of the conditions enforced, the approximate Hessian matrix is a sparse diagonal matrix. In addition, the resultant approximations are separable. Hence, the proposed approximation functions are very well-suited for use in gradient-based sequential approximate optimization requiring computationally expensive simulations; a typical example is structural design problems with many design variables and constraints. We derived a wide selection of approximations from the family of ISE approximating functions; these include approximations based on the substitution of reciprocal and exponential intervening variables. A comparison with popular approximating functions previously proposed illustrates the accuracy and flexibility of the new family of approximation functions. In fact, a number of popular approximating functions previously proposed for structural optimization applications derive from our ISE. Based on the similarly named paper presented at the Sixth World Congress on Structural and Multidisciplinary Optimization, Rio de Janeiro, Brazil, May 2005