Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points

This article develops two new algorithms for constrained expensive black-box optimization that use radial basis function surrogates for the objective and constraint functions. These algorithms are called COBRA and Extended ConstrLMSRBF and, unlike previous surrogate-based approaches, they can be used for high-dimensional problems where all initial points are infeasible. They both follow a two-phase approach where the first phase finds a feasible point while the second phase improves this feasible point. COBRA and Extended ConstrLMSRBF are compared with alternative methods on 20 test problems and on the MOPTA08 benchmark automotive problem (D.R. Jones, Presented at MOPTA 2008), which has 124 decision variables and 68 black-box inequality constraints. The alternatives include a sequential penalty derivative-free algorithm, a direct search method with kriging surrogates, and two multistart methods. Numerical results show that COBRA algorithms are competitive with Extended ConstrLMSRBF and they generally outperform the alternatives on the MOPTA08 problem and most of the test problems.     

Efficient estimation of first passage probability of high-dimensional nonlinear systems

An efficient method for estimating low first passage probabilities of high-dimensional nonlinear systems based on asymptotic estimation of low probabilities is presented. The method does not require any a priori knowledge of the system, i.e. it is a black-box method, and has very low requirements on the system memory. Consequently, high-dimensional problems can be handled, and nonlinearities in the model neither bring any difficulty in applying it nor lead to considerable reduction of its efficiency. These characteristics suggest that the method is a powerful candidate for complicated problems. First, the failure probabilities of three well-known nonlinear systems are estimated. Next, a reduced degree-of-freedom model of a wind turbine is developed and is exposed to a turbulent wind field. The model incorporates very high dimensions and strong nonlinearities simultaneously. The failure probability of the wind turbine model is estimated down to very low values; this demonstrates the efficiency and power of the method on a realistic high-dimensional highly nonlinear system.     

Efficient estimation of first passage probability of high-dimensional nonlinear systems

An efficient method for estimating low first passage probabilities of high-dimensional nonlinear systems based on asymptotic estimation of low probabilities is presented. The method does not require any a priori knowledge of the system, i.e. it is a black-box method, and has very low requirements on the system memory. Consequently, high-dimensional problems can be handled, and nonlinearities in the model neither bring any difficulty in applying it nor lead to considerable reduction of its efficiency. These characteristics suggest that the method is a powerful candidate for complicated problems. First, the failure probabilities of three well-known nonlinear systems are estimated. Next, a reduced degree-of-freedom model of a wind turbine is developed and is exposed to a turbulent wind field. The model incorporates very high dimensions and strong nonlinearities simultaneously. The failure probability of the wind turbine model is estimated down to very low values; this demonstrates the efficiency and power of the method on a realistic high-dimensional highly nonlinear system.     

Research on high-dimensional model representation with various metamodels

Constructing approximation models with surrogate modelling is often carried out in engineering design to save computational cost. However, the problem of the ‘curse of dimensionality’ still exists, and high-dimensional model representation (HDMR) has been proven to be very efficient in solving high-dimensional, computationally expensive black-box problems. This article proposes a new HDMR by combining separate stand-alone metamodels to form an ensemble based on cut-HDMR. It can improve prediction accuracy and alleviate prediction uncertainty for different problems compared with previous HDMRs. In this article, 10 representative mathematical examples and two engineering examples are used to illustrate the proposed technique and previous HDMRs. Furthermore, a comprehensive comparison of four metrics between the ensemble HDMR and other single HDMRs is presented, with a wide scope of dimensionalities. The results show that the single HDMRs perform well on specified examples but the ensemble HDMR provides more accurate predictions for all the test problems.     

Efficient design optimization of duct-burners for combined-cycle and cogenerative plants

This article proposes an efficient gradient-based optimization procedure for black-box simulation codes and its application to the thermo-fluid-dynamic design optimization of a duct-burner for combined-cycle and cogenerative plants. The article also provides a discussion on some criteria that should drive the design optimization of these components, almost neglected by the scientific literature. Using a widely employed commercial (black-box) code, a new enhanced-mixing duct-burner has been first devised. Before looking at its design optimization, experimental investigations have been performed to assess the reliability of the modelling and the accuracy of the numerical predictions. Then, a finite-difference gradient-based optimization procedure that can be combined with black-box analysis codes has been developed: its efficiency relies on the simultaneous convergence of the flow solution and of the optimization process, as well as on the use of nested grid levels. After its validation, the proposed progressive optimization technique has been applied to two examples of thermo-fluid-dynamic design optimization of the new duct-burner: the first application aims at minimizing the outlet temperature gradient, whereas the second application aims at reducing the near-wall temperatures and shortening the flame, so as to strengthen its anchorage, while reducing the body heating and the thermal NO _x formation.     

求解包装生产中复杂问题的布谷鸟算法改进

目的 为了解决在求解复杂的高维函数优化问题时存在的求解精度不够高和易陷入局部最优等问题,提出一种基于莱维飞行发现概率的变步长布谷鸟搜索算法(LFCS).方法 在相同环境下,选取6个不同难度、不同类型的测试函数,将LFCS算法与IPSO,IDE,IABC,CS算法比较,分析算法的收敛速度和收敛精度.结果 相比其他4种算法,LFCS算法迭代次数更少,收敛速度更快,收敛精度更高.结论 无论是低维函数还是高维函数,LFCS算法在收敛速度和收敛精度方面都有所提高,尤其是针对复杂的高维函数优化问题,在取值范围较大的情况下,LFCS算法能够更快、更准地找到最优解.     

A global optimization algorithm for simulation-based problems via the extended DIRECT scheme

This article presents a global optimization algorithm via the extension of the DIviding RECTangles (DIRECT) scheme to handle problems with computationally expensive simulations efficiently. The new optimization strategy improves the regular partition scheme of DIRECT to a flexible irregular partition scheme in order to utilize information from irregular points. The metamodelling technique is introduced to work with the flexible partition scheme to speed up the convergence, which is meaningful for simulation-based problems. Comparative results on eight representative benchmark problems and an engineering application with some existing global optimization algorithms indicate that the proposed global optimization strategy is promising for simulation-based problems in terms of efficiency and accuracy.     

Efficient global optimization for high-dimensional constrained problems by using the Kriging models combined with the partial least squares method

In many engineering optimization problems, the number of function evaluations is often very limited because of the computational cost to run one high-fidelity numerical simulation. Using a classic optimization algorithm, such as a derivative-based algorithm or an evolutionary algorithm, directly on a computational model is not suitable in this case. A common approach to addressing this challenge is to use black-box surrogate modelling techniques. The most popular surrogate-based optimization algorithm is the efficient global optimization (EGO) algorithm, which is an iterative sampling algorithm that adds one (or many) point(s) per iteration. This algorithm is often based on an infill sampling criterion, called expected improvement, which represents a trade-off between promising and uncertain areas. Many studies have shown the efficiency of EGO, particularly when the number of input variables is relatively low. However, its performance on high-dimensional problems is still poor since the Kriging models used are time-consuming to build. To deal with this issue, this article introduces a surrogate-based optimization method that is suited to high-dimensional problems. The method first uses the ‘locating the regional extreme’ criterion, which incorporates minimizing the surrogate model while also maximizing the expected improvement criterion. Then, it replaces the Kriging models by the KPLS(+K) models (Kriging combined with the partial least squares method), which are more suitable for high-dimensional problems. Finally, the proposed approach is validated by a comparison with alternative methods existing in the literature on some analytical functions and on 12-dimensional and 50-dimensional instances of the benchmark automotive problem ‘MOPTA08’.     

Collocation-based stochastic finite element analysis for random field problems

A stochastic response surface method (SRSM) which has been previously proposed for problems dealing only with random variables is extended in this paper for problems in which physical properties exhibit spatial random variation and may be modeled as random fields. The formalism of the extended SRSM is similar to the spectral stochastic finite element method (SSFEM) in the sense that both of them utilize Karhunen–Loeve (K–L) expansion to represent the input, and polynomial chaos expansion to represent the output. However, the coefficients in the polynomial chaos expansion are calculated using a probabilistic collocation approach in SRSM. This strategy helps us to decouple the finite element and stochastic computations, and the finite element code can be treated as a black box, as in the case of a commercial code. The collocation-based SRSM approach is compared in this paper with an existing analytical SSFEM approach, which uses a Galerkin-based weighted residual formulation, and with a black-box SSFEM approach, which uses Latin Hypercube sampling for the design of experiments. Numerical examples are used to illustrate the features of the extended SRSM and to compare its efficiency and accuracy with the existing analytical and black-box versions of SSFEM.     

Structural reliability assessment by salp swarm algorithm–based FORM

The first-order reliability method (FORM) is well recognized as an efficient approach for reliability analysis. Rooted in considering the reliability problem as a constrained optimization of a function, the traditional FORM makes use of gradient-based optimization techniques to solve it. However, the gradient-based optimization techniques may result in local convergence or even divergence for the highly nonlinear or high-dimensional performance function. In this paper, a hybrid method combining the Salp Swarm Algorithm (SSA) and FORM is presented. In the proposed method, a Lagrangian objective function is constructed by the exterior penalty function method to facilitate meta-heuristic optimization strategies. Then, SSA with strong global optimization ability for highly nonlinear and high-dimensional problems is utilized to solve the Lagrangian objective function. In this regard, the proposed SSA-FORM is able to overcome the limitations of FORM including local convergence and divergence. Finally, the accuracy and efficiency of the proposed SSA-FORM are compared with two gradient-based FORMs and several heuristic-based FORMs through eight numerical examples. The results show that the proposed SSA-FORM can be generally applied for reliability analysis involving low-dimensional, high-dimensional, and implicit performance functions.     

Expected improvement based infill sampling for global robust optimization of constrained problems

A novel adaptive sampling scheme for efficient global robust optimization of constrained problems is proposed. The method addresses expensive to simulate black-box constrained problems affected by uncertainties for which only the bounds are known, while the probability distribution is not available. An iterative strategy for global robust optimization that adaptively samples the Kriging metamodel of the computationally expensive problem is proposed. The presented approach is tested on several benchmark problems and the average performance based on 100 runs is evaluated. The applicability of the method to engineering problems is also illustrated by applying robust optimization on an integrated photonic device affected by manufacturing uncertainties. The numerical results show consistent convergence to the global robust optimum using a limited number of expensive simulations.     

Trust regions in Kriging-based optimization with expected improvement

The Kriging-based Efficient Global Optimization (EGO) method works well on many expensive black-box optimization problems. However, it does not seem to perform well on problems with steep and narrow global minimum basins and on high-dimensional problems. This article develops a new Kriging-based optimization method called TRIKE (Trust Region Implementation in Kriging-based optimization with Expected improvement) that implements a trust-region-like approach where each iterate is obtained by maximizing an Expected Improvement (EI) function within some trust region. This trust region is adjusted depending on the ratio of the actual improvement to the EI. This article also develops the Kriging-based CYCLONE (CYClic Local search in OptimizatioN using Expected improvement) method that uses a cyclic pattern to determine the search regions where the EI is maximized. TRIKE and CYCLONE are compared with EGO on 28 test problems with up to 32 dimensions and on a 36-dimensional groundwater bioremediation application in appendices supplied as an online supplement available at http://dx.doi.org/10.1080/0305215X.2015.1082350. The results show that both algorithms yield substantial improvements over EGO and they are competitive with a radial basis function method.     

Robust optimization based on analytical evaluation of uncertainty propagation

Optimization under uncertainty requires proper handling of those input parameters that contain scatter. Scatter in input parameters propagates through the process and causes scatter in the output. Stochastic methods (e.g. Monte Carlo) are very popular for assessing uncertainty propagation using black-box function metamodels. However, they are expensive. Therefore, in this article a direct method of calculating uncertainty propagation has been employed based on the analytical integration of a metamodel of a process. Analytical handling of noise variables not only improves the accuracy of the results but also provides the gradients of the output with respect to input variables. This is advantageous in the case of gradient-based optimization. Additionally, it is shown that the analytical approach can be applied during sequential improvement of the metamodel to obtain a more accurate representative model of the black-box function and to enhance the search for the robust optimum.     

Calculation of KII in crack face contacts using X-FEM. Application to fretting fatigue

In this work, the modeling of LEFM problems that imply crack face closure and contact using the extended finite element method (X-FEM) is presented aiming at its application to fretting fatigue problems. An assessment of the accuracy in the calculation of K_II is performed for two different techniques to model crack face contacts in X-FEM: one is based on the use of additional elements to establish the contact and the other on a segment-to-segment (or mortar) approach. It is concluded that only the segment-to-segment approach can lead to optimal convergence rates of the error in K_II. The crack face contact modeling has also been applied to a fretting fatigue problem, where the estimation of K_II under crack closure conditions plays an important role in the stage I of fatigue crack propagation. The effect of the crack face friction coefficient has been studied and its influence on the range of K_II has been ascertained during loading and unloading cycles.     

A decreasing inertia weight particle swarm optimizer

It has been over ten years since the pioneering work of particle swarm optimization (PSO) espoused by Kennedy and Eberhart. Since then, various modifications, well suited to particular application areas, have been reported widely in the literature. The evolutionary concept of PSO is clear-cut in nature, easy to implement in practice, and computationally efficient in comparison to other evolutionary algorithms. The above-mentioned merits are primarily the motivation of this article to investigate PSO when applied to continuous optimization problems. The performance of conventional PSO on the solution quality and convergence speed deteriorates when the function to be optimized is multimodal or with a large problem size. Toward that end, it is of great practical value to develop a modified particle swarm optimizer suitable for solving high-dimensional, multimodal optimization problems. In the first part of the article, the design of experiments (DOE) has been conducted comprehensively to examine the influences of each parameter in PSO. Based upon the DOE results, a modified PSO algorithm, termed Decreasing-Weight Particle Swarm Optimization (DW-PSO), is addressed. Two performance measures, the success rate and number of function evaluations, are used to evaluate the proposed method. The computational comparisons with the existing PSO algorithms show that DW-PSO exhibits a noticeable advantage, especially when it is performed to solve high-dimensional problems.     

气动肌肉的最小二乘支持向量机迟滞模型

针对传统迟滞模型存在的待辨识参数多、参数辨识过程复杂和辨识精度低等问题,采用最小二乘支持向量机对气动肌肉的位移/气压迟滞开展建模研究。通过非线性映射将原始数据空间映射到高维空间,将原系统的非线性问题变成高维空间中的线性问题,借助于最小二乘法求解该线性方程组,从而提高其求解速度及收敛精度。在气动肌肉迟滞特性实验的基础上,采用所建数学模型,与经典的PI模型进行对比。结果表明,采用最小二乘支持向量机建立的数学模型具有更高的建模精度,均方差和平均误差相比PI模型分别减小了99.21%和99.1%,该方法可为后续气动肌肉的迟滞补偿控制提供有效的手段。     

An efficient global optimization algorithm for multifactor dynamic systems

Abstract

In this study we present an efficient global optimization method, DIviding RECTangle (DIRECT) algorithm, for parametric analysis of dynamic systems. In a bounded constrained problem the DIRECT algorithm explores multiple potentially optimal subspaces in one search. The algorithm also eliminates the need for derivative calculations which are required in some efficient gradient‐based methods. In this study the first optimization example is to find the dynamic parameters of a tennis racket. The second example is a biomechanical parametric study of a heel‐toe running model governed by six factors. The effectiveness of the DIRECT algorithm is compared with a genetic algorithm in an analysis of heel‐toe running. The result shows that the DIRECT algorithm obtains an improved result in 83% less execution time. It is demonstrated that the straightforward DIRECT algorithm provides a general procedure for solving global optimization problems efficiently and confidently.     

A combined r-h adaptive strategy based on material forces and error assessment for plane problems and bimaterial interfaces

An r-h adaptive scheme has been proposed and formulated for analysis of bimaterial interface problems using adaptive finite element method. It involves a combination of the configurational force based r-adaption with weighted laplacian smoothing and mesh enrichment by h-refinement. The Configurational driving force is evaluated by considering the weak form of the material force balance for bimaterial inerface problems. These forces assembled at nodes act as an indicator for r-adaption. A weighted laplacian smoothing is performed for smoothing the mesh. The h-adaptive strategy is based on a modifed weighted energy norm of error evaluated using supercovergent estimators. The proposed method applies specific non sliding interface strain compatibility requirements across inter material boundaries consistent with physical principles to obtain modified error estimators. The best sequence of combining r- and h-adaption has been evolved from numerical study. The study confirms that the proposed combined r-h adaption is more efficient than a purely h-adaptive approach and more flexible than a purely r-adaptive approach with better convergence characteristics and helps in obtaining optimal finite element meshes for a specified accuracy.     

Hybridizing evolutionary strategies with continuation methods for solving multi-objective problems

Two techniques for the numerical treatment of multi-objective optimization problems—a continuation method and a particle swarm optimizer—are combined in order to unite their particular advantages. Continuation methods can be applied very efficiently to perform the search along the Pareto set, even for high-dimensional models, but are of local nature. In contrast, many multi-objective particle swarm optimizers tend to have slow convergence, but instead accomplish the ‘global task’ well. An algorithm which combines these two techniques is proposed, some convergence results for continuous models are provided, possible realizations are discussed, and finally some numerical results are presented indicating the strength of this novel approach.     

Computation of resonant modes for axisymmetric Maxwell cavities using hp‐version edge finite elements

The computation of the resonant frequencies for closed cavities is not a trivial task: Multi‐materials and sharp corners all give rise to highly singular eigenfunctions. However, an approach using hp‐finite elements is well suited to such problems and, with the correct combination of h‐ and p‐refinements, it yields the theoretically predicated exponential rates of convergence. In this paper, we present a novel approach to the solution of axisymmetric cavity problems which uses a hierarchic H¹ and H (curl) conforming finite element basis. A selection of numerical examples is included and these demonstrate that the exponential rates of convergence are achieved in practice. Copyright © 2005 John Wiley & Sons, Ltd.