Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions

This paper presents a new algorithm for derivative-free optimization of expensive black-box objective functions subject to expensive black-box inequality constraints. The proposed algorithm, called ConstrLMSRBF, uses radial basis function (RBF) surrogate models and is an extension of the Local Metric Stochastic RBF (LMSRBF) algorithm by Regis and Shoemaker (2007a) [1] that can handle black-box inequality constraints. Previous algorithms for the optimization of expensive functions using surrogate models have mostly dealt with bound constrained problems where only the objective function is expensive, and so, the surrogate models are used to approximate the objective function only. In contrast, ConstrLMSRBF builds RBF surrogate models for the objective function and also for all the constraint functions in each iteration, and uses these RBF models to guide the selection of the next point where the objective and constraint functions will be evaluated. Computational results indicate that ConstrLMSRBF is better than alternative methods on 9 out of 14 test problems and on the MOPTA08 problem from the automotive industry (Jones, 2008 [2]). The MOPTA08 problem has 124 decision variables and 68 inequality constraints and is considered a large-scale problem in the area of expensive black-box optimization. The alternative methods include a Mesh Adaptive Direct Search (MADS) algorithm (Abramson and Audet, 2006 [3]; Audet and Dennis, 2006 [4]) that uses a kriging-based surrogate model, the Multistart LMSRBF algorithm by Regis and Shoemaker (2007a) [1] modified to handle black-box constraints via a penalty approach, a genetic algorithm, a pattern search algorithm, a sequential quadratic programming algorithm, and COBYLA (Powell, 1994 [5]), which is a derivative-free trust-region algorithm. Based on the results of this study, the results in Jones (2008) [2] and other approaches presented at the ISMP 2009 conference, ConstrLMSRBF appears to be among the best, if not the best, known algorithm for the MOPTA08 problem in the sense of providing the most improvement from an initial feasible solution within a very limited number of objective and constraint function evaluations.     

Self-adjusting parameter control for surrogate-assisted constrained optimization under limited budgets

Constrained optimization of high-dimensional numerical problems plays an important role in many scientific and industrial applications. Function evaluations in many industrial applications are severely limited and often only little analytical information about objective function and constraint functions is available. For such expensive black-box optimization tasks, the constraint optimization algorithm COBRA (Constrained Optimization By Radial Basis Function Approximation) was proposed, making use of RBF (radial basis function) surrogate modeling for both objective and constraint functions. COBRA has shown remarkable success in solving reliably complex benchmark problems in less than 500 function evaluations. Unfortunately, COBRA requires careful adjustment of parameters in order to do so.In this work we present a new algorithm SACOBRA (Self-Adjusting COBRA), which is based on COBRA and capable of achieving high-quality results with very few function evaluations and no parameter tuning. It is shown with the help of performance profiles on a set of benchmark problems (G-problems, MOPTA08) that SACOBRA consistently outperforms COBRA algorithms with different fixed parameter settings. We analyze the importance of the new elements in SACOBRA and show that each element of SACOBRA plays a role to boost up the overall optimization performance. We discuss the reasons and get in this way a better understanding of high-quality RBF surrogate modeling.     

A hybrid surrogate and pattern search optimization method and application to microelectronics

A hybrid method for robust and efficient optimization process is developed by integrating a new response surface method and pattern search algorithm. The method is based on: (1) multipoint approximations of the objective and constraint functions, (2) a multiquadric radial basis function (RBF) for the zeroth-order function approximation and a new RBF plus polynomial-based moving least-squares approximation for the first-order enhanced function approximation, and (3) a pattern search algorithm to impose a descent condition and applied adaptive subregion management strategy. Several numerical examples are presented to illustrate accuracy and computational efficiency of the proposed method for both function approximation and design optimization. To demonstrate the effectiveness of the proposed hybrid method, it is applied to obtain optimum designs of a microelectronic packaging system. A two-stage optimization approach is proposed for the design optimization. The material properties of microelectronic packaging system and the shape parameters of solder ball are selected as design variables. Through design optimization, significant improvements of durability performances are obtained using the proposed hybrid optimization method.     

Cooperative learning for radial basis function networks using particle swarm optimization

This paper presents a new evolutionary cooperative learning scheme, able to solve function approximation and classification problems with improved accuracy and generalization capabilities. The proposed method optimizes the construction of radial basis function (RBF) networks, based on a cooperative particle swarm optimization (CPSO) framework. It allows for using variable-width basis functions, which increase the flexibility of the produced models, while performing full network optimization by concurrently determining the rest of the RBF parameters, namely center locations, synaptic weights and network size. To avoid the excessive number of design variables, which hinders the optimization task, a compact representation scheme is introduced, using two distinct swarms. The first swarm applies the non-symmetric fuzzy means algorithm to calculate the network structure and RBF kernel center coordinates, while the second encodes the basis function widths by introducing a modified neighbor coverage heuristic. The two swarms work together in a cooperative way, by exchanging information towards discovering improved RBF network configurations, whereas a suitably tailored reset operation is incorporated to help avoid stagnation. The superiority of the proposed scheme is illustrated through implementation in a wide range of benchmark problems, and comparison with alternative approaches.     

一种改进PSO优化RBF神经网络的新方法

为了克服神经网络模型结构和参数难以设置的缺点,提出了一种改进粒子群优化的径向基函数(RBF)神经网络的新方法.首先将最近邻聚类用于RBF神经网络隐层中心向量的确定,同时对引入适应度值择优选取的原则对基本粒子群算法进行改进,采用改进粒子群(IMPSO)算法对最近邻聚类的聚类半径进行优化,合理的确定了RBF神经网络的隐层结构.将改进PSO优化的RBF神经网络应用于非线性函数逼近和混沌时间序列预测,经实验仿真验证.与基本粒子群(PSO)算法,收缩因子粒子群(CFA PSO)算法优化的RBF神经网络相比较,其在识别精度和收敛速度上都有了显著的提高.     

A surrogate-assisted evolution strategy for constrained multi-objective optimization

In many real-world optimization problems, several conflicting objectives must be achieved and optimized simultaneously and the solutions are often required to satisfy certain restrictions or constraints. Moreover, in some applications, the numerical values of the objectives and constraints are obtained from computationally expensive simulations. Many multi-objective optimization algorithms for continuous optimization have been proposed in the literature and some have been incorporated or used in conjunction with expert and intelligent systems. However, relatively few of these multi-objective algorithms handle constraints, and even fewer, use surrogates to approximate the objective or constraint functions when these functions are computationally expensive. This paper proposes a surrogate-assisted evolution strategy (ES) that can be used for constrained multi-objective optimization of expensive black-box objective functions subject to expensive black-box inequality constraints. Such an algorithm can be incorporated into an intelligent system that finds approximate Pareto optimal solutions to simulation-based constrained multi-objective optimization problems in various applications including engineering design optimization, production management and manufacturing. The main idea in the proposed algorithm is to generate a large number of trial offspring in each generation and use the surrogates to predict the objective and constraint function values of these trial offspring. Then the algorithm performs an approximate non-dominated sort of the trial offspring based on the predicted objective and constraint function values, and then it selects the most promising offspring (those with the smallest predicted ranks from the non-dominated sort) to become the actual offspring for the current generation that will be evaluated using the expensive objective and constraint functions. The proposed method is implemented using cubic radial basis function (RBF) surrogate models to assist the ES. The resulting RBF-assisted ES is compared with the original ES and to NSGA-II on 20 test problems involving 2–15 decision variables, 2–5 objectives and up to 13 inequality constraints. These problems include well-known benchmark problems and application problems in manufacturing and robotics. The numerical results showed that the RBF-assisted ES generally outperformed the original ES and NSGA-II on the problems used when the computational budget is relatively limited. These results suggest that the proposed surrogate-assisted ES is promising for computationally expensive constrained multi-objective optimization.     

Numerical experience with a derivative-free trust-funnel method for nonlinear optimization problems with general nonlinear constraints

A trust-funnel method is proposed for solving nonlinear optimization problems with general nonlinear constraints. It extends the one presented by Gould and Toint [Nonlinear programming without a penalty function or a filter. Math. Prog. 122(1):155–196, 2010], originally proposed for equality-constrained optimization problems only, to problems with both equality and inequality constraints and where simple bounds are also considered. As the original one, our method makes use of neither filter nor penalty functions and considers the objective function and the constraints as independently as possible. To handle the bounds, an active-set approach is employed. We then exploit techniques developed for derivative-free optimization (DFO) to obtain a method that can also be used to solve problems where the derivatives are unavailable or are available at a prohibitive cost. The resulting approach extends the DEFT-FUNNEL algorithm presented by Sampaio and Toint [A derivative-free trust-funnel method for equality-constrained nonlinear optimization. Comput. Optim. Appl. 61(1):25–49, 2015], which implements a derivative-free trust-funnel method for equality-constrained problems. Numerical experiments with the extended algorithm show that our approach compares favourably to other well-known model-based algorithms for DFO.     

异构集成代理辅助多目标粒子群优化算法

针对代理辅助进化算法在减少昂贵适应度评估时难以通过少量样本点构造高质量代理模型的问题,提出异构集成代理辅助多目标粒子群优化算法。该方法通过使用加权平均法将Kriging模型和径向基函数网络模型组合成高精度的异构集成模型,达到增强算法处理不确定性信息能力的目的。基于集成学习的两种代理模型分别应用于全局搜索和局部搜索,在多目标粒子群优化算法框架基础上,新提出的方法为每个目标函数自适应地构造了异构集成模型,利用其模型的非支配解来指导粒子群的更新,得出目标函数的最优解集。实验结果表明,所提方法提高了代理模型的搜索能力,减少了评估次数,并且随着搜索维度的增加,其计算复杂性也具有更好的可扩展性。     

基于RBF-AR的船舶变形极短期预报

统一空间基准是海上作战平台实现精准探测打击的重要保证,而船体角变形的存在将严重影响空间基准的建立。针对这一问题,提出一种基于状态相依自回归（state-dependent auto-regressive, SD-AR）与径向基（radial basis function, RBF）神经网络的极短期变形预报方法,实现船体角形变的实时预报,为后续角变形的补偿提供依据。不同于传统的时间序列预报方法,该模型用一组RBF网络来逼近SD-AR模型中的函数系数,并采用一种结构化的非线性参数优化方法（structured nonlinear parameter optimization method, SNPOM）辨识该模型。基于该RBF-AR预报模型,给出了船舶变形预报算法设计并进行了仿真实验。实验结果表明,该方法在船体变形预测精度上优于传统时间序列预测方法,具有较好的应用前景。     

A RBF-based constrained global optimization algorithm for problems with computationally expensive objective and constraints

This paper presents a metamodel-based constrained optimization method, called Radial basis function-based Constrained Global Optimization (RCGO), to solve optimization problems involving computationally expensive objective function and inequality constraints. RCGO is an extension of the adaptive metamodel-based global optimization (AMGO) algorithm which can handle unconstrained black-box optimization problems. Firstly, a sequential sampling method is implemented to obtain the initial points for building the radial basis function (RBF) approximations to all computational expensive functions while enforcing a feasible solution. Then, an auxiliary objective function subject to the approximate constraints is constructed to determine the next iterative point and further improve the solution. During the process, a distance function with a group of exponents is introduced in the auxiliary function to balance the local exploitation and the global exploration. The RCGO method is tested on a series of benchmark problems, and the results demonstrate that RCGO needs fewer costly evaluations and can be applied for costly constrained problems with all infeasible start points. And the test results on the 30D problems demonstrate that RCGO has advantages in solving the problems. The proposed method is then applied to the design of a cycloid gear pump and desirable results are obtained.