基于梯度投影法与随机优化算法的约束优化方法

针对带有线性等式和不等式约束的无确定函数形式的约束优化问题,提出一种利用梯度投影法与遗传算法、同时扰动随机逼近等随机算法相结合的优化方法。该方法利用遗传算法进行全局搜索,利用同时扰动随机逼近算法进行局部搜索,算法在每次进化时根据线性约束计算父个体处的梯度投影方向,以产生新个体,从而能够严格保证新个体满足全部约束条件。将上述约束优化算法应用于典型约束优化问题,其仿真结果表明了所提出算法的可行性和收敛性。     

A new unconstrained global optimization method based on clustering and parabolic approximation

A new unconstrained global optimization method based on clustering and parabolic approximation (GOBC-PA) is proposed. Although the proposed method is basically similar to other evolutionary and stochastic methods, it represents a significant advancement of global optimization technology for four important reasons. First, it is orders of magnitude faster than existing optimization methods for global optimization of unconstrained problems. Second, it has significantly better repeatability, numerical stability, and robustness than current methods in dealing with high dimensionally and many local minima functions. Third, it can easily and faster find the local minimums using the parabolic approximation instead of gradient descent or crossover operations. Fourth, it can easily adapted to any theoretical or industrial systems which are using the heuristic methods as an intelligent system, such as neural network and neuro-fuzzy inference system training, packing or allocation of objects, game optimization problems. In this study, we assume that the best cluster center gives the position of the possible global optimum. The usage of clustering and curve fitting techniques brings multi-start and local search properties to the proposed method. The experimental studies, such as performed on benchmark functions, a real world optimization problem and tuning the neural network parameters for classification problems, show that the proposed methodology is simple, faster and, it demonstrates a superior performance when compared with some state of the art methods.     

Approximate branch-and-bound global optimization using B-spline hypervolumes

This paper presents a B-spline-based branch-and-bound algorithm for unconstrained global optimization. The key components of the branch-and-bound, a well-known algorithm paradigm for global optimization, are a subdivision scheme and a bound calculation scheme. For these schemes, we first introduce a B-spline hypervolume to approximate an objective function defined in a design space, where the approximation is based on Latin-hypercube sampling points. We then describe a proposed algorithm for finding global solutions approximately within a prescribed tolerance. The algorithm includes two procedures that are performed iteratively until all stopping conditions are satisfied. One involves subdivision into mutually disjoint subspaces and computation of their bound information, both of which are accomplished by using B-spline hypervolumes. The other updates a search tree that represents a hierarchical structure of subdivided subspaces during the solution process. Finally, we examine the computational performance of the proposed algorithm on various test problems that cover most of the difficulties encountered in global optimization. The results show that the proposed algorithm is complete without using heuristics and has good potential for application in large-scale NP-hard optimization.     

The application of gradient-only optimization methods for problems discretized using non-constant methods

We study the minimization of objective functions containing non-physical jump discontinuities. These discontinuities arise when (partial) differential equations are discretized using non-constant methods and the resulting numerical solutions are used in computing the objective function. Although the functions may become discontinuous, gradient information may be computed at every point. Gradient information is computable everywhere since every point has an associated discretization for which (semi) analytical sensitivities can be calculated. Rather than the construction of global approximations using only function value information to overcome the discontinuities, we propose to use only the gradient information. We elaborate on the modifications of classical gradient based optimization algorithms for use in gradient-only approaches, and we then present gradient-only optimization strategies using both BFGS and a new spherical quadratic approximation for sequential approximate optimization (SAO). We then use the BFGS and SAO algorithms to solve three problems of practical interest, both unconstrained and constrained.     

A hybrid cooperative search algorithm for constrained optimization

Many engineering design problems can be formulated as constrained optimization problems which often consist of many mixed equality and inequality constraints. In this article, a hybrid coevolutionary method is developed to solve constrained optimization problems formulated as min–max problems. The new method is fast and capable of global search because of combining particle swarm optimization and gradient search to balance exploration and exploitation. It starts by transforming the problem into unconstrained one using an augmented Lagrangian function, then using two groups to optimize different components of the solution vector in a cooperative procedure. In each group, the final stage of the search procedure is accelerated by via a simple local search method on the best point reached by the preceding exploration based search. We validated the effectiveness and robustness of the proposed algorithm using several engineering problems taken from the specialised literature.     

A new spectral conjugate gradient method for large-scale unconstrained optimization

The spectral conjugate gradient methods, with simple construction and nice numerical performance, are a kind of effective methods for solving large-scale unconstrained optimization problems. In this paper, based on quasi-Newton direction and quasi-Newton condition, and motivated by the idea of spectral conjugate gradient method as well as Dai-Kou's selecting technique for conjugate parameter [SIAM J. Optim. 23 (2013), pp. 296–320], a new approach for generating spectral parameters is presented, where a new double-truncating technique, which can ensure both the sufficient descent property of the search directions and the bounded property of the sequence of spectral parameters, is introduced. Then a new associated spectral conjugate gradient method for large-scale unconstrained optimization is proposed. Under either the strong Wolfe line search or the generalized Wolfe line search, the proposed method is always globally convergent. Finally, a large number of comparison numerical experiments on large-scale instances from one thousand to two million variables are reported. The numerical results show that the proposed method is more promising.     

A hybrid shuffled complex evolution approach with pattern search for unconstrained optimization

The difficulties associated with using classical mathematical programming methods on complex optimization problems have contributed to the development of alternative and efficient numerical approaches. Recently, to overcome the limitations of classical optimization methods, researchers have proposed a wide variety of meta-heuristics for searching near-optimum solutions to problems. Among the existing meta-heuristic algorithms, a relatively new optimization paradigm is the Shuffled Complex Evolution at the University of Arizona (SCE-UA) which is a global optimization strategy that combines concepts of the competition evolution theory, downhill simplex procedure of Nelder-Mead, controlled random search and complex shuffling. In an attempt to reduce processing time and improve the quality of solutions, particularly to avoid being trapped in local optima, in this paper is proposed a hybrid SCE-UA approach. The proposed hybrid algorithm is the combination of SCE-UA (without Nelder-Mead downhill simplex procedure) and a pattern search approach, called SCE-PS, for unconstrained optimization. Pattern search methods are derivative-free, meaning that they do not use explicit or approximate derivatives. Moreover, pattern search algorithms are direct search methods well suitable for the global optimization of highly nonlinear, multiparameter, and multimodal objective functions. The proposed SCE-PS method is tested with six benchmark optimization problems. Simulation results show that the proposed SCE-PS improves the searching performance when compared with the classical SCE-UA and a genetic algorithm with floating-point representation for all the tested problems. As evidenced by the performance indices based on the mean performance of objective function in 30 runs and mean of computational time, the SCE-PS algorithm has demonstrated to be effective and efficient at locating best-practice optimal solutions for unconstrained optimization.     

混沌映射与精英高斯扰动的非线性灰狼优化算法

针对传统灰狼优化算法处理复杂优化问题时易于陷入局部最优,提出基于混沌Tent映射与精英高斯扰动的非线性灰狼优化算法.根据混沌Tent映射与对立学习机制,保证较优个体的同时,设计种群初始化方法,可使个体尽可能均匀分布;为有效均衡个体的局部开发和全局勘探能力,设计一种非线性收敛因子控制策略;在头狼选取上引入面向精英个体的高...     

混沌梯度组合优化算法

提出一种混沌梯度组合全局优化算法，并对该算法进行了收敛性分析．算法首先采用改进的变步长梯度法得到某个优化值，然后利用变尺度混沌搜索跳出局部极小，经过反复组合迭代，直至到达最优解．仿真结果表明，该算法能充分发挥梯度法寻优的快速性和混沌法寻优的全局搜索能力．     

An improved teaching-learning-based optimization algorithm for numerical and engineering optimization problems

The teaching-learning-based optimization (TLBO) algorithm, one of the recently proposed population-based algorithms, simulates the teaching-learning process in the classroom. This study proposes an improved TLBO (ITLBO), in which a feedback phase, mutation crossover operation of differential evolution (DE) algorithms, and chaotic perturbation mechanism are incorporated to significantly improve the performance of the algorithm. The feedback phase is used to enhance the learning style of the students and to promote the exploration capacity of the TLBO. The mutation crossover operation of DE is introduced to increase population diversity and to prevent premature convergence. The chaotic perturbation mechanism is used to ensure that the algorithm can escape the local optimal. Simulation results based on ten unconstrained benchmark problems and five constrained engineering design problems show that the ITLBO algorithm is better than, or at least comparable to, other state-of-the-art algorithms.     

A nonmonotone scaled conjugate gradient algorithm for large-scale unconstrained optimization

This paper proposes a nonmonotone scaled conjugate gradient algorithm for solving large-scale unconstrained optimization problems, which combines the idea of scaled memoryless Broyden–Fletcher–Goldfarb–Shanno preconditioned conjugate gradient method with the nonmonotone technique. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under appropriate assumptions, the method is proven to possess global convergence for nonconvex smooth functions, and R-linear convergence for strongly convex functions. Preliminary numerical results and related comparisons show the efficiency of the proposed method in practical computation.     

Descent three-term conjugate gradient methods based on secant conditions for unconstrained optimization

The conjugate gradient method is an effective method for large-scale unconstrained optimization problems. Recent research has proposed conjugate gradient methods based on secant conditions to establish fast convergence of the methods. However, these methods do not always generate a descent search direction. In contrast, Y. Narushima, H. Yabe, and J.A. Ford [A three-term conjugate gradient method with sufficient descent property for unconstrained optimization, SIAM J. Optim. 21 (2011), pp. 212–230] proposed a three-term conjugate gradient method which always satisfies the sufficient descent condition. This paper makes use of both ideas to propose descent three-term conjugate gradient methods based on particular secant conditions, and then shows their global convergence properties. Finally, numerical results are given.     

A RBF-based constrained global optimization algorithm for problems with computationally expensive objective and constraints

This paper presents a metamodel-based constrained optimization method, called Radial basis function-based Constrained Global Optimization (RCGO), to solve optimization problems involving computationally expensive objective function and inequality constraints. RCGO is an extension of the adaptive metamodel-based global optimization (AMGO) algorithm which can handle unconstrained black-box optimization problems. Firstly, a sequential sampling method is implemented to obtain the initial points for building the radial basis function (RBF) approximations to all computational expensive functions while enforcing a feasible solution. Then, an auxiliary objective function subject to the approximate constraints is constructed to determine the next iterative point and further improve the solution. During the process, a distance function with a group of exponents is introduced in the auxiliary function to balance the local exploitation and the global exploration. The RCGO method is tested on a series of benchmark problems, and the results demonstrate that RCGO needs fewer costly evaluations and can be applied for costly constrained problems with all infeasible start points. And the test results on the 30D problems demonstrate that RCGO has advantages in solving the problems. The proposed method is then applied to the design of a cycloid gear pump and desirable results are obtained.     

An efficient modified Polak–Ribière–Polyak conjugate gradient method with global convergence properties

The conjugate gradient (CG) method is one of the most popular methods for solving large-scale unconstrained optimization problems. In this paper, a new modified version of the CG formula that was introduced by Polak, Ribière, and Polyak is proposed for problems that are bounded below and have a Lipschitz-continuous gradient. The new parameter provides global convergence properties when the strong Wolfe-Powell (SWP) line search or the weak Wolfe-Powell (WWP) line search is employed. A proof of a sufficient descent condition is provided for the SWP line search. Numerical comparisons between the proposed parameter and other recent CG modifications are made on a set of standard unconstrained optimization problems. The numerical results demonstrate the efficiency of the proposed CG parameter compared with the other CG parameters.     

An exact penalty function-based differential search algorithm for constrained global optimization

Differential search (DS) is a recently developed derivative-free global heuristic optimization algorithm for solving unconstrained optimization problems. In this paper, by applying the idea of exact penalty function approach, a DS algorithm, where an S-type dynamical penalty factor is introduced so as to achieve a better balance between exploration and exploitation, is developed for constrained global optimization problems. To illustrate the applicability and effectiveness of the proposed approach, a comparison study is carried out by applying the proposed algorithm and other widely used evolutionary methods on 24 benchmark problems. The results obtained clearly indicate that the proposed method is more effective and efficient over the other widely used evolutionary methods for most these benchmark problems.     

Two modified spectral conjugate gradient methods and their global convergence for unconstrained optimization

In this paper, two modified spectral conjugate gradient methods which satisfy sufficient descent property are developed for unconstrained optimization problems. For uniformly convex problems, the first modified spectral type of conjugate gradient algorithm is proposed under the Wolfe line search rule. Moreover, the search direction of the modified spectral conjugate gradient method is sufficiently descent for uniformly convex functions. Furthermore, according to the Dai–Liao's conjugate condition, the second spectral type of conjugate gradient algorithm can generate some sufficient decent direction at each iteration for general functions. Therefore, the second method could be considered as a modification version of the Dai–Liao's algorithm. Under the suitable conditions, the proposed algorithms are globally convergent for uniformly convex functions and general functions. The numerical results show that the approaches presented in this paper are feasible and efficient.     

State-of-the-art in parallel nonlinear optimization

This survey is concerned with variants of nonlinear optimization methods designed for implementation on parallel computers. First, we consider a variety of methods for unconstrained minimization. We consider a particular type of parallelism (simultaneous function and gradient evaluations), and we concentrate on the main sources of inspiration: conjugate directions, homogeneous functions, variable-metric updates, and multi-dimensional searches. The computational process for solving small and medium-size constrained optimization problems is usually based on unconstrained optimization. This provides a straightforward opportunity for the introduction of parallelism. In the present survey, however, we focus on promising approaches for solving large, well-structured constrained problems: dualization of problems with separable objective and constraint functions, and decomposition of hierarchical problems with linking variables (typical for Bender's decomposition in the linear case). Finally, we outline the key issues in future computational studies of parallel nonlinear optimization algorithms.     

Multipoint cubic surrogate function for sequential approximate optimization

Multipoint cubic approximations are investigated as surrogate functions for nonlinear objective and constraint functions in the context of sequential approximate optimization. The proposed surrogate functions match actual function and gradient values, including the current expansion point, thus satisfying the zero and first-order necessary conditions for global convergence to a local minimum of the original problem. Function and gradient information accumulated from multiple design points during the iteration history is used in estimating a reduced Hessian matrix and selected cubic terms in a design subspace appropriate for problems with many design variables. The resulting approximate response surface promises to accelerate convergence to an optimal design within the framework of a trust region algorithm. The hope is to realize computational savings in solving large numerical optimization problems. Numerical examples demonstrate the effectiveness of the new multipoint surrogate function in reducing errors over large changes in design variables.     

A hybrid surrogate and pattern search optimization method and application to microelectronics

A hybrid method for robust and efficient optimization process is developed by integrating a new response surface method and pattern search algorithm. The method is based on: (1) multipoint approximations of the objective and constraint functions, (2) a multiquadric radial basis function (RBF) for the zeroth-order function approximation and a new RBF plus polynomial-based moving least-squares approximation for the first-order enhanced function approximation, and (3) a pattern search algorithm to impose a descent condition and applied adaptive subregion management strategy. Several numerical examples are presented to illustrate accuracy and computational efficiency of the proposed method for both function approximation and design optimization. To demonstrate the effectiveness of the proposed hybrid method, it is applied to obtain optimum designs of a microelectronic packaging system. A two-stage optimization approach is proposed for the design optimization. The material properties of microelectronic packaging system and the shape parameters of solder ball are selected as design variables. Through design optimization, significant improvements of durability performances are obtained using the proposed hybrid optimization method.     

求解优化问题的混合PSO-Solver算法

融合了粒子群算法(PSO) 和Solver 加载宏,形成混合PSO-Solver算法进行优化问题的求解。PSO作为全局搜索算法首先给出问题的全局可行解,Solver则是基于梯度信息的局部搜索工具,对粒子群算法得出的解再进行改进,二者互相结合,既加快了全局搜索的速度,又有效地避免了陷入局部最优。算法用VBA语言进行编程,简单且易于实现。通过对无约束优化问题和约束优化问题的求解,以及和标准PSO、其他一些混合算法的比较表明,PSO-Solver算法能够有效地提高求解过程的收敛速度和解的精确性。