A scheme for determining stepsizes for unconstrained optimization methods

We present a new scheme for determining stepsizes for iterative unconstrained minimization methods. This scheme provides a stepsize estimate for the efficient Armijo-type stepsize determination rule and improves its performance. As examples for the new scheme, we also present a new gradient algorithm and a new conjugate gradient algorithm. These two algorithms are readily implementable and eventually demand only one trial stepsize at each iteration. Their global convergence is established without any convexity assumptions. The convergence ratio associated with the gradient algorithm is shown to converge to the canonical convergence ratio (that is, the best possible convergence ratio). The convergence rate of the conjugate gradient algorithm is n-step superlinear and n-step quadratic.     

A class of nonmonotone stabilization trust region methods

A class of trust region methods in unconstrained optimization is presented, by adopting a nonmonotone stabilization strategy. Under some regularity conditions, the convergence properties of these methods are discussed. Extensive numerical results which are reported show that these methods are very efficient.     

Multi-fidelity wing aerostructural optimization using a trust region filter-SQP algorithm

A trust region filter-SQP method is used for wing multi-fidelity aerostructural optimization. Filter method eliminates the need for a penalty function, and subsequently a penalty parameter. Besides, it can easily be modified to be used for multi-fidelity optimization. A low fidelity aerostructural analysis tool is presented, that computes the drag, weight and structural deformation of lifting surfaces as well as their sensitivities with respect to the design variables using analytical methods. That tool is used for a mono-fidelity wing aerostructral optimization using a trust region filter-SQP method. In addition to that, a multi-fidelity aerostructural optimization has been performed, using a higher fidelity CFD code to calibrate the results of the lower fidelity model. In that case, the lower fidelity tool is used to compute the objective function, constraints and their derivatives to construct the quadratic programming subproblem. The high fidelity model is used to compute the objective function and the constraints used to generate the filter. The results of the high fidelity analysis are also used to calibrate the results of the lower fidelity tool during the optimization. This method is applied to optimize the wing of an A320 like aircraft for minimum fuel burn. The results showed about 9 % reduction in the aircraft mission fuel burn.     

Descent three-term conjugate gradient methods based on secant conditions for unconstrained optimization

The conjugate gradient method is an effective method for large-scale unconstrained optimization problems. Recent research has proposed conjugate gradient methods based on secant conditions to establish fast convergence of the methods. However, these methods do not always generate a descent search direction. In contrast, Y. Narushima, H. Yabe, and J.A. Ford [A three-term conjugate gradient method with sufficient descent property for unconstrained optimization, SIAM J. Optim. 21 (2011), pp. 212–230] proposed a three-term conjugate gradient method which always satisfies the sufficient descent condition. This paper makes use of both ideas to propose descent three-term conjugate gradient methods based on particular secant conditions, and then shows their global convergence properties. Finally, numerical results are given.     

Deep reinforcement learning methods for structure-guided processing path optimization

Journal of Intelligent Manufacturing - A major goal of materials design is to find material structures with desired properties and in a second step to find a processing path to reach one of these...     

Two modified spectral conjugate gradient methods and their global convergence for unconstrained optimization

In this paper, two modified spectral conjugate gradient methods which satisfy sufficient descent property are developed for unconstrained optimization problems. For uniformly convex problems, the first modified spectral type of conjugate gradient algorithm is proposed under the Wolfe line search rule. Moreover, the search direction of the modified spectral conjugate gradient method is sufficiently descent for uniformly convex functions. Furthermore, according to the Dai–Liao's conjugate condition, the second spectral type of conjugate gradient algorithm can generate some sufficient decent direction at each iteration for general functions. Therefore, the second method could be considered as a modification version of the Dai–Liao's algorithm. Under the suitable conditions, the proposed algorithms are globally convergent for uniformly convex functions and general functions. The numerical results show that the approaches presented in this paper are feasible and efficient.     

A modified trust region algorithm for nonlinear equations with new updating rule of trust region radius

In this article, we present a trust region algorithm for the nonlinear equations with a new updating rule of the trust region radius, which takes some function of the residual. We show that under the local error bound condition which is weaker than the non-singularity, the new algorithm converges quadratically to some solution of the nonlinear equations. Numerical results show that the new algorithm performs very well for some singular nonlinear equations.     

Convergence of trust region augmented Lagrangian methods using variable fidelity approximation data

To date the primary focus of most constrained approximate optimization strategies is that application of the method should lead to improved designs. Few researchers have focused on the development of constrained approximate optimization strategies that are assured of converging to a Karush-Kuhn-Tucker (KKT) point for the problem. Recent work by the authors based on a trust region model management strategy has shown promise in managing the convergence of constrained approximate optimization in application to a suite of single level optimization test problems. Using a trust-region model management strategy, coupled with an augmented Lagrangian approach for constrained approximate optimization, the authors have shown in application studies that the approximate optimization process converges to a KKT point for the problem. The approximate optimization strategy sequentially builds a cumulative response surface approximation of the augmented Lagrangian which is then optimized subject to a trust region constraint. In this research the authors develop a formal proof of convergence for the response surface approximation based optimization algorithm. Previous application studies were conducted on single level optimization problems for which response surface approximations were developed using conventional statistical response sampling techniques such as central composite design to query a high fidelity model over the design space. In this research the authors extend the scope of application studies to include the class of multidisciplinary design optimization (MDO) test problems. More importantly the authors show that response surface approximations constructed from variable fidelity data generated during concurrent subspace optimization (CSSOs) can be effectively managed by the trust region model management strategy. Results for two multidisciplinary test problems are presented in which convergence to a KKT point is observed. The formal proof of convergence and the successful MDO application of the algorithm using variable fidelity data generated by CSSO are original contributions to the growing body of research in MDO.Nomenclature k Lagrangian iteration - s approximate minimization iteration - i, j, l variable indices - m number of inequality constraints - n number of design variables - p number of equality constraints - f(x) objective function - g(x) inequality constraint vector - g _j (x) j-th inequality constraint - h(x) equality constraint vector - h _j (x) i-th equality constraint - c(x) generalized constraint vector - c _i (x) i-th generalized constraint - c ₁,c ₂,c ₃,c ₄ real constants - m(x) approximate model - q(x) approximate model - q(x) piecewise approximation - r _p penalty parameter - t, t ₁,t ₂ step size length - x design vector, dimensionn - x _l l-th design variable - x ^U upper bound vector, dimensionn - x _l ^U l-th design upper bound - x ^L lower bound vector, dimensionn - x _l ^L l-th design lower bound - B approximation of the Hessian - K constraints residual - S design space - , ₁, ₂, scalars - ₁, ₂ convergence tolerances - ₀, ₁, ₂, , trust region parameters - Lagrange multiplier vector, dimensionm+p - _i i-th Lagrange multiplier - trust region ratio - (x) alternative form for inequality constraints - (x, ,r _p ) augmented Lagrangian function - approximation of the augmented Lagrangian function - fidelity control - . Euclidean norm - , inner product - gradient operator with respect to design vector x - P(y(x)) projection operator; projects the vector y onto the set of feasible directions at x - trust region radius - x step size     

Dynamic analysis of box girders with tee-stiffening using unconstrained optimization techniques

The paper presents a numerical procedure for dynamic analysis of box girders with tee-stiffeners utilizing unconstrained optimization techniques. Unlike the finite element or finite strip methods, the procedure does not require discretization to the whole structure, thus resulting in great savings in computational time. The potential and kinetic energy of the assembled structure is expressed in terms generalized functions that describe the longitudinal and transverse displacement profiles. The problem is then converted into uunconstrained optimization problem to determine the magnitude of the lowest natural frequency and the associated mode shape. Results are presented showing the sensitivity the natural frequency to the stiffener depth (d) and the flange width (b). It is shown that the number of longitudinal and transverse stiffeners largely influence the magnitude of the natural frequency (λ) of the box girder. Design guidelines are also provided to optimize the dynamic response of the structure. The procedure is very practical and can be utilized in the industry for the analysis of box girders.     

An enhanced monarch butterfly optimization with self-adaptive crossover operator for unconstrained and constrained optimization problems

Natural Computing - Inspired by the phenomenon of migration of monarch butterflies, Wang et al. developed a novel promising swarm intelligence algorithm, called monarch butterfly optimization...     

A heuristic iterated-subspace minimization method with pattern search for unconstrained optimization

Recently, an increasing attention was paid on different procedures for an unconstrained optimization problem when the information of the first derivatives is unavailable or unreliable. In this paper, we consider a heuristic iterated-subspace minimization method with pattern search for solving such unconstrained optimization problems. The proposed method is designed to reduce the total number of function evaluations for the implementation of high-dimensional problems. Meanwhile, it keeps the advantages of general pattern search algorithm, i.e., the information of the derivatives is not needed. At each major iteration of such a method, a low-dimensional manifold, the iterated subspace, is constructed. And an approximate minimizer of the objective function in this manifold is then determined by a pattern search method. Numerical results on some classic test examples are given to show the efficiency of the proposed method in comparison with a conventional pattern search method and a derivative-free method.     

A hybrid shuffled complex evolution approach with pattern search for unconstrained optimization

The difficulties associated with using classical mathematical programming methods on complex optimization problems have contributed to the development of alternative and efficient numerical approaches. Recently, to overcome the limitations of classical optimization methods, researchers have proposed a wide variety of meta-heuristics for searching near-optimum solutions to problems. Among the existing meta-heuristic algorithms, a relatively new optimization paradigm is the Shuffled Complex Evolution at the University of Arizona (SCE-UA) which is a global optimization strategy that combines concepts of the competition evolution theory, downhill simplex procedure of Nelder-Mead, controlled random search and complex shuffling. In an attempt to reduce processing time and improve the quality of solutions, particularly to avoid being trapped in local optima, in this paper is proposed a hybrid SCE-UA approach. The proposed hybrid algorithm is the combination of SCE-UA (without Nelder-Mead downhill simplex procedure) and a pattern search approach, called SCE-PS, for unconstrained optimization. Pattern search methods are derivative-free, meaning that they do not use explicit or approximate derivatives. Moreover, pattern search algorithms are direct search methods well suitable for the global optimization of highly nonlinear, multiparameter, and multimodal objective functions. The proposed SCE-PS method is tested with six benchmark optimization problems. Simulation results show that the proposed SCE-PS improves the searching performance when compared with the classical SCE-UA and a genetic algorithm with floating-point representation for all the tested problems. As evidenced by the performance indices based on the mean performance of objective function in 30 runs and mean of computational time, the SCE-PS algorithm has demonstrated to be effective and efficient at locating best-practice optimal solutions for unconstrained optimization.     

Multiobjective optimization of laminated composite parts with curvilinear fibers using Kriging-based approaches

This paper describes the multiobjective optimization of parts made with curvilinear fiber composites. Two structures are studied: a square plate and a fuselage-like section. The square plate is designed in two ways. First, classical lamination theory (CLT) is used to obtain the structural response for a plate with straight fibers designed for maximum buckling load and maximum stiffness. The same plate is then designed with curved fibers using finite element analysis (FEA) to determine the structural response. Next, the fuselage-like section is designed using the same FEA approach. The problems have three to twelve variables. To enable the resulting Pareto front to be visualized more clearly, only two objectives are considered. The first two optimization problems are unconstrained, while the last one is constrained by two project requirements. To overcome the problem of long computational run time when using FEA, Kriging-based approaches are used. Three such approaches suitable for multiobjective problems are compared: (i) the efficient global optimization algorithm (EGO) is applied to a single-objective function consisting of a weighted combination of the objectives, (ii) a technique that involves sequential maximization of the expected hypervolume improvement, and (iii) a novel approach proposed here based on sequential minimization of the variance of the predicted Pareto front. Comparison of the results using the inverted generational distance (IGD) metric revealed that the approach (iii) had the best performance (mean) and best robustness (standard deviation) for all the cases studied.     

Water cycle algorithm with evaporation rate for solving constrained and unconstrained optimization problems

This paper presents a modified version of the water cycle algorithm (WCA). The fundamental concepts and ideas which underlie the WCA are inspired based on the observation of water cycle process and how rivers and streams flow to the sea. New concept of evaporation rate for different rivers and streams is defined so called evaporation rate based WCA (ER-WCA), which offers improvement in search. Furthermore, the evaporation condition is also applied for streams that directly flow to sea based on the new approach. The ER-WCA shows a better balance between exploration and exploitation phases compared to the standard WCA. It is shown that the ER-WCA offers high potential in finding all global optima of multimodal and benchmark functions. The WCA and ER-WCA are tested using several multimodal benchmark functions and the obtained optimization results show that in most cases the ER-WCA converges to the global solution faster and offers more accurate results than the WCA and other considered optimizers. Based on the performance of ER-WCA on a number of well-known benchmark functions, the efficiency of the proposed method with respect to the number of function evaluations (computational effort) and accuracy of function value are represented.     

Data mining with various optimization methods

Road traffic represents the main source of noise in urban environments that is proven to significantly affect human mental and physical health and labour productivity. Thus, in order to control noise sound level in urban areas, it is very important to develop methods for modelling the road traffic noise. As observed in the literature, the models that deal with this issue are mainly based on regression analysis, while other approaches are very rare. In this paper a novel approach for modelling traffic noise that is based on optimization is presented. Four optimization techniques were used in simulation in this work: genetic algorithms, Hooke and Jeeves algorithm, simulated annealing and particle swarm optimization. Two different scenarios are presented in this paper. In the first scenario the optimization methods use the whole measurement dataset to find the most suitable parameters, whereas in the second scenario optimized parameters were found using only some of the measurement data, while the rest of the data was used to evaluate the predictive capabilities of the model. The goodness of the model is evaluated by the coefficient of determination and other statistical parameters, and results show agreement of high extent between measured data and calculated values in both scenarios. In addition, the model was compared with classical statistical model, and superior capabilities of proposed model were demonstrated. The simulations were done using the originally developed user friendly software package.     

Experience with approximate reliability-based optimization methods

Traditional reliability-based design optimization (RBDO) requires a double loop iteration process. The inner optimization loop is to find the most probable point (MPP) and the outer is the regular optimization loop to optimize the RBDO problem with reliability objectives or constraints. It is well known that the computation can be prohibitive when the associated function evaluation is expensive. As a result, many approximate RBDO methods, which convert the double loop to a single loop, have been developed. In this work, several approximate RBDO methods are coded, discussed, and tested against a double loop algorithm through four design problems.     

PSO with path relinking for resource allocation using simulation optimization

This paper proposes a PSO-based optimization approach with a particular path relinking technique for moving particles. PSO is evaluated for two combinatorial problems. One under uncertainty, which represents a new application of PSO with path relinking in a stochastic scenario. PSO is considered first in a deterministic scenario for solving the Task Assignment Problem (TAP) and hereafter for a resource allocation problem in a petroleum terminal. This is considered for evaluating PSO in a problem subject to uncertainty whose performance can only be evaluated by simulation. In this case, a discrete event simulation is built for modeling a real-world facility whose typical operations of receiving and transferring oil from tankers to a refinery are made through intermediary storage tanks. The simulation incorporates uncertain data and operational details for optimization that are not considered in other mathematical optimization models. Experiments have been carried out considering issues that affect the choice of parameters for both optimization and simulation. The results show advantages of the proposed approach when compared with Genetic Algorithm and OptQuest (a commercial optimization package).