Structural optimization using augmented Lagrangian methods with secant Hessian updating

The performance of a new implementation of the augmented Lagrangian method is evaluated on a range of explicit and structural sizing optimization problems. The results are compared with those obtained using other mathematical programming methods. The implementation uses a first-order Lagrange multiplier update and the Hessian of the augmented Lagrangian function is approximated using partitioned secant updating. A number of different secant updates are evaluated. The results show the formulation to be superior to other implementations of augmented Lagrangian methods reported in the literature and that, under certain conditions, the method approaches the performance of the state-of-the-art SQP and SAM methods. Of the secant updates, the symmetric-rank-one update, is superior to the other updates including the BFGS scheme. It is suggested that the individual function, secant updating employed may be usefully applied in contexts where structural analysis and optimization are performed simultaneously, as in the simultaneous analysis and design method. In such cases the functions are partially separable and the associated Hessians are of low rank.     

Adaptive augmented Lagrangian methods: algorithms and practical numerical experience

In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis et al. [An adaptive augmented Lagrangian method for large-scale constrained optimization, Math. Program. 152 (2015), pp. 201–245.]. The first focal point of this paper is a new variant of the approach that employs a line search rather than a trust region strategy, where a critical algorithmic feature for the line search strategy is the use of convexified piecewise quadratic models of the AL function for computing the search directions. We prove global convergence guarantees for our line search algorithm that are on par with those for the previously proposed trust region method. A second focal point of this paper is the practical performance of the line search and trust region algorithm variants in Matlab software, as well as that of an adaptive penalty parameter updating strategy incorporated into the Lancelot software. We test these methods on problems from the CUTEst and COPS collections, as well as on challenging test problems related to optimal power flow. Our numerical experience suggests that the adaptive algorithms outperform traditional AL methods in terms of efficiency and reliability. As with traditional AL algorithms, the adaptive methods are matrix-free and thus represent a viable option for solving large-scale problems.     

Convergence of trust region augmented Lagrangian methods using variable fidelity approximation data

To date the primary focus of most constrained approximate optimization strategies is that application of the method should lead to improved designs. Few researchers have focused on the development of constrained approximate optimization strategies that are assured of converging to a Karush-Kuhn-Tucker (KKT) point for the problem. Recent work by the authors based on a trust region model management strategy has shown promise in managing the convergence of constrained approximate optimization in application to a suite of single level optimization test problems. Using a trust-region model management strategy, coupled with an augmented Lagrangian approach for constrained approximate optimization, the authors have shown in application studies that the approximate optimization process converges to a KKT point for the problem. The approximate optimization strategy sequentially builds a cumulative response surface approximation of the augmented Lagrangian which is then optimized subject to a trust region constraint. In this research the authors develop a formal proof of convergence for the response surface approximation based optimization algorithm. Previous application studies were conducted on single level optimization problems for which response surface approximations were developed using conventional statistical response sampling techniques such as central composite design to query a high fidelity model over the design space. In this research the authors extend the scope of application studies to include the class of multidisciplinary design optimization (MDO) test problems. More importantly the authors show that response surface approximations constructed from variable fidelity data generated during concurrent subspace optimization (CSSOs) can be effectively managed by the trust region model management strategy. Results for two multidisciplinary test problems are presented in which convergence to a KKT point is observed. The formal proof of convergence and the successful MDO application of the algorithm using variable fidelity data generated by CSSO are original contributions to the growing body of research in MDO.Nomenclature k Lagrangian iteration - s approximate minimization iteration - i, j, l variable indices - m number of inequality constraints - n number of design variables - p number of equality constraints - f(x) objective function - g(x) inequality constraint vector - g _j (x) j-th inequality constraint - h(x) equality constraint vector - h _j (x) i-th equality constraint - c(x) generalized constraint vector - c _i (x) i-th generalized constraint - c ₁,c ₂,c ₃,c ₄ real constants - m(x) approximate model - q(x) approximate model - q(x) piecewise approximation - r _p penalty parameter - t, t ₁,t ₂ step size length - x design vector, dimensionn - x _l l-th design variable - x ^U upper bound vector, dimensionn - x _l ^U l-th design upper bound - x ^L lower bound vector, dimensionn - x _l ^L l-th design lower bound - B approximation of the Hessian - K constraints residual - S design space - , ₁, ₂, scalars - ₁, ₂ convergence tolerances - ₀, ₁, ₂, , trust region parameters - Lagrange multiplier vector, dimensionm+p - _i i-th Lagrange multiplier - trust region ratio - (x) alternative form for inequality constraints - (x, ,r _p ) augmented Lagrangian function - approximation of the augmented Lagrangian function - fidelity control - . Euclidean norm - , inner product - gradient operator with respect to design vector x - P(y(x)) projection operator; projects the vector y onto the set of feasible directions at x - trust region radius - x step size     

An augmented Lagrangian method for VLSI global placement

By ignoring some cell overlaps, global placement computes the best position for each cell to minimize the wirelength. It is an important stage in very large scale integration (VLSI) physical design, since circuit performance heavily depends on the placement results. In this paper, we propose an augmented Lagrangian method to solve the VLSI global placement problem. In the proposed method, a cautious dynamic density weight strategy is used to balance the wirelength objective and the density constraints, and an adaptive step size is used to obtain a trade-off between runtime and solution quality. The proposed method is tested on the IBM mixed-size benchmarks and the International Symposium on Physical Design 2006 placement contest benchmarks. Experimental results show that our global placement method outperforms the state-of-the-art placement approaches in terms of solution quality on most of the benchmarks.     

An efficient augmented Lagrangian method for support vector machine

ABSTRACT

Support vector machine (SVM) has proved to be a successful approach for machine learning. Two typical SVM models are the L1-loss model for support vector classification (SVC) and ε-L1-loss model for support vector regression (SVR). Due to the non-smoothness of the L1-loss function in the two models, most of the traditional approaches focus on solving the dual problem. In this paper, we propose an augmented Lagrangian method for the L1-loss model, which is designed to solve the primal problem. By tackling the non-smooth term in the model with Moreau–Yosida regularization and the proximal operator, the subproblem in augmented Lagrangian method reduces to a non-smooth linear system, which can be solved via the quadratically convergent semismooth Newton's method. Moreover, the high computational cost in semismooth Newton's method can be significantly reduced by exploring the sparse structure in the generalized Jacobian. Numerical results on various datasets in LIBLINEAR show that the proposed method is competitive with the most popular solvers in both speed and accuracy.     

A separable augmented Lagrangian algorithm for optimal structural design

Structural and Multidisciplinary Optimization - We propose an iterative separable augmented Lagrangian algorithm (SALA) for optimal structural design, with SALA being a subset of the alternating...     

An adaptative augmented Lagrangian method for three-dimensional multimaterial flows

The direct numerical simulation of incompressible multimaterial flows, based on predictor/corrector and volume of fluid (VOF) approaches is presented. An original adaptative augmented Lagrangian method is proposed to solve the predictor solution, satisfying at the same time the conservation equations as well as the incompressibility constraint. This algorithm is based on an Uzawa optimisation technique. The corrector solution is obtained with a projection method on a divergence free subspace. Several examples of two- and three-dimensional flows are proposed to illustrate the ability of the method to deal with unsteady, multimaterial problems.     

Multi-modality in augmented Lagrangian coordination for distributed optimal design

This paper presents an empirical study of the convergence characteristics of augmented Lagrangian coordination (ALC) for solving multi-modal optimization problems in a distributed fashion. A number of test problems that do not satisfy all assumptions of the convergence proof for ALC are selected to demonstrate the convergence characteristics of ALC algorithms. When only a local search is employed at the subproblems, local solutions to the original problem are often attained. When a global search is performed at subproblems, global solutions to the original, non-decomposed problem are found for many of the examples. Although these findings are promising, ALC with a global subproblem search may yield only local solutions in the case of non-convex coupling functions or disconnected feasible domains. Results indicate that for these examples both the starting point and the sequence in which subproblems are solved determines which solution is obtained. We illustrate that the main cause for this behavior lies in the alternating minimization inner loop, which is inherently of a local nature.     

Block-separable linking constraints in augmented Lagrangian coordination

Augmented Lagrangian coordination (ALC) is a provably convergent coordination method for multidisciplinary design optimization (MDO) that is able to treat both linking variables and linking functions (i.e. system-wide objectives and constraints). Contrary to quasi-separable problems with only linking variables, the presence of linking functions may hinder the parallel solution of subproblems and the use of the efficient alternating directions method of multipliers. We show that this unfortunate situation is not the case for MDO problems with block-separable linking constraints. We derive a centralized formulation of ALC for block-separable constraints, which does allow parallel solution of subproblems. Similarly, we derive a distributed coordination variant for which subproblems cannot be solved in parallel, but that still enables the use of the alternating direction method of multipliers. The approach can also be used for other existing MDO coordination strategies such that they can include block-separable linking constraints.     

An augmented Lagrangian decomposition method for quasi-separable problems in MDO

Several decomposition methods have been proposed for the distributed optimal design of quasi-separable problems encountered in Multidisciplinary Design Optimization (MDO). Some of these methods are known to have numerical convergence difficulties that can be explained theoretically. We propose a new decomposition algorithm for quasi-separable MDO problems. In particular, we propose a decomposed problem formulation based on the augmented Lagrangian penalty function and the block coordinate descent algorithm. The proposed solution algorithm consists of inner and outer loops. In the outer loop, the augmented Lagrangian penalty parameters are updated. In the inner loop, our method alternates between solving an optimization master problem and solving disciplinary optimization subproblems. The coordinating master problem can be solved analytically; the disciplinary subproblems can be solved using commonly available gradient-based optimization algorithms. The augmented Lagrangian decomposition method is derived such that existing proofs can be used to show convergence of the decomposition algorithm to Karush–Kuhn–Tucker points of the original problem under mild assumptions. We investigate the numerical performance of the proposed method on two example problems.     

An augmented Lagrangian optimization method for inflatable structures analysis problems

This paper describes the development of an augmented Lagrangian optimization method for the numerical simulation of the inflation process in the design of inflatable space structures. Although the Newton–Raphson scheme was proven to be efficient for solving many nonlinear problems, it can lead to lack of convergence when it is applied to the simulation of the inflation process. As a result, it is recommended to use an optimization algorithm to find the minimum energy configuration that satisfies the equilibrium equations characterizing the final shape of the inflated structure subject to an internal pressure. On top of that, given that some degrees of freedom may be linked, the optimum may be constrained, and specific optimization methods for constrained problems must be considered. The paper presents the formulation and the augmented Lagrangian method (ALM) developed in SAMCEF Mecano for inflatable structures analysis problems. The related quasi-unconstrained optimization problem is solved with a nonlinear conjugate gradient method. The Wolfe conditions are used in conjunction with a cubic interpolation for the line search. Equality constraints are considered and can be easily treated by the ALM formulation. Numerical applications present simulations of unconstrained and constrained inflation processes (i.e., where the motion of some nodes is ruled by a rigid body element restriction and/or problems including contact conditions).Part of this paper was presented at the sixth world congress of Structural and Multidisciplinary Optimization held in Rio de Janeiro, June 2005.     

Using augmented Lagrangian particle swarm optimization for constrained problems in engineering">Using augmented Lagrangian particle swarm optimization for constrained problems in engineering

The comparatively new stochastic method of particle swarm optimization (PSO) has been applied to engineering problems especially of nonlinear, non-differentiable, or non-convex type. Its robustness and its simple applicability without the need for cumbersome derivative calculations make PSO an attractive optimization method. However, engineering optimization tasks often consist of problem immanent equality and inequality constraints which are usually included by inadequate penalty functions when using stochastic algorithms. The simple structure of basic particle swarm optimization characterized by only a few lines of computer code allows an efficient implementation of a more sophisticated treatment of such constraints. In this paper, we present an approach which utilizes the simple structure of the basic PSO technique and combines it with an extended non-stationary penalty function approach, called augmented Lagrange multiplier method, for constraint handling where ill conditioning is a far less harmful problem and the correct solution can be obtained even for finite penalty factors. We describe the basic PSO algorithm and the resulting method for constrained problems as well as the results from benchmark tests. An example of a stiffness optimization of an industrial hexapod robot with parallel kinematics concludes this paper and shows the applicability of the proposed augmented Lagrange particle swarm optimization to engineering problems.     

Sensitivity analysis for frictional contact problems in the augmented Lagrangian formulation

Direct differentiation method of sensitivity analysis is developed for frictional contact problems. As a result of the augmented Lagrangian treatment of contact constraints, the direct problem is solved simultaneously for the displacements and Lagrange multipliers using the Newton method. The main purpose of the paper is to show that this formulation of the augmented Lagrangian method is particularly suitable for sensitivity analysis because the direct differentiation method leads to a non-iterative exact sensitivity problem to be solved at each time increment. The approach is applied to a general class of three-dimensional frictional contact problems, and numerical examples are provided involving large deformations, multibody contact interactions, and contact smoothing techniques.     

ALGAMES: a fast augmented Lagrangian solver for constrained dynamic games

Autonomous Robots - Dynamic games are an effective paradigm for dealing with the control of multiple interacting actors. This paper introduces augmented Lagrangian GAME-theoretic solver (ALGAMES),...     

Multilevel preconditioned augmented Lagrangian techniques for 2nd order mixed problems

We are concerned with the efficient solution of saddle point problems arising from the mixed discretization of 2nd order elliptic problems in two dimensions. We consider the mixed discretization of the boundary value problem by means of lowest order Raviart-Thomas elements. This leads to a saddle point problem, which can be tackled by Uzawa-like iterative solvers. We suggest a prior modification of the saddle point problem according to the augmented Lagrangian approach (cf. [16]) in order to make it more amenable to the iterative procedure. In order to boost the speed of iterative methods, we additionally employ a multilevel preconditioner first presented by Vassilevski and Wang in [26]. It is based on a special splitting of the space of vector valued fluxes, which exploits the close relationship between piecewise linear continuous finite element functions and divergence free fluxes. We prove that this splitting gives rise to an optimal preconditioner: it achieves condition numbers bounded independently on the depth of refinement. The proof is set in the framework of Schwarz methods (cf. [28, 30]). It relies on established results about standard multilevel methods as well as a strengthened Cauchy-Schwarz inequality forRT ₀-spaces.     

Combination of augmented Lagrangian technique and ST preconditioner for saddle point problems

For symmetric indefinite linear systems, we introduce a new triangular preconditioner based on symmetric and triangular (ST) decomposition. A new (1, 1) block is obtained by augmented Lagrangian technique. The new ST preconditioner is introduced by the combination of the new (1, 1) block and symmetric and triangular (ST) decomposition. Then a preconditioned system can be obtained by preconditioning technique, which is superior to the original system in terms of condition number. We study the spectral properties of preconditioned system, such as eigenvalues, the estimation of condition number and then give the quasi-optimal parameter. Numerical examples are given to indicate that the new preconditioner has obvious efficiency advantages. Finally, we conclude that the new ST preconditioner is a better option to deal with large and sparse problems.     

A hybrid differential evolution augmented Lagrangian method for constrained numerical and engineering optimization

We present a new hybrid method for solving constrained numerical and engineering optimization problems in this paper. The proposed hybrid method takes advantage of the differential evolution (DE) ability to find global optimum in problems with complex design spaces while directly enforcing feasibility of constraints using a modified augmented Lagrangian multiplier method. The basic steps of the proposed method are comprised of an outer iteration, in which the Lagrangian multipliers and various penalty parameters are updated using a first-order update scheme, and an inner iteration, in which a nonlinear optimization of the modified augmented Lagrangian function with simple bound constraints is implemented by a modified differential evolution algorithm. Experimental results based on several well-known constrained numerical and engineering optimization problems demonstrate that the proposed method shows better performance in comparison to the state-of-the-art algorithms.     

Simultaneous structural analysis and design based on augmented Lagrangian duality

Solution procedures in structural optimization are commonly based on a nested approach where approximations of the analysis and design problems are solved alternately in an iterative scheme. In this paper, we study a simultaneous approach based on an integrated formulation of the analysis and design problems. An advantage of the simultaneous approach, when compared to the nested one, is that the dependence between the analysis and design variables is imposed explicitly. In the nested approach, this dependence is implicitly determined through the solution of the analysis problem. Earlier simultaneous approaches mostly utilize various penalty function reformulations. In this paper, we make use of two augmented Lagrangian schemes, which avoid the numerical ill-conditioning inherent in penalty reformulations. These schemes give rise to Lagrangian subproblems with somewhat different properties, and two efficient techniques are adapted for their solution. The first is a projected Newton method, and the second is a simplicial decomposition scheme. Computational results for bar-truss structures show that the proposed schemes are viable approaches for solving the integrated formulation, and that they are promising for future developments.     

Comparing augmented reality visualization methods for assembly procedures

Virtual Reality - Assembly processes require now more than ever a systematic way to improve efficiency complying with increasing product demand. Several industrial scenarios have been using...