A new recurslve method for the analysis of linear time invariant dynamic systems via double-term triangular functions （DTTF） in state space environment

This paper presents a new recursive method for system analysis via double-term triangular functions (DTTF) in state space environment. The proposed method uses orthogonal triangular function sets and proves to be more accurate as compared to single term Walsh series (STWS) method with respect to mean integral square error (MISE). This has been established theoretically and comparison of error with respect to MISE is presented for clarity. A numerical example is treated to establish the proposed method. Relevant curves for the solutions of states of the dynamic system are also presented with plots of percentage error for DTTF-based analysis.     

A hybrid differential evolution augmented Lagrangian method for constrained numerical and engineering optimization

We present a new hybrid method for solving constrained numerical and engineering optimization problems in this paper. The proposed hybrid method takes advantage of the differential evolution (DE) ability to find global optimum in problems with complex design spaces while directly enforcing feasibility of constraints using a modified augmented Lagrangian multiplier method. The basic steps of the proposed method are comprised of an outer iteration, in which the Lagrangian multipliers and various penalty parameters are updated using a first-order update scheme, and an inner iteration, in which a nonlinear optimization of the modified augmented Lagrangian function with simple bound constraints is implemented by a modified differential evolution algorithm. Experimental results based on several well-known constrained numerical and engineering optimization problems demonstrate that the proposed method shows better performance in comparison to the state-of-the-art algorithms.     

An augmented Lagrangian optimization method for inflatable structures analysis problems

This paper describes the development of an augmented Lagrangian optimization method for the numerical simulation of the inflation process in the design of inflatable space structures. Although the Newton–Raphson scheme was proven to be efficient for solving many nonlinear problems, it can lead to lack of convergence when it is applied to the simulation of the inflation process. As a result, it is recommended to use an optimization algorithm to find the minimum energy configuration that satisfies the equilibrium equations characterizing the final shape of the inflated structure subject to an internal pressure. On top of that, given that some degrees of freedom may be linked, the optimum may be constrained, and specific optimization methods for constrained problems must be considered. The paper presents the formulation and the augmented Lagrangian method (ALM) developed in SAMCEF Mecano for inflatable structures analysis problems. The related quasi-unconstrained optimization problem is solved with a nonlinear conjugate gradient method. The Wolfe conditions are used in conjunction with a cubic interpolation for the line search. Equality constraints are considered and can be easily treated by the ALM formulation. Numerical applications present simulations of unconstrained and constrained inflation processes (i.e., where the motion of some nodes is ruled by a rigid body element restriction and/or problems including contact conditions).Part of this paper was presented at the sixth world congress of Structural and Multidisciplinary Optimization held in Rio de Janeiro, June 2005.     

Using augmented Lagrangian particle swarm optimization for constrained problems in engineering">Using augmented Lagrangian particle swarm optimization for constrained problems in engineering

The comparatively new stochastic method of particle swarm optimization (PSO) has been applied to engineering problems especially of nonlinear, non-differentiable, or non-convex type. Its robustness and its simple applicability without the need for cumbersome derivative calculations make PSO an attractive optimization method. However, engineering optimization tasks often consist of problem immanent equality and inequality constraints which are usually included by inadequate penalty functions when using stochastic algorithms. The simple structure of basic particle swarm optimization characterized by only a few lines of computer code allows an efficient implementation of a more sophisticated treatment of such constraints. In this paper, we present an approach which utilizes the simple structure of the basic PSO technique and combines it with an extended non-stationary penalty function approach, called augmented Lagrange multiplier method, for constraint handling where ill conditioning is a far less harmful problem and the correct solution can be obtained even for finite penalty factors. We describe the basic PSO algorithm and the resulting method for constrained problems as well as the results from benchmark tests. An example of a stiffness optimization of an industrial hexapod robot with parallel kinematics concludes this paper and shows the applicability of the proposed augmented Lagrange particle swarm optimization to engineering problems.     

An augmented Lagrangian decomposition method for quasi-separable problems in MDO

Several decomposition methods have been proposed for the distributed optimal design of quasi-separable problems encountered in Multidisciplinary Design Optimization (MDO). Some of these methods are known to have numerical convergence difficulties that can be explained theoretically. We propose a new decomposition algorithm for quasi-separable MDO problems. In particular, we propose a decomposed problem formulation based on the augmented Lagrangian penalty function and the block coordinate descent algorithm. The proposed solution algorithm consists of inner and outer loops. In the outer loop, the augmented Lagrangian penalty parameters are updated. In the inner loop, our method alternates between solving an optimization master problem and solving disciplinary optimization subproblems. The coordinating master problem can be solved analytically; the disciplinary subproblems can be solved using commonly available gradient-based optimization algorithms. The augmented Lagrangian decomposition method is derived such that existing proofs can be used to show convergence of the decomposition algorithm to Karush–Kuhn–Tucker points of the original problem under mild assumptions. We investigate the numerical performance of the proposed method on two example problems.     

A sharp augmented Lagrangian-based method in constrained non-convex optimization

In this paper, a novel sharp Augmented Lagrangian-based global optimization method is developed for solving constrained non-convex optimization problems. The algorithm consists of outer and inner loops. At each inner iteration, the discrete gradient method is applied to minimize the sharp augmented Lagrangian function. Depending on the solution found the algorithm stops or updates the dual variables in the inner loop, or updates the upper or lower bounds by going to the outer loop. The convergence results for the proposed method are presented. The performance of the method is demonstrated using a wide range of nonlinear smooth and non-smooth constrained optimization test problems from the literature.     

An augmented Lagrangian relaxation for analytical target cascading using the alternating direction method of multipliers

Analytical target cascading is a method for design optimization of hierarchical, multilevel systems. A quadratic penalty relaxation of the system consistency constraints is used to ensure subproblem feasibility. A typical nested solution strategy consists of inner and outer loops. In the inner loop, the coupled subproblems are solved iteratively with fixed penalty weights. After convergence of the inner loop, the outer loop updates the penalty weights. The article presents an augmented Lagrangian relaxation that reduces the computational cost associated with ill-conditioning of subproblems in the inner loop. The alternating direction method of multipliers is used to update penalty parameters after a single inner loop iteration, so that subproblems need to be solved only once. Experiments with four examples show that computational costs are decreased by orders of magnitude ranging between 10 and 1000.     

An augmented Lagrangian optimization method for contact analysis problems, 2: numerical evaluation

The ALM2 solution procedure is evaluated by solving two simple contact analysis problems for different friction conditions. These example problems are devised to have closed form solutions. This way there is no uncertainty about the target solution for evaluation of the proposed algorithm as well as existing algorithms. The numerical results with ALM2 are compared with the analytical solutions as well as with the penalty, Lagrange multiplier and existing augmented Lagrangian methods. All the algorithms are analysed for stick and slip friction conditions. The example problems are used to show clearly the dependence of the existing solution methods on the number of load steps and penalty values. It is concluded that convergence of incremental solution schemes employed in these methods does not guarantee accuracy of the contact solution even with the use of solution enhancement schemes such as automatic load stepping and contact load prediction. The example problems are also used to demonstrate solution independence of the proposed ALM2 procedure from penalty values, and from the number of load steps. The proposed formulation for calculation of frictional forces and the ALM solution algorithm have worked quite well for the example problems. However, the algorithm needs to be developed and evaluated for more complex contact analysis problems.     

An augmented Lagrangian optimization method for contact analysis problems, 1: formulation and algorithm

A review of existing augmented Lagrangian methods (ALM) for contact analysis problems reveals that they have not been implemented with automatic penalty updates as intended in their original development. Therefore, although the methods are an improvement over the penalty methods, solution with them still depends on the user-specified penalty values for the contact constraints. To overcome this drawback, an ALM is developed and discussed for contact analysis problems that automatically update the user-specified penalty values to obtain the final appropriate values. Further, to solve the frictional contact analysis problem accurately, a two-phase formulation is proposed. Solution of the Phase 1 problem removes penetration of the contacting nodes and brings them exactly to their initial contact points. In addition, a new contact constraint is introduced which allows determination of the precise friction force at the contacting nodes. Phase 2 of the formulation checks the friction conditions and solves the friction problem to bring the structure to an equilibrium state. Phases 1 and 2 are then combined to provide a general algorithm for multi-node frictional contact problems. The two-phase procedure also removes dependence of the contact solution on the number of load steps for the elastostatic problem. Numerical evaluation of the formulation and the algorithm is presented in Part 2 of the paper.     

Splitting augmented Lagrangian method for optimization problems with a cardinality constraint and semicontinuous variables

We propose a new splitting augmented Lagrangian method (SALM) for solving a class of optimization problems with both cardinality constraint and semicontinuous variables constraint. The proposed approach, inspired by the penalty decomposition method in [Z.S. Lu and Y. Zhang, Sparse approximation via penalty decomposition methods, SIAM J. Optim. 23(4) (2013), pp. 2448–2478], splits the problem into two subproblems using auxiliary variables. SALM solves two subproblems alternatively. Furthermore, we prove the convergence of SALM, under certain assumptions. Finally, SALM is implemented on the portfolio selection problem and the compressed sensing problem, respectively. Numerical results show that SALM outperforms the well-known tailored approach in CPLEX 12.6 and the penalty decomposition method, respectively.     

An Alternating Augmented Lagrangian method for constrained nonconvex optimization

ABSTRACT

We consider the problem of minimizing a smooth nonconvex function over a structured convex feasible set, that is, defined by two sets of constraints that are easy to treat when considered separately. In order to exploit the structure of the problem, we define an equivalent formulation by duplicating the variables and we consider the augmented Lagrangian of this latter formulation. Following the idea of the Alternating Direction Method of Multipliers (ADMM), we propose an algorithm where a two-blocks decomposition method is embedded within an augmented Lagrangian framework. The peculiarities of the proposed algorithm are the following: (1) the computation of the exact solution of a possibly nonconvex subproblem is not required; (2) the penalty parameter is iteratively updated once an approximated stationary point of the augmented Lagrangian is determined. Global convergence results are stated under mild assumptions and without requiring convexity of the objective function. Although the primary aim of the paper is theoretical, we perform numerical experiments on a nonconvex problem arising in machine learning, and the obtained results show the practical advantages of the proposed approach with respect to classical ADMM.