The continuous direct-adjoint approach for second order sensitivities in viscous aerodynamic inverse design problems

A novel continuous adjoint approach for the computation of the second order sensitivities of the objective function used in inverse design problems is proposed. In the framework of the Newton method, the proposed approach can be used to efficiently cope with inverse design problems in viscous flows, where the target is a given pressure distribution along the solid walls. It consists of two steps and will, thus, be referred to as the direct-adjoint approach. At the first step, the direct differentiation method is used to compute the first order sensitivities of the flow variables with respect to the design variables and build the gradient of the objective function. At the second step, the adjoint approach is used to compute the second order sensitivities. The final Hessian expression is free of field integrals and its computation requires the solution of N + 1 equivalent flow (system) solutions for N design variables. Since the CPU cost of using the Newton method, with exact gradient and Hessian data at each cycle, becomes prohibitively high, an approach that computes the exact Hessian only once and then updates it in an approximated manner through the BFGS formula, is used instead. The accuracy of the Hessian matrix components, computed using the direct-adjoint approach is demonstrated on the inverse design of a diffuser and a cascade airfoil.     

Aerodynamic Shape Optimization Using First and Second Order Adjoint and Direct Approaches

This paper focuses on discrete and continuous adjoint approaches and direct differentiation methods that can efficiently be used in aerodynamic shape optimization problems. The advantage of the adjoint approach is the computation of the gradient of the objective function at cost which does not depend upon the number of design variables. An extra advantage of the formulation presented below, for the computation of either first or second order sensitivities, is that the resulting sensitivity expressions are free of field integrals even if the objective function is a field integral. This is demonstrated using three possible objective functions for use in internal aerodynamic problems; the first objective is for inverse design problems where a target pressure distribution along the solid walls must be reproduced; the other two quantify viscous losses in duct or cascade flows, cast as either the reduction in total pressure between the inlet and outlet or the field integral of entropy generation. From the mathematical point of view, the three functions are defined over different parts of the domain or its boundaries, and this strongly affects the adjoint formulation. In the second part of this paper, the same discrete and continuous adjoint formulations are combined with direct differentiation methods to compute the Hessian matrix of the objective function. Although the direct differentiation for the computation of the gradient is time consuming, it may support the adjoint method to calculate the exact Hessian matrix components with the minimum CPU cost. Since, however, the CPU cost is proportional to the number of design variables, a well performing optimization scheme, based on the exactly computed Hessian during the starting cycle and a quasi Newton (BFGS) scheme during the next cycles, is proposed.     

Computation of the Hessian matrix in aerodynamic inverse design using continuous adjoint formulations

Continuous adjoint formulations for the computation of (first and) second order derivatives of the objective function governing inverse design problems in 2D inviscid flows are presented. These are prerequisites for the use of the very efficient exact Newton method. Four new formulations based on all possible combinations of the direct differentiation method and the continuous adjoint approach to compute the sensitivity derivatives of objective functions, constrained by the flow equations, are presented. They are compared in terms of the expected CPU cost to compute the Hessian of the objective function used in single-objective optimization problems with N degrees of freedom. The less costly among them was selected for further study and tested in inverse design problems solved by means of the Newton method. The selected approach, which will be referred to as the direct-adjoint one, since it performs direct differentiation for the gradient and, then, uses the adjoint approach to compute the Hessian, requires as many as N+2 equivalent flow solutions for each Newton step. The major part of the CPU cost (N equivalent flow solutions) is for the computation of the gradient but, fortunately, this task is directly amenable to parallelization. The method is used to reconstruct ducts or cascade airfoils for a known pressure distribution along their solid boundaries, at inviscid flow conditions. The examined cases aim at demonstrating the accuracy of the proposed method in computing the exact Hessian matrix as well as the efficiency of the exact Newton method as an optimization tool in aerodynamic design.     

First-order sequential convex programming using approximate diagonal QP subproblems

Optimization algorithms based on convex separable approximations for optimal structural design often use reciprocal-like approximations in a dual setting; CONLIN and the method of moving asymptotes (MMA) are well-known examples of such sequential convex programming (SCP) algorithms. We have previously demonstrated that replacement of these nonlinear (reciprocal) approximations by their own second order Taylor series expansion provides a powerful new algorithmic option within the SCP class of algorithms. This note shows that the quadratic treatment of the original nonlinear approximations also enables the restatement of the SCP as a series of Lagrange-Newton QP subproblems. This results in a diagonal trust-region SQP type of algorithm, in which the second order diagonal terms are estimated from the nonlinear (reciprocal) intervening variables, rather than from historic information using an exact or a quasi-Newton Hessian approach. The QP formulation seems particularly attractive for problems with far more constraints than variables (when pure dual methods are at a disadvantage), or when both the number of design variables and the number of (active) constraints is very large.     

Probabilistic sensitivity analysis for novel second-order reliability method (SORM) using generalized chi-squared distribution

Reliability-based design optimization (RBDO) requires evaluation of sensitivities of probabilistic constraints. To develop RBDO utilizing the recently proposed novel second-order reliability method (SORM) that improves conventional SORM approaches in terms of accuracy, the sensitivities of the probabilistic constraints at the most probable point (MPP) are required. Thus, this study presents sensitivity analysis of the novel SORM at MPP for more accurate RBDO. During analytic derivation in this study, it is assumed that the Hessian matrix does not change due to the small change of design variables. The calculation of the sensitivity based on the analytic derivation requires evaluation of probability density function (PDF) of a linear combination of non-central chi-square variables, which is obtained by utilizing general chi-squared distribution. In terms of accuracy, the proposed probabilistic sensitivity analysis is compared with the finite difference method (FDM) using the Monte Carlo simulation (MCS) through numerical examples. The numerical examples demonstrate that the analytic sensitivity of the novel SORM agrees very well with the sensitivity obtained by FDM using MCS when a performance function is quadratic in U-space and input variables are normally distributed. It is further shown that the proposed sensitivity is accurate enough compared with FDM results even for a higher order performance function.     

Adjoint network method applied to the performance sensitivities of microwave amplifiers

This work focuses on the performance sensitivities of microwave amplifiers using the “adjoint network and adjoint variable” method, via “wave” approaches, which includes sensitivities of the transducer power gain, noise figure, and magnitudes and phases of the input and output reflection coefficients. The method can be extended to sensitivities of the other performance measure functions. The adjoint‐variable methods for design‐sensitivity analysis offer computational speed and accuracy. They can be used for efficiency‐based gradient optimization, in tolerance and yield analyses. In this work, an arbitrarily configured microwave amplifier is considered: firstly, each element in the network is modeled by the scattering matrix formulation, then the topology of the network is taken into account using the connection scattering‐matrix formulation. The wave approach is utilized in the evaluation of all the performance‐measurement functions, then sensitivity invariants are formulated using Tellegen's theorem. Performance sensitivities of the T‐ and Π‐types of distributed‐parameter amplifiers are considered as a worked example. The numerical results of T‐ and Π‐type amplifiers for the design targets of noise figure F_req = 0.46 dB ? 1,12 and V_ireq = 1, G_Treq = 12 dB ? 15.86 in the frequency range 2–11 GHz are given in comparison to each other. Furthermore, analytical methods of the “gain factorisation” and “chain sensitivity parameter” are applied to the gain and noise sensitivities as well. In addition, “numerical perturbation” is applied to calculation of all the sensitivities. © 2006 Wiley Periodicals, Inc. Int J RF and Microwave CAE, 2006.     

Continuous Adjoint Methods for Turbulent Flows,Applied to Shape and Topology Optimization: Industrial Applications

This article focuses on the formulation, validation and application of the continuous adjoint method for turbulent flows in aero/hydrodynamic optimization. Though discrete adjoint has been extensively used in the past to compute objective function gradients with respect to (w.r.t.) the design variables under turbulent flow conditions, the development of the continuous adjoint variant for these flows is not widespread in the literature, hindering, to an extend, the computation of exact sensitivity derivatives. The article initially presents a general formulation of the continuous adjoint method for incompressible flows, under the commonly used assumption of “frozen turbulence”. Then, the necessary addenda are presented in order to deal with the differentiation of both low- and high-Reynolds (with wall functions) number turbulence models; the latter requires the introduction of the so-called “adjoint wall functions”. An approach to dealing with distance variations is also presented. The developed methods are initially validated in \(2D\) cases and then applied to industrial shape and topology optimization problems, originating from the automotive and hydraulic turbomachinery industries.     

Matrix-free algorithm for the optimization of multidisciplinary systems

Multidisciplinary engineering systems are usually modeled by coupling software components that were developed for each discipline independently. The use of disparate solvers complicates the optimization of multidisciplinary systems and has been a long-standing motivation for optimization architectures that support modularity. The individual discipline feasible (IDF) formulation is particularly attractive in this respect. IDF achieves modularity by introducing optimization variables and constraints that effectively decouple the disciplinary solvers during each optimization iteration. Unfortunately, the number of variables and constraints can be significant, and the IDF constraint Jacobian required by most conventional optimization algorithms is prohibitively expensive to compute. Furthermore, limited-memory quasi-Newton approximations, commonly used for large-scale problems, exhibit linear convergence rates that can struggle with the large number of design variables introduced by the IDF formulation. In this work, we show that these challenges can be overcome using a reduced-space inexact-Newton-Krylov algorithm. The proposed algorithm avoids the need for the explicit constraint Jacobian and Hessian by using a Krylov iterative method to solve the Newton steps. The Krylov method requires matrix-vector products, which can be evaluated in a matrix-free manner using second-order adjoints. The Krylov method also needs to be preconditioned, and a key contribution of this work is a novel and effective preconditioner that is based on approximating a monolithic solution of the (linearized) multidisciplinary system. We demonstrate the efficacy of the algorithm by comparing it with the popular multidisciplinary feasible formulation on two test problems.     

Rapid design closure of microwave components by means of feature‐based optimization and adjoint sensitivities

In this article, fast design closure of microwave components using feature‐based optimization (FBO) and adjoint sensitivities is discussed. FBO is one of the most recent optimization techniques that exploits a particular structure of the system response to “flatten” the functional landscape handled during the optimization process, which leads to reducing its computational complexity. When combined with gradient‐based search involving adjoint sensitivities, the design cost becomes even lower, allowing us to find the optimum design using just a few electromagnetic (EM) simulations of the structure at hand. Here, operation and performance of the algorithm is demonstrated using a waveguide filter and a miniaturized microstrip rat‐race coupler (RRC). Comparative studies indicate considerable savings that can be achieved even compared with adjoint‐based gradient search. In case of RRC, numerical results are supported by experimental validation.     

A level set based topology optimization method using the discretized signed distance function as the design variables

This paper deals with a new topology optimization method based on the level set method. In the proposed method, the discretized signed distance function, a kind of level set function, is used as the design variables, and these are then updated using their sensitivities. The signed distance characteristic of the design variables are maintained by performing a re-initialization at every update during the iterated optimization procedure. In this paper, a minimum mean compliance problem and a compliant mechanism design problem are formulated based on the level set method. In the formulations of these design problems, a perimeter constraint is imposed to overcome the ill-posedness of the structural optimization problem. The sensitivity analysis for the above structural optimization problems is conducted based on the adjoint variable method. The augmented Lagrangian method is incorporated to deal with multiple constraints. Finally, several numerical examples that include multiple constraints are provided to confirm the validity of the method, and it is shown that appropriate optimal structures are obtained.     

A BFGS-SQP method for nonsmooth,nonconvex, constrained optimization and its evaluation using relative minimization profiles

We propose an algorithm for solving nonsmooth, nonconvex, constrained optimization problems as well as a new set of visualization tools for comparing the performance of optimization algorithms. Our algorithm is a sequential quadratic optimization method that employs Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton Hessian approximations and an exact penalty function whose parameter is controlled using a steering strategy. While our method has no convergence guarantees, we have found it to perform very well in practice on challenging test problems in controller design involving both locally Lipschitz and non-locally-Lipschitz objective and constraint functions with constraints that are typically active at local minimizers. In order to empirically validate and compare our method with available alternatives—on a new test set of 200 problems of varying sizes—we employ new visualization tools which we call relative minimization profiles. Such profiles are designed to simultaneously assess the relative performance of several algorithms with respect to objective quality, feasibility, and speed of progress, highlighting the trade-offs between these measures when comparing algorithm performance.     

An external reconstruction approach (ERA) to linear programming

An externally initiated optimum-searching procedure for large-scale LP optimization and design is presented. It appears to be potentially useful for LP problems characterized by large number of constraints relative to the number of variables. Although ERA (External Reconstruction Approach) still relies on LP subproblems (and their simplex-type solution procedures), its global philosophy is clearly nonsimplex and nonellipsoid in nature. In addition to its computational potential, ERA'S major advantages are: renewed flexibility, built-in parallelism, suitability to design and optimization, ability to handle mixed (both “soft” and “hard”) constraints. A numerical example is appended.     

Improved semi-analytic sensitivity analysis combined with a higher order approximation scheme in the framework of adjoint variable method

In the design sensitivity analysis, the adjoint variable method has been widely used for the sensitivity calculation. The adjoint variable method can reduce computation time and save computer resources because it can provide the sensitivity values only at the positions in which designers are willing to obtain. However, exact analytical differentiation with respect to the design variables is commonly employed in adjoint variable method. Although the exact derivative assures the accurate sensitivity, it is cumbersome to take differentiation in an exact manner for every given type of finite element. Therefore, in the present study, a new improved semi-analytic design sensitivity method is proposed in the framework of adjoint variable method. Recently, a numerical inaccuracy trouble in the traditional semi-analytic method has been settled by the rigid body mode separation technique and high order approximation scheme. Combining the adjoint variable method with improved semi-analytic design sensitivity scheme, the design sensitivity value can be calculated accurately and efficiently. Through numerical examples, the efficiency and accuracy of the proposed semi-analytic sensitivity scheme in the adjoint variable method are demonstrated.     

On improving normal boundary intersection method for generation of Pareto frontier

Gradient-based methods, including Normal Boundary Intersection (NBI), for solving multi-objective optimization problems require solving at least one optimization problem for each solution point. These methods can be computationally expensive with an increase in the number of variables and/or constraints of the optimization problem. This paper provides a modification to the original NBI algorithm so that continuous Pareto frontiers are obtained “in one go,” i.e., by solving only a single optimization problem. Discontinuous Pareto frontiers require solving a significantly fewer number of optimization problems than the original NBI algorithm. In the proposed method, the optimization problem is solved using a quasi-Newton method whose history of iterates is used to obtain points on the Pareto frontier. The proposed and the original NBI methods have been applied to a collection of 16 test problems, including a welded beam design and a heat exchanger design problem. The results show that the proposed approach significantly reduces the number of function calls when compared to the original NBI algorithm.     

Singular value decomposition for dynamic system design sensitivity analysis

A computer-based method for automatic generation and efficient numerical solution of mixed differential-algebraic equations for dynamic and design sensitivity analysis of dynamic systems is developed. The equations are written in terms of a maximal set of Cartesian coordinates to facilitate general formulation of kinematic and design constraints and forcing functions. Singular value decomposition of the system Jacobian matrix generates a set of composite generalized coordinates that are best suited to represent the system. The coordinates naturally partition into optimal independent and dependent sets, and integration of only the independent coordinates generates all of the system information. An adjoint variable method is used to compute design sensitivities of dynamic performance measures of the system. A general-purpose computer program incorporating these capabilities has been developed. A numerical example is presented to illustrate accuracy and properties of the method.     

A contact force model considering constant external forces for impact analysis in multibody dynamics

The adjoint method is an elegant approach for the computation of the gradient of a cost function to identify a set of parameters. An additional set of differential equations has to be solved to compute the adjoint variables, which are further used for the gradient computation. However, the accuracy of the numerical solution of the adjoint differential equation has a great impact on the gradient. Hence, an alternative approach is the discrete adjoint method, where the adjoint differential equations are replaced by algebraic equations. Therefore, a finite difference scheme is constructed for the adjoint system directly from the numerical time integration method. The method provides the exact gradient of the discretized cost function subjected to the discretized equations of motion.     

Sampling-based approach for design optimization in the presence of interval variables

This paper proposes a methodology for sampling-based design optimization in the presence of interval variables. Assuming that an accurate surrogate model is available, the proposed method first searches the worst combination of interval variables for constraints when only interval variables are present or for probabilistic constraints when both interval and random variables are present. Due to the fact that the worst combination of interval variables for probability of failure does not always coincide with that for a performance function, the proposed method directly uses the probability of failure to obtain the worst combination of interval variables when both interval and random variables are present. To calculate sensitivities of the constraints and probabilistic constraints with respect to interval variables by the sampling-based method, behavior of interval variables at the worst case is defined by the Dirac delta function. Then, Monte Carlo simulation is applied to calculate the constraints and probabilistic constraints with the worst combination of interval variables, and their sensitivities. A merit of using an MCS-based approach in the X-space is that it does not require gradients of performance functions and transformation from X-space to U-space for reliability analysis, thus there is no approximation or restriction in calculating sensitivities of constraints or probabilistic constraints. Numerical results indicate that the proposed method can search the worst case probability of failure with both efficiency and accuracy and that it can perform design optimization with mixture of random and interval variables by utilizing the worst case probability of failure search.     

Continuous adjoint approach to the Spalart-Allmaras turbulence model for incompressible flows

A continuous adjoint formulation for the computation of the sensitivities of integral functions used in steady-flow, incompressible aerodynamics is presented. Unlike earlier continuous adjoint methods, this paper computes the adjoint to both the mean-flow and turbulence equations by overcoming the frequently made assumption that the variation in turbulent viscosity can be neglected. The development is based on the Spalart-Allmaras turbulence model, using the adjoint to the corresponding differential equation and boundary conditions. The proposed formulation is general and can be used with any other integral function. Here, the continuous adjoint method yielding the sensitivities of the total pressure loss functional for duct flows with respect to the normal displacements of the solid wall nodes is presented. Using three duct flow problems, it is demonstrated that the adjoint to the turbulence equations should be taken into account to compute the sensitivity derivatives of this functional with high accuracy. The so-computed derivatives almost coincide with “reference” sensitivities resulting from the computationally expensive direct differentiation. This is not, however, the case of the sensitivities computed without solving the turbulence adjoint equation, which deviates from the reference values. The role of all newly appearing terms in the adjoint equations, their boundary conditions and the gradient expression is investigated, significant and insignificant terms are identified and a study on the Reynolds number effect is included.     

Taking “biology” just seriously enough: Commentary on “On the Mapping of Genotype to Phenotype in Evolutionary Algorithms” by Peter A. Whigham,Grant Dick,and James Maclaurin

“On the Mapping of Genotype to Phenotype in Evolutionary Algorithms,” by Peter A. Whigham, Grant Dick, and James Maclaurin, is a welcome reminder that evolutionary computation practitioners should be wary of taking their biological analogies too seriously. But more importantly, it is a reminder to practitioners to consider carefully their representations and operators, rather than blindly implementing a biological analogy without sufficient attention to the constraints of software engineering. “It works in biology, so it should work in EC” is poor, even lazy, software design. The primary contribution of this paper is exactly what a commentary should be: to (re)ignite discussions about how biological inspiration should inform EC practice.     

An implicit, exact dual adjoint solution method for turbulent flows on unstructured grids

An implicit algorithm for solving the discrete adjoint system based on an unstructured-grid discretization of the Navier-Stokes equations is presented. The method is constructed such that an adjoint solution exactly dual to a direct differentiation approach is recovered at each time step, yielding a convergence rate which is asymptotically equivalent to that of the primal system. The new approach is implemented within a three-dimensional unstructured-grid framework and results are presented for inviscid, laminar, and turbulent flows. Improvements to the baseline solution algorithm, such as line-implicit relaxation and a tight coupling of the turbulence model, are also presented. By storing nearest-neighbor terms in the residual computation, the dual scheme is computationally efficient, while requiring twice the memory of the flow solution. The current implementation allows for multiple right-hand side vectors, enabling simultaneous adjoint solutions for several cost functions or constraints with minimal additional storage requirements, while reducing the solution time compared to serial applications of the adjoint solver. The scheme is expected to have a broad impact on computational problems related to design optimization as well as error estimation and grid adaptation efforts.