Optimal design of multi-response experiments using semi-definite programming

Optimal design of multi-response experiments for estimating the parameters of multi-response linear models is a challenging problem. The main drawback of the existing algorithms is that they require the solution of many optimization problems in the process of generating an optimal design that involve cumbersome manual operations. Furthermore, all the existing methods generate approximate design and no method for multi-response n-exact design has been cited in the literature. This paper presents a unified formulation for multi-response optimal design problem using Semi-Definite Programming (SDP) that can generate D-, A- and E-optimal designs. The proposed method alleviates the difficulties associated with the existing methods. It solves a one-shot optimization model whose solution selects the optimal design points among all possible points in the design space. We generate both approximate and n-exact designs for multi-response models by solving SDP models with integer variables. Another advantage of the proposed method lies in the amount of computation time taken to generate an optimal design for multi-response models. Several test problems have been solved using an existing interior-point based SDP solver. Numerical results show the potentials and efficiency of the proposed formulation as compared with those of other existing methods. The robustness of the generated designs with respect to the variance-covariance matrix is also investigated.     

Tool selection for optimal part production: a Lagrangian relaxation approach

This paper extends previous work on implementation problems associated with a flexible system that produces flat sheet-metal parts with interior holes. The paper makes four main contributions. First, we formulate the problem of selecting tooling and design standards to minimize the cost of producing parts as an optimization model. Second, we develop a projected subgradient algorithm for the Lagrangian relaxation of the problem by using the model's special structure to develop relationships between the Lagrangian multipliers. Third, we demonstrate that the algorithm produces close to optimal solutions (duality gap less than 2%) very quickly on a number of problems derived using a substantial data set obtained from a Chicago area firm. Fourth, an important variant of the traditional repair kit problem is shown to be a special case of the tool selection problem.     

A nonlinear programming technique to compute a tight lower bound for the real structured singular value

The real structured singular value (RSSV, or real μ) is a useful measure to analyze the robustness of linear systems subject to structured real parametric uncertainty, and surely a valuable design tool for the control systems engineers. We formulate the RSSV problem as a nonlinear programming problem and use a new computation technique, F-modified subgradient (F-MSG) algorithm, for its lower bound computation. The F-MSG algorithm can handle a large class of nonconvex optimization problems and requires no differentiability. The RSSV computation is a well known NP hard problem. There are several approaches that propose lower and upper bounds for the RSSV. However, with the existing approaches, the gap between the lower and upper bounds is large for many problems so that the benefit arising from usage of RSSV is reduced significantly. Although the F-MSG algorithm aims to solve the nonconvex programming problems exactly, its performance depends on the quality of the standard solvers used for solving subproblems arising at each iteration of the algorithm. In the case it does not find the optimal solution of the problem, due to its high performance, it practically produces a very tight lower bound. Considering that the RSSV problem can be discontinuous, it is found to provide a good fit to the problem. We also provide examples for demonstrating the validity of our approach.     

Design optimization using multiple dominance relations

A challenge in engineering design is to choose suitable objectives and constraints from many quantities of interest, while ensuring an optimization is both meaningful and computationally tractable. We propose an optimization formulation that can take account of more quantities of interest than existing formulations, without reducing the tractability of the problem. This formulation searches for designs that are optimal with respect to a binary relation within the set of designs that are optimal with respect to another binary relation. We then propose a method of finding such designs in a single optimization by defining an overall ranking function to use in optimizers, reducing the cost required to solve this formulation. In a design under uncertainty problem, our method obtains the most robust design that is not stochastically dominated faster than a multiobjective optimization. In a car suspension design problem, our method obtains superior designs according to a k-optimality condition than previously suggested multiobjective approaches to this problem. In an airfoil design problem, our method obtains designs closer to the true lift/drag Pareto front using the same computational budget as a multiobjective optimization.     

Optimal design of gating systems by gradient search methods

A numerical optimization technique based on gradient-search is applied to obtain an optimal design of a typical gating system used for the gravity process to produce aluminum parts. This represents a novel application of coupling nonlinear optimization techniques with a foundry process simulator, and it is motivated by the fact that a scientifically guided search for better designs based on techniques that take into account the mathematical structure of the problem is preferred to commonly found trial-and-error approaches. The simulator applies the finite volume method and the VOF algorithm for CFD analysis. The direct gradient optimization algorithm, sequential quadratic programming (SQP), was used to solve both a 2D and a 3D gating system design problems using two design variables. The results clearly show the effectiveness of the proposed approach for finding high quality castings when compared with current industry practices.     

LAGRANGEAN RELAXATION AND SUBGRADIENT OPTIMIZATION APPLIED TO OPTIMAL DESIGN WITH DISCRETE SIZING

The discrete sizing problem in optimal design is adressed. Lagrangean dual approaches earlier published are briefly reviewed and it is noted that quite sophisticated procedures have been used to solve the dual problems. The simple concept of Lagrangean relaxation combined with subgradient optimization and Lagrangean heuristics has, however, not been applied to the discrete sizing problem. In this paper a scheme based on this concept is described and tested on some small problems. The results indicate that subgradient optimization is completely capable of solving the dual problem. Moreover it is possible to devise heuristics that construct feasible solutions to the original problem, using the Lagrangean subproblem solution.     

Using Genetic Algorithms to Design Experiments: A Review

Genetic algorithms (GAs) have been used in many disciplines to optimize solutions for a broad range of problems. In the last 20 years, the statistical literature has seen an increase in the use and study of this optimization algorithm for generating optimal designs in a diverse set of experimental settings. These efforts are due in part to an interest in implementing a novel methodology as well as the hope that careful application of elements of the GA framework to the unique aspects of a designed experiment problem might lead to an efficient means of finding improved or optimal designs. In this paper, we explore the merits of using this approach, some of the aspects of design that make it a unique application relative to other optimization scenarios, and discuss elements which should be considered for an effective implementation. We conclude that the current GA implementations can, but do not always, provide a competitive methodology to produce substantial gains over standard optimal design strategies. We consider both the probability of finding a globally optimal design as well as the computational efficiency of this approach. Copyright © 2014 John Wiley & Sons, Ltd.     

Application of semi-definite programming to the design of multi-response experiments

The need to be able to design experiments with multiple responses is becoming apparent in many real-world applications. The generation of an optimal design to estimate the parameters of a multi-response model is a challenging problem. Currently available algorithms require the solution of many optimization problems in order to generate an optimal design. In this paper, the problem of multi-response D-optimal design is formulated as a semi-definite programming model and a relaxed form of it is solved using interior-point solvers. The main advantage of the proposed method lies in the amount of computation time taken to generate a D-optimal design for multi-response models. The proposed method is tested on several test problems and is shown to be very efficient with optimal designs being found very quickly in all cases. The robustness of the generated designs with respect to the variance-covariance matrix is also assessed for the test problems in order to show how a sensitivity analysis can be performed. The characteristics of the proposed method are also compared with those of other existing methods.     

Optimal subgradient methods: computational properties for large-scale linear inverse problems

This paper studies the computational properties of the optimal subgradient algorithm (OSGA) for applications of linear inverse problems involving high-dimensional data. First, such convex problems are formulated as a class of convex problems with multi-term composite objective functions involving linear mappings. Next, an efficient procedure for computing the first-order oracle for such problems is provided and OSGA is equipped with some prox-functions such that the OSGA subproblem is solved in a closed form. Further, a comprehensive comparison among the most popular first-order methods is given. Then, several Nesterov-type optimal methods (originally proposed for smooth problems) are adapted to solve nonsmooth problems by simply passing a subgradient instead of the gradient, where the results of these subgradient methods are competitive and totally interesting for solving nonsmooth problems. Finally, numerical results with several inverse problems (deblurring with isotropic total variation, elastic net, and \(\ell _1\)-minimization) show the efficiency of OSGA and the adapted Nesterov-type optimal methods for large-scale problems. For the deblurring problem, the efficiency measures of the improvement on the signl-to-noise ratio and the peak signal-to-noise ratio are used. The software package implementing OSGA is publicly available.     

Multiobjective stacking sequence optimization for laminated composite structures

A Pareto-based multiobjective evolutionary algorithm is presented for stacking sequence optimization of composite structural parts. Special attention has been paid to engineering design guidelines for stacking sequence design. These guidelines are included into the formulation of the optimization problem as constraints or additional objectives. A new initialization strategy is proposed following mechanical considerations. The method is applied to the optimal design of a composite plate for weight minimization and maximization of the buckling margins under three hundred load cases that make also the originality of this work. It is shown that the introduction of new ply orientations compared to the classical 0°, ±45° and 90° plies can lead to significantly improved optimal designs.     

SIMULATED ANNEALING AND PARALLEL PROCESSING: AN IMPLEMENTATION FOR CONSTRAINED GLOBAL DESIGN OPTIMIZATION

Global optimization becomes important as more and more complex designs are evaluated and optimized for superior performance. Often parametric designs are highly constrained, adding complexity to the design problem. In this work simulated annealing (SA), a stochastic global optimization technique, is implemented by augmenting it with a feasibility improvement scheme (FIS) that makes it possible to formulate and solve a constrained optimization problem without resorting to artificially modifying the objective function. The FIS is also found to help recover from the infeasible design space rapidly. The effectiveness of the improved algorithm is demonstrated by solving a welded beam design problem and a two part stamping optimization problem. Large scale practical design problems may prohibit the efficient use of computationally intensive iterative algorithms such as SA. Hence the FIS augmented SA algorithm is implemented on an Intel iPSC/860 parallel super-computer using a data parallel structure of the algorithm for the solution of large scale optimization problems. The numerical results demonstrate the effectiveness of the FIS as well as the parallel version of the SA algorithm. Expressions are developed for the estimation of the speedup of iterative algorithms running on a parallel computer with hyper-cube interconnection topology. Computational speedup in excess of 8 is achieved using 16 processors. The timing results given for the example problems provide guidelines to designers in the use of parallel computers for iterative processes.     

THE OPTIMAL DESIGN OF DRAINAGE SYSTEMS

This paper derives from a research project investigating the systematic use of formal optimization techniques in computer aided building design. A comprehensive model of drainage design suitable for optimization is developed. After limited success with a very powerful general purpose optimization program based upon the Geometric Programming technique, a problem dependent algorithm is developed based upon Dynamic Programming. The latter technique is shown in actual designs to produce optimal drainage designs simply and cheaply.     

Novel orthogonal simulated annealing with fractional factorial analysis to solve global optimization problems

A classical simulated annealing (SA) method is a generic probabilistic and heuristic approach to solving global optimization problems. It uses a stochastic process based on probability, rather than a deterministic procedure, to seek the minima or maxima in the solution space. Although the classical SA method can find the optimal solution to most linear and nonlinear optimization problems, the algorithm always requires numerous numerical iterations to yield a good solution. The method also usually fails to achieve optimal solutions to large parameter optimization problems. This study incorporates well-known fractional factorial analysis, which involves several factorial experiments based on orthogonal tables to extract intelligently the best combination of factors, with the classical SA to enhance the numerical convergence and optimal solution. The novel combination of the classical SA and fractional factorial analysis is termed the orthogonal SA herein. This study also introduces a dynamic penalty function to handle constrained optimization problems. The performance of the proposed orthogonal SA method is evaluated by computing several representative global optimization problems such as multi-modal functions, noise-corrupted data fitting, nonlinear dynamic control, and large parameter optimization problems. The numerical results show that the proposed orthogonal SA method markedly outperforms the classical SA in solving global optimization problems with linear or nonlinear objective functions. Additionally, this study addressed two widely used nonlinear functions, proposed by Keane and Himmelblau to examine the effectiveness of the orthogonal SA method and the presented penalty function when applied to the constrained problems. Moreover, the orthogonal SA method is applied to two engineering optimization design problems, including the designs of a welded beam and a coil compression spring, to evaluate the capacity of the method for practical engineering design. The computational results show that the proposed orthogonal SA method is effective in determining the optimal design variables and the value of objective function.     

Multi-objective optimization of generalized reliability design problems using feature models—A concept for early design stages

Reliability optimization problems such as the redundancy allocation problem (RAP) have been of considerable interest in the past. However, due to the restrictions of the design space formulation, they may not be applicable in all practical design problems. A method with high modelling freedom for rapid design screening is desirable, especially in early design stages. This work presents a novel approach to reliability optimization. Feature modelling, a specification method originating from software engineering, is applied for the fast specification and enumeration of complex design spaces. It is shown how feature models can not only describe arbitrary RAPs but also much more complex design problems. The design screening is accomplished by a multi-objective evolutionary algorithm for probabilistic objectives. Comparing averages or medians may hide the true characteristics of this distributions. Therefore the algorithm uses solely the probability of a system dominating another to achieve the Pareto optimal set. We illustrate the approach by specifying a RAP and a more complex design space and screening them with the evolutionary algorithm.     

Multimaterial topology design for optimal elastic and thermal response with material-specific temperature constraints

We present an original method for multimaterial topology optimization with elastic and thermal response considerations. The material distribution is represented parametrically using a formulation in which finite element–style shape functions are used to determine the local material properties within each finite element. We optimize a multifunctional structure that is designed for a combination of structural stiffness and thermal insulation. We conduct parallel uncoupled finite element analyses to simulate the elastic and thermal response of the structure by solving the two-dimensional Poisson problem. We explore multiple optimization problem formulations, including structural design for minimum compliance subject to local temperature constraints so that the optimized design serves as both a support structure and a thermal insulator. We also derive and implement an original multimaterial aggregation function that allows the designer to simultaneously enforce separate maximum temperature thresholds based upon the melting point of the various design materials. The nonlinear programming problem is solved using gradient-based optimization with adjoint sensitivity analysis. We present results for a series of two-dimensional example problems. The results demonstrate that the proposed algorithm consistently converges to feasible multimaterial designs with the desired elastic and thermal performance.     

Bayesian Design of Experiments Using Approximate Coordinate Exchange

The construction of decision-theoretical Bayesian designs for realistically complex nonlinear models is computationally challenging, as it requires the optimization of analytically intractable expected utility functions over high-dimensional design spaces. We provide the most general solution to date for this problem through a novel approximate coordinate exchange algorithm. This methodology uses a Gaussian process emulator to approximate the expected utility as a function of a single design coordinate in a series of conditional optimization steps. It has flexibility to address problems for any choice of utility function and for a wide range of statistical models with different numbers of variables, numbers of runs and randomization restrictions. In contrast to existing approaches to Bayesian design, the method can find multi-variable designs in large numbers of runs without resorting to asymptotic approximations to the posterior distribution or expected utility. The methodology is demonstrated on a variety of challenging examples of practical importance, including design for pharmacokinetic models and design for mixed models with discrete data. For many of these models, Bayesian designs are not currently available. Comparisons are made to results from the literature, and to designs obtained from asymptotic approximations. Supplementary materials for this article are available online.     

Using the sequential linear integer programming method as a post‐processor for stress‐constrained topology optimization problems

This paper deals with topology optimization of load‐carrying structures defined on discretized continuum design domains. In particular, the minimum compliance problem with stress constraints is considered. The finite element method is used to discretize the design domain into n finite elements and the design of a certain structure is represented by an n‐dimensional binary design variable vector. In order to solve the problems, the binary constraints on the design variables are initially relaxed and the problems are solved with both the method of moving asymptotes and the sparse non‐linear optimizer solvers for continuous optimization in order to compare the two solvers. By solving a sequence of problems with a sequentially lower limit on the amount of grey allowed, designs that are close to ‘black‐and‐white’ are obtained. In order to get locally optimal solutions that are purely {0, 1}ⁿ, a sequential linear integer programming method is applied as a post‐processor. Numerical results are presented for some different test problems. Copyright © 2008 John Wiley & Sons, Ltd.     

Crashworthiness design optimization using successive response surface approximations

 Finite Element (FE) method is among the most powerful tools for crash analysis and simulation. Crashworthiness design of structural members requires repetitive and iterative application of FE simulation. This paper presents a crashworthiness design optimization methodology based on efficient and effective integration of optimization methods, FE simulations, and approximation methods. Optimization methods, although effective in general in solving structural design problems, loose their power in crashworthiness design. Objective and constraint functions in crashworthiness optimization problems are often non-smooth and highly non-linear in terms of design variables and follow from a computationally costly (FE) simulation. In this paper, a sequential approximate optimization method is utilized to deal with both the high computational cost and the non-smooth character. Crashworthiness optimization problem is divided into a series of simpler sub-problems, which are generated using approximations of objective and constraint functions. Approximations are constructed by using statistical model building technique, Response Surface Methodology (RSM) and a Genetic algorithm. The approximate optimization method is applied to solve crashworthiness design problems. These include a cylinder, a simplified vehicle and New Jersey concrete barrier optimization. The results demonstrate that the method is efficient and effective in solving crashworthiness design optimization problems. Received: 30 January 2002 / Accepted: 12 July 2002 Sponsorship for this research by the Federal Highway Administration of US Department of Transportation is gratefully acknowledged. Dr. Nielen Stander at Livermore Software Technology Corporation is also gratefully acknowledged for providing subroutines to create D-optimal experimental designs and the simplified vehicle model.     

Response surface optimization for a nonlinearly constrained irregular experimental design space

Finding optimum conditions for process factors in an engineering optimization problem with response surface functions requires structured data collection using experimental design. When the experimental design space is constrained owing to external factors, its design space may form an asymmetrical and irregular shape and thus standard experimental design methods become ineffective. Computer-generated optimal designs, such as D-optimal designs, provide alternatives. While several iterative exchange algorithms for D-optimal designs are available for a linearly constrained irregular design space, it has not been clearly understood how D-optimal design points need to be generated when the design space is nonlinearly constrained and how response surface models are optimized. This article proposes an algorithm for generating the D-optimal design points that satisfy both feasibility and optimality conditions by using piecewise linear functions on the design space. The D-optimality-based response surface design models are proposed and optimization procedures are then analysed.