Parallel computation of unsteady three-dimensional incompressible viscous flow using an unstructured multigrid method

The development and validation of a parallel unstructured tetrahedral non-nested multigrid (MG) method for simulation of unsteady 3D incompressible viscous flow is presented. The Navier-Stokes solver is based on the artificial compressibility method (ACM) and a higher-order characteristics-based finite-volume scheme on unstructured MG. Unsteady flow is calculated with an implicit dual time stepping scheme. The parallelization of the solver is achieved by a MG domain decomposition approach (MG-DD), using the Single Program Multiple Data (SPMD) programming paradigm. The Message-Passing Interface (MPI) Library is used for communication of data and loop arrays are decomposed using the OpenMP standard. The parallel codes using single grid and MG are used to simulate steady and unsteady incompressible viscous flows for a 3D lid-driven cavity flow for validation and performance evaluation purposes. The speedups and efficiencies obtained by both the parallel single grid and MG solvers are reasonably good for all test cases, using up to 32 processors on the SGI Origin 3400. The parallel results obtained agree well with those of serial solvers and with numerical solutions obtained by other researchers, as well as experimental measurements.     

An automatic Euler solver using the unstructured upwind method

A fully-automatic grid generator and an unstructured upwind method for the Euler equations are developed in order to achieve automation in flow computations. The unstructured grid is generated using two techniques: a geometry-adaptive refinement; and a solution-adaptive refinement. The former introduces information about the flowfield geometry into the grid, and the latter introduces the fluid physics. A combination of these two techniques enables the generation of a grid in a fully-automatic manner. Flowfields are solved by an unstructured upwind solver, which is an extension of the flux-vector splitting method of Van Leer for use on arbitrary-shaped unstructured meshes. This robust flow solver with the automatic grid generator can be a useful CFD tool for routine engineering work. The method is applied to external and internal flow problems to demonstrate its capability.     

Efficient nonlinear static aeroelastic wing analysis

The objective of this work is to demonstrate a computationally efficient, high-fidelity, integrated static aeroelastic analysis procedure. The aerodynamic analysis consists of solving the nonlinear Euler equations by using an upwind cell-centered finite-volume scheme on unstructured tetrahedral meshes. The use of unstructured grids enhances the discretization of irregularly shaped domains and the interaction compatibility with the wing structure. The structural analysis utilizes finite elements to model the wing so that accurate structural deflections are obtained and allows the capability for computing detailed stress information for the configuration. Parameters are introduced to control the interaction of the computational fluid dynamics and structural analyses; these control parameters permit extremely efficient static aeroelastic computations. To demonstrate and evaluate this procedure, static aeroelastic analysis results for a flexible wing in low subsonic, high subsonic (subcritical), transonic (supercritical), and supersonic flow conditions are presented.     

A block LU-SGS implicit unsteady incompressible flow solver on hybrid dynamic grids for 2D external bio-fluid simulations

A hybrid dynamic grid generation technique for two-dimensional (2D) morphing bodies and a block lower-upper symmetric Gauss-Seidel (BLU-SGS) implicit dual-time-stepping method for unsteady incompressible flows are presented for external bio-fluid simulations. To discretize the complicated computational domain around 2D morphing configurations such as fishes and insect/bird wings, the initial grids are generated by a hybrid grid strategy firstly. Body-fitted quadrilateral (quad) grids are generated first near solid bodies. An adaptive Cartesian mesh is then generated to cover the entire computational domain. Cartesian cells which overlap the quad grids are removed from the computational domain, and a gap is produced between the quad grids and the adaptive Cartesian grid. Finally triangular grids are used to fill this gap. During the unsteady movement of morphing bodies, the dynamic grids are generated by a coupling strategy of the interpolation method based on ‘Delaunay graph’ and local remeshing technique. With the motion of moving/morphing bodies, the grids are deformed according to the motion of morphing body boundaries firstly with the interpolation strategy based on ‘Delaunay graph’ proposed by Liu and Qin. Then the quality of deformed grids is checked. If the grids become too skewed, or even intersect each other, the grids are regenerated locally. After the local remeshing, the flow solution is interpolated from the old to the new grid. Based on the hybrid dynamic grid technique, an efficient implicit finite volume solver is set up also to solve the unsteady incompressible flows for external bio-fluid dynamics. The fully implicit equation is solved using a dual-time-stepping approach, coupling with the artificial compressibility method (ACM) for incompressible flows. In order to accelerate the convergence history in each sub-iteration, a block lower-upper symmetric Gauss-Seidel implicit method is introduced also into the solver. The hybrid dynamic grid generator is tested by a group of cases of morphing bodies, while the implicit unsteady solver is validated by typical unsteady incompressible flow case, and the results demonstrate the accuracy and efficiency of present solver. Finally, some applications for fish swimming and insect wing flapping are carried out to demonstrate the ability for 2D external bio-fluid simulations.     

Parallel multigrid on hierarchical hybrid grids: a performance study on current high performance computing clusters

This article studies the performance and scalability of a geometric multigrid solver implemented within the hierarchical hybrid grids (HHG) software package on current high performance computing clusters up to nearly 300,000 cores. HHG is based on unstructured tetrahedral finite elements that are regularly refined to obtain a block‐structured computational grid. One challenge is the parallel mesh generation from an unstructured input grid that roughly approximates a human head within a 3D magnetic resonance imaging data set. This grid is then regularly refined to create the HHG grid hierarchy. As test platforms, a BlueGene/P cluster located at Jülich supercomputing center and an Intel Xeon 5650 cluster located at the local computing center in Erlangen are chosen. To estimate the quality of our implementation and to predict runtime for the multigrid solver, a detailed performance and communication model is developed and used to evaluate the measured single node performance, as well as weak and strong scaling experiments on both clusters. Thus, for a given problem size, one can predict the number of compute nodes that minimize the overall runtime of the multigrid solver. Overall, HHG scales up to the full machines, where the biggest linear system solved on Jugene had more than one trillion unknowns. Copyright © 2012 John Wiley & Sons, Ltd.     

An Efficient Correction Method to Obtain a Formally Third-Order Accurate Flow Solver for Node-Centered Unstructured Grids

A new method of obtaining third-order accuracy on unstructured grid flow solvers is presented. The method involves a simple correction to a traditional linear Galerkin scheme on tetrahedra and can be conveniently added to existing second-order accurate node-centered flow solvers. The correction involves gradients of the flux computed with a quadratic least squares approximation. However, once the gradients are computed, no second derivative information or high-order quadrature is necessary to achieve third-order accuracy. The scheme is analyzed both analytically using truncation error, and numerically using solution error for an exact solution to the Euler equations. Two demonstration cases for steady, inviscid flow reveal increased accuracy and excellent shock capturing with no loss in steady-state convergence rate. Computational timing results are presented which show the additional expense from the correction is modest compared to the increase in accuracy.     

A FV-TD electromagnetic solver using adaptive Cartesian grids

A second-order finite-volume (FV) method has been developed to solve the time-domain (TD) Maxwell equations, which govern the dynamics of electromagnetic waves. The computational electromagnetic (CEM) solver is capable of handling arbitrary grids, including structured, unstructured, and adaptive Cartesian grids, which are topologically arbitrary. It is argued in this paper that the adaptive Cartesian grid is better than a tetrahedral grid for complex geometries considering both efficiency and accuracy. A cell-wise linear reconstruction scheme is employed to achieve second-order spatial accuracy. Second-order time accuracy is obtained through a two-step Runge-Kutta scheme. Issues on automatic adaptive Cartesian grid generation such as cell-cutting and cell-merging are discussed. A multi-dimensional characteristic absorbing boundary condition (MDC-ABC) is developed at the truncated far-field boundary to reduce reflected waves from this artificial boundary. The CEM solver is demonstrated with several test cases with analytical solutions.     

Moving-body simulations using overset framework with rigid body dynamics

The simulation of flow past bodies in relative motion is a challenging task due to the presence of complex flow features, moving grids, and rigid body movements under the action of external forces and moments. A generalized grid-based overset framework is presented for the simulation of this class of problems. The equations that govern the fluid flows are cast in an integral form and are solved using a cell-centered finite volume upwind scheme. The rigid body dynamics equations are formulated using quaternion and are solved using fourth-order Runge–Kutta (RK) time integration. The overset framework and the six degree of freedom (6-DOF) rigid body dynamics simulators are developed in a library form for easy incorporation into existing flow solvers. The details of the flow solver, the 6-DOF library, and the overset framework are presented in this paper along with the validation results of the developed system.     

CFD-based analysis and two-level aerodynamic optimization on graphics processing units

This paper presents the porting of 2D and 3D Navier–Stokes equations solvers for unstructured grids, from the CPU to the graphics processing unit (GPU; NVIDIA’s Ge-Force GTX 280 and 285), using the CUDA language. The performance of the GPU implementations, with single, double or mixed precision arithmetic operations, is compared to that of the CPU code.Issues regarding the optimal handling of the unstructured grid topology on the GPU, particularly for vertex-centered CFD algorithms, are discussed. Restructuring the existing codes was necessary in order to maximize the parallel efficiency of the GPU implementations. The mixed precision implementation, in which the left-hand-side operators are computed with single precision, was shown to bridge the gap between the single and double precision speed-ups. Based on the different speed-ups and prediction accuracy of the aforementioned GPU implementations of the Navier–Stokes equations solver, a hierarchical optimization method which is suitable for GPUs is proposed and demonstrated in inviscid and turbulent 2D flow problems. The search for the optimal solution(s) splits into two levels, both relying upon evolutionary algorithms (EAs) though with different evaluation tools each. The low level EA uses the very fast single precision GPU implementation with relaxed convergence criteria for the inexpensive evaluation of candidate solutions. Promising solutions are regularly broadcast to the high level EA which uses the mixed precision GPU implementation of the same flow solver. Single- and two-objective aerodynamic shape optimization problems are solved using the developed software.     

Simulation of unsteady two-phase flows using a parallel Eulerian–Lagrangian approach

The present study details the implementation of a time accurate method for the tracking of particles being acted upon by a continuous gas phase and gravity. The Lagrangian particle tracking approach was implemented within the framework of a parallel, incompressible, unstructured, node-centered finite-volume flow solver. The paper gives a method for selecting time steps for individual particles such that interactions with the continuum phase are updated at particle locations nearest the continuum-phase nodes while constraining the particle from passing beyond boundaries of the relevant adjacent cell. An implementation of this technique for three-dimensional nonuniform multi-element unstructured grids is given in the context of domain decomposition for implementation on distributed-memory parallel computers. Results of simulations with and without particle–particle collisions compare favorably with experimental validation results.     

A hybrid method for unsteady inviscid fluid flow

We show how a stable and accurate hybrid procedure for fluid flow can be constructed. Two separate solvers, one using high order finite difference methods and another using the node-centered unstructured finite volume method are coupled in a truly stable way. The two flow solvers run independently and receive and send information from each other by using a third coupling code. Exact solutions to the Euler equations are used to verify the accuracy and stability of the new computational procedure. We also demonstrate the capability of the new procedure in a calculation of the flow in and around a model of a coral.     

Efficient nonlinear solvers for Laplace–Beltrami smoothing of three-dimensional unstructured grids

The Laplace–Beltrami system of nonlinear, elliptic, partial differential equations has utility in the generation of computational grids on complex and highly curved geometry. Discretization of this system using the finite-element method accommodates unstructured grids, but generates a large, sparse, ill-conditioned system of nonlinear discrete equations. The use of the Laplace–Beltrami approach, particularly in large-scale applications, has been limited by the scalability and efficiency of solvers. This paper addresses this limitation by developing two nonlinear solvers based on the Jacobian-Free Newton–Krylov (JFNK) methodology. A key feature of these methods is that the Jacobian is not formed explicitly for use by the underlying linear solver. Iterative linear solvers such as the Generalized Minimal RESidual (GMRES) method do not technically require the stand-alone Jacobian; instead its action on a vector is approximated through two nonlinear function evaluations. The preconditioning required by GMRES is also discussed. Two different preconditioners are developed, both of which employ existing Algebraic Multigrid (AMG) methods. Further, the most efficient preconditioner, overall, for the problems considered is based on a Picard linearization. Numerical examples demonstrate that these solvers are significantly faster than a standard Newton–Krylov approach; a speedup factor of approximately 26 was obtained for the Picard preconditioner on the largest grids studied here. In addition, these JFNK solvers exhibit good algorithmic scaling with increasing grid size.     

Compact third-order multidimensional upwind discretization for steady and unsteady flow simulations

We propose a new third-order multidimensional upwind algorithm for the solution of the flow equations on tetrahedral cells unstructured grids. This algorithm has a compact stencil (cell-based computations) and uses a finite element idea when computing the residual over the cell to achieve its third-order (spatial) accuracy. The construction of the new scheme is presented. The asymptotic accuracy for steady or unsteady, inviscid or viscous flow situations is proved using numerical experiments. The new high-order discretization proves to have excellent parallel scalability. Our studies show the advantages of the new compact third-order scheme when compared with the classical second-order multidimensional upwind schemes.     

One- and two-equation turbulence models for the prediction of complex cascade flows using unstructured grids

One- and two-equation, low-Reynolds eddy-viscosity turbulence models are employed in the context of a primitive variable, finite volume, Navier-Stokes solver for unstructured grids. Through the study of the complex flow in a controlled-diffusion compressor cascade at off-design conditions, the ability of the models under consideration to predict the laminar separation bubble close to the leading edge and the boundary layer development is investigated. In order to control the unphysical growth of turbulent kinetic energy near the leading edge stagnation point, appropriate modifications to the conventional models are employed and tested. All of them improve the leading edge flow patterns and significantly affect the size of the predicted laminar separation bubble. The use of an adequately refined mesh around the airfoil, that is formed by triangles placed in a quasi-structured way, allows for the generation of grid elements of moderate aspect ratios. This helps to readily overcome any relevant problems of accuracy; a second-order upwind scheme without flux limiters or least squares approximations is successfully employed for the gradients. The test case includes quasi-3D effects by considering the streamtube thickness variation in the governing equations.     

Adaptive multigrid for finite element computations in plasticity

The solution of the system of equilibrium equations is the most time-consuming part in large-scale finite element computations of plasticity problems. The development of efficient solution methods are therefore of utmost importance to the field of computational plasticity. Traditionally, direct solvers have most frequently been used. However, recent developments of iterative solvers and preconditioners may impose a change. In particular, preconditioning by the multigrid technique is especially favorable in FE applications.The multigrid preconditioner uses a number of nested grid levels to improve the convergence of the iterative solver. Prolongation of fine-grid residual forces is done to coarser grids and computed corrections are interpolated to the fine grid such that the fine-grid solution successively is improved. By this technique, large 3D problems, invincible for solvers based on direct methods, can be solved in acceptable time at low memory requirements. By means of a posteriori error estimates the computational grid could successively be refined (adapted) until the solution fulfils a predefined accuracy level. In contrast to procedures where the preceding grids are erased, the previously generated grids are used in the multigrid algorithm to speed up the solution process.The paper presents results using the adaptive multigrid procedure to plasticity problems. In particular, different error indicators are tested.     

Adaptive Embedded/Immersed Unstructured Grid Techniques

Embedded mesh, immersed body or ficticious domain techniques have been used for many years as a way to discretize geometrically complex domains with structured grids. The use of such techniques within adaptive, unstructured grid solvers is relatively recent. The combination of body-fitted functionality for some portion of the domain, together with embedded mesh or immersed body functionality for another portion of the domain offers great advantages, which are increasingly being exploited. The present paper reviews the methodology from an implementational perspective.     

Hydrodynamic design using a derivative-free method

A derivative-free shape optimization tool for computational fluid dynamics (CFD) is developed in order to facilitate the implementation of complex flow solvers in the design procedure. A modified Rosenbrocks method is used, which needs neither gradient evaluations nor approximations. This approach yields a robust and flexible tool and gives the capability of performing optimizations involving complex configurations and phenomena. The flow solver implemented solves the Reynolds-averaged Navier–Stokes equations (RANSE) on unstructured grids, using near-wall, low-Reynolds-number turbulence models. Free surface effects are taken into account by a pseudosteady surface tracking method. A mesh deformation strategy based on both lineal and torsional springs analogies is used to update the mesh while maintaining the quality of the grid near the wall for two-dimensional problems. A free-form-deformation technique is used to manage the mesh and the shape perturbations for three-dimensional cases. Two hydrodynamic applications are presented, concerning first the design of a two-dimensional hydrofoil in relation with the free-surface elevation and then the three-dimensional optimization of a hull shape, at full scale.     

An upwind discretization scheme for the finite volume lattice Boltzmann method

The fact that the classic lattice Boltzmann method is restricted to Cartesian Grids has inspired several researchers to apply Finite Volume [Nannelli F, Succi S. The lattice Boltzmann equation on irregular lattices. J Stat Phys 1992;68:401–7; Peng G, Xi H, Duncan C, Chou SH. Finite volume scheme for the lattice Boltzmann method on unstructured meshes. Phys Rev E 1999;59:4675–82; Chen H. Volumetric formulation of the lattice Boltzmann method for fluid dynamics: basic concept. Phys Rev E 1998;58:3955–63] or Finite Element [Lee T, Lin CL. A characteristic Galerkin method for discrete Boltzmann equation. J Comp Phys 2001;171:336–56; Shi X, Lin J, Yu Z. Discontinuous Galerkin spectral element lattice Boltzmann method on triangular element. Int J Numer Methods Fluids 2003;42:1249–61] methods to the Discrete Boltzmann equation. The finite volume method proposed by Peng et al. works on unstructured grids, thus allowing an increased geometrical flexibility. However, the method suffers from substantial numerical instability compared to the standard LBE models. The computational efficiency of the scheme is not competitive with standard methods.We propose an alternative way of discretizing the convection operator using an upwind scheme, as opposed to the central scheme described by Peng et al. We apply our method to some test problems in two spatial dimensions to demonstrate the improved stability of the new scheme and the significant improvement in computational efficiency. Comparisons with a lattice Boltzmann solver working on a hierarchical grid were done and we found that currently finite volume methods for the discrete Boltzmann equation are not yet competitive as stand alone fluid solvers.     

On the behavior of upwind schemes in the low Mach number limit. IV: P0 approximation on triangular and tetrahedral cells

Finite Volume upwind schemes for the Euler equations in the low Mach number regime face a problem of lack of convergence toward the solutions of the incompressible system. However, if applied to cell centered triangular grid, this problem disappears and convergence toward the incompressible solution is recovered. Extending the work of [3] that prove this result for regular triangular grid, we give here a general proof of this fact for arbitrary unstructured meshes. In addition, we also show that this result is equally valid for unstructured three dimensional tetrahedral meshes.     

An algorithm for ideal multigrid convergence for the steady Euler equations

A fast multigrid solver for the steady incompressible Euler equations is presented. Unlike time-marching schemes this approach uses relaxation of the steady equations. Application of this method results in a discretization that correctly distinguishes between the advection and elliptic parts of the operator, allowing efficient smoothers to be constructed. Solvers for both unstructured triangular grids and structured quadrilateral grids have been written. Flows in two-dimensional channels and over airfoils have been computed. Using Gauss–Seidel relaxation with the grid vertices ordered in the flow direction, ideal multigrid convergence rates of nearly one order-of-magnitude residual reduction per multigrid cycle are achieved, independent of the grid spacing. This approach also may be applied to the compressible Euler equations and the incompressible Navier–Stokes equations.