Accelerating multi-dimensional combustion simulations using GPU and hybrid explicit/implicit ODE integration

Simulating multi-dimensional combustion with detailed kinetics often requires solving a large number of ordinary differential equation (ODE) problems at each global time step. In many cases, the ODE integrations account for the bulk of the total wall-clock time for the simulation. This paper introduces CHEMEQ2-GPU – a new explicit stiff ODE solver (based on the existing CHEMEQ2 solver) that exploits the parallel architecture of the modern graphics processing unit (GPU) to accelerate ODE integration in multi-dimensional combustion simulations. We also demonstrate efficient application of the CPU and GPU as co-processors, for further speedup. We describe a hybrid explicit/implicit ODE solver approach that combines the strengths of both solver types running simultaneously on the GPU and CPU, respectively. A dynamic load balancing scheme was used to assign the kinetics ODE integrations over all grid points to either the CPU-based implicit solver DVODE (which is the more efficient solver for highly stiff grid points) or CHEMEQ2-GPU (more efficient for moderately stiff or non-stiff grid points). We demonstrate CHEMEQ2-GPU and the hybrid approach in 3-D simulations of homogeneous charge compression ignition (HCCI) engines. The test cases applied two different n-heptane reaction mechanisms (a large detailed model and a small skeletal model) and three different mesh sizes. Engine simulations were performed using KIVA-CHEMKIN. CHEMEQ2 was about 2–3 times faster than DVODE, with similar prediction accuracy. The CHEMEQ2-GPU speedup relative to CHEMEQ2 increased linearly with the number of grid points for the range of meshes tested in this work. Assuming ideal linear scaling of simulation time with number of processors, the speed of CHEMEQ2-GPU on the Tesla C2050 GPU was equivalent to CHEMEQ2 running on approximately 13 parallel 2.8 GHz CPU processors for the finest mesh; and the hybrid solver approach was equivalent to CHEMEQ2 on ～15 such CPU processors. In summary, CHEMEQ2-GPU provided the additional computing power of 14 parallel CPU processors (for the finest mesh tested) and the hybrid solver approach demonstrated a method to efficiently apply these additional co-processors with existing CPU cores for combustion simulations. CHEMEQ2-GPU scales favorably with the number of grid points and is available by request to the authors. This work presents opportunities for further development, particularly in CPU/GPU load balancing algorithms.     

A high-resolution Navier–Stokes solver for direct numerical simulation of free shear flow

Abstract

A high-resolution, Navier–Stokes solver is developed for direct numerical simulation (DNS) of free shear flow. All terms in Navier–Stokes equations are discretized using higher order methods. Diffusion term is discretized using fourth order central difference scheme while second order Adams–Bashforth is used for time derivative. Advecting velocity is approximated using fourth order Lagrangian interpolation. For the approximation of advected velocity, a blended fifth order-upwind scheme is proposed. Developed high resolution solver is used for DNS of round jet in transitional and turbulent regimes. A novel open outlet boundary condition (OOBC) is proposed which has the ability to dynamically adjust according to prevailing local condition at the outlet thereby minimizing reflections from outlet. Ability of blended fifth order upwind scheme and fifth order WENO is assessed in terms of algorithmic efficiency as well as fidelity of simulations. It is demonstrated that the proposed blended fifth order upwind scheme outperforms the WENO scheme in terms of algorithmic efficiency. Assessment of fidelity of simulations reveals that WENO displays a tendency to over-predict momentum advection in transitional as well as fully turbulent regime of the round jet. In contrast, the proposed advection scheme is not faced with such limitation.     

A finite volume method to solve the Navier–Stokes equations for incompressible flows on unstructured meshes

A method to solve the Navier–Stokes equations for incompressible viscous flows and the convection and diffusion of a scalar is proposed in the present paper. This method is based upon a fractional time step scheme and the finite volume method on unstructured meshes. A recently proposed diffusion scheme with interesting theoretical and numerical properties is tested and integrated into the Navier–Stokes solver. Predictions of Poiseuille flows, backward-facing step flows and lid-driven cavity flows are then performed to validate the method. We finally demonstrate the versatility of the method by predicting buoyancy force driven flows of a Boussinesq fluid (natural convection of air in a square cavity with Rayleigh numbers of 10 ³ and 10 ⁶ ).     

Numerical simulation of two-fluid magnetohydrodynamic channel flows

Abstract

A study of liquid metal magnetohydrodynamic (MHD) flows is important for analyzing flows in the Test Blanket Module of the proposed International Thermonuclear Experimental Reactor. Possible accident scenarios involve the leakage of coolant into the liquid metal flow. To address this situation, the present article considers the simplified case of two-dimensional isothermal flow of two fluids of different viscosities and electrical conductivities. We present analytical solutions to the fully developed flows and a solver based on OpenFOAM. We validate the solver performance with the analytical solution, and also the limiting cases of single fluid MHD flow and two-fluid hydrodynamic flow.     

A semi-implicit numerical scheme for a transient two-fluid three-field model on an unstructured grid

A three-dimensional (3D) unstructured hydrodynamic solver for transient two-phase flows has been developed for a 3D component of a nuclear system code and a component-scale analysis tool. A two-fluid three-field model is used for the two-phase flows. The three fields represent a continuous liquid, an entrained liquid, and a vapor field. An unstructured grid is adopted for realistic simulations of the flows in a complicated geometry. The semi-implicit ICE (Implicit Continuous-fluid Eulerian) numerical scheme has been adapted for an unstructured non-staggered grid. This paper presents the numerical method and the preliminary results of the calculations. The results show that the modified ICE scheme is robust and predicts the phase changes and the flow transitions due to a boiling and a flashing very well.     

Extending a low-order upwind-biased scheme to solve turbulent flames using detailed chemistry model

Abstract

Achieving more accurate reacting flow numerical solutions apparently demand employing higher-order schemes, utilizing finer grids, and benefiting from more advanced chemistry models. One major objective of this work is to extend an inclusive low-order upwind-biased scheme in the context of finite-volume-element method to predict turbulent reacting flows on coarse grid resolutions very reliably. In this regard, a low-order upwind-biased scheme is suitably extended to approximate the mixture fraction variances at the cell-faces. This scheme implements the reacting flow physics explicitly in deriving the proposed mixture fraction variance expressions. These physical implementations enhance the derived expressions to result in superior turbulent reacting flow solutions even on coarse grid resolutions. To assess the accuracy of new expressions, we simulate a sample turbulent non-premixed flame with strong non-equilibrium effects of turbulence on chemistry. The comparisons show that the current low-order scheme is robust enough to predict the complex structure of non-premixed flames very reliably even on coarse grids.     

Enhancing the Performance of a Parallel Solver for Turbulent Reacting Flow Simulations

This article describes results of an effort to improve the parallel efficiency of a solver for turbulent reacting flows on two computer architectures. The compact finite-difference scheme employed for the solution of the differential equations involves the inversion of multiple tridiagonal matrices at each time step. Detailed performance evaluation of the standard LU, parallel partition LU, and parallel diagonal dominant algorithms are presented. The speed-up and efficiencies of these parallel strategies are critically compared and evaluated based on both computation and communication complexities, on the CRAY XT4 and IBM Blue Gene/P architectures.     

Large eddy simulation of mean and oscillating flow in a side-dump ramjet combustor

Ramjet flows are very sensitive to combustion instabilities that are difficult to predict using numerical simulations. This paper describes compressible large eddy simulation on unstructured grids used to investigate nonreacting and reacting flows in a simplified twin-inlet ramjet combustor. The reacting flow is compared to experimental results published by ONERA in terms of mean fields. Simulations show a specific flow topology controlled by the impingement of the two air jets issuing from the twin air inlets and by multiple complex recirculation zones. In a second part, all unsteady modes appearing in the reacting LES are analyzed using spectral maps and POD (proper orthogonal decomposition) tools. A Helmholtz solver also computes the frequencies and structures of all acoustic modes in the ramjet. Pure longitudinal, transverse and combined modes are identified by all three diagnostics. In addition, a mode-by-mode analysis of the Rayleigh criterion is presented thanks to POD. This method shows that the most intense structure (at 3750 Hz) is the first transverse acoustic mode of the combustor chamber and the Rayleigh criterion obtained with POD illustrates how this transverse mode couples with unsteady combustion.     

Solutions for variable density low Mach number flows with a compressible pressure-based algorithm

Two difficulties are always encountered in the simulation of variable density low Mach number flows by the primitive variable methods: (1) the decoupling of pressure and velocity caused by the extremely small velocity; (2) the spurious oscillations, nonsmooth solutions and sharp resolution caused by discontinuities, making that the flow cannot be successfully simulated by low-ordered schemes. In this paper, a pressure-based compressible flow solver with a scheme of high-resolution is developed to overcome these difficulties. In the present work, we adopt pressure correction algorithm to overcome the decoupling and the MUSCL scheme to predict the discontinuities at low Mach number. We applied the proposed compressible flow solver to simulate the compressible flows in a planar nozzle, arc bumps and the lid-driven cavity and found that the numerical results are in good agreement with those reported in the previous works, indicating that the numerical algorithm developed in this work is a reliable and accurate tool for studying thermal variable density low Mach number flows.     

ExaWind: Open-source CFD for hybrid-RANS/LES geometry-resolved wind turbine simulations in atmospheric flows

Predictive high-fidelity modeling of wind turbines with computational fluid dynamics, wherein turbine geometry is resolved in an atmospheric boundary layer, is important to understanding complex flow accounting for design strategies and operational phenomena such as blade erosion, pitch-control, stall/vortex-induced vibrations, and aftermarket add-ons. The biggest challenge with high-fidelity modeling is the realization of numerical algorithms that can capture the relevant physics in detail through effective use of high-performance computing. For modern supercomputers, that means relying on GPUs for acceleration. In this paper, we present ExaWind, a GPU-enabled open-source incompressible-flow hybrid-computational fluid dynamics framework, comprising the near-body unstructured grid solver Nalu-Wind, and the off-body block-structured-grid solver AMR-Wind, which are coupled using the Topology Independent Overset Grid Assembler. Turbine simulations employ either a pure Reynolds-averaged Navier–Stokes turbulence model or hybrid turbulence modeling wherein Reynolds-averaged Navier–Stokes is used for near-body flow and large eddy simulation is used for off-body flow. Being two-way coupled through overset grids, the two solvers enable simulation of flows across a huge range of length scales, for example, 10 orders of magnitude going from O(μm) boundary layers along the blades to O(10 km) across a wind farm. In this paper, we describe the numerical algorithms for geometry-resolved turbine simulations in atmospheric boundary layers using ExaWind. We present verification studies using canonical flow problems. Validation studies are presented using megawatt-scale turbines established in literature. Additionally presented are demonstration simulations of a small wind farm under atmospheric inflow with different stability states.     

A semi-analytical boundary collocation solver for the inverse Cauchy problems in heat conduction under 3D FGMs with heat source

Abstract

In this article, the inverse Cauchy problems in heat conduction under 3D functionally graded materials (FGMs) with heat source are solved by using a semi-analytical boundary collocation solver. In the present semi-analytical solver, the combined boundary particle method and regularization technique is employed to deal with ill-pose inverse Cauchy problems. The domain mapping method and variable transformation are introduced to derive the high-order general solutions satisfying the heat conduction equation of 3D FGMs. Thanks to these derived high-order general solutions, the proposed scheme can only require the boundary discretization to recover the solutions of the heat conduction equations with a heat source. The regularization technique is used to eliminate the effect of the noisy measurement data on the accessible boundary surface of 3D FGMs. The efficiency of the proposed solver for inverse Cauchy problems is verified under several typical benchmark examples related to 3D FGM with specific spatial variations (quadratic, exponential and trigonometric functions).     

Implementation of a fast fluid dynamics model in OpenFOAM for simulating indoor airflow

ABSTRACT

This study implemented fast fluid dynamics (FFD) in Open Field Operation and Manipulation and used a local searching method that made the FFD solver applicable to unstructured meshes. Because the split scheme used in FFD is not conservative, this investigation developed a combined scheme that used a split scheme for the continuity and momentum equations and an iterative scheme for scalar equations. The combined scheme ensures conservation of the scalars. This investigation used two two-dimensional cases and one three-dimensional case, with the experimental data, to test the FFD solver. The predicted results were similar with different types of mesh and numerical scheme and agreed in general with the experimental data.     

Mean-field/PDF numerical approach for polydispersed turbulent two-phase flows

The purpose of this paper is to give an overview in the realm of numerical computations of polydispersed turbulent two-phase flows, using a mean-field/PDF approach. In this approach, the numerical solution is obtained by resorting to a hybrid method, where the mean fluid properties are computed by solving mean-field (RANS) equations with a classical finite volume procedure whereas the local instantaneous properties of the particles are determined by solving stochastic differential equations (SDEs). The fundamentals of the general formalism are recalled and particular attention is focused on a specific theoretical issue: the treatment of the multiscale character of the dynamics of the discrete particles, i.e. the consistency of the system of SDEs in asymptotic cases. Then, the main lines of the particle/mesh algorithm are given and some specific problems, related to the integration of the SDEs, are discussed, for example, issues related to the specificity of the treatment of the averaging and projection operators, the time integration of the SDEs (weak numerical schemes consistent with all asymptotic cases), and the computation of the source terms. Practical simulations, for three different flows, are performed in order to demonstrate the ability of both the models and the numericals to cope with the stringent specificities of polydispersed turbulent two-phase flows.     

DEVELOPMENTS ON AN UNSTEADY BOUNDARY-LAYER ANALYSIS: INTERNAL AND EXTERNAL FLOWS

Abstract

A boundary-layer solution procedure for two-dimensional, compressible unsteady flows has been developed for non-Cartesian generalized grids consistent with first-order boundary-layer theory. The scheme is applicable to both internal and external unsteady flows. Example results demonstrate advantages of using a non-Cartesian grid for external flows. Boundary-layer solutions were reported for the first time for a number of flows that had been computed only by quite different approaches, including the numerical solutions to the Navier-Stokes equations. The results computed for several test cases were found to be in good agreement with data available in the literature.     

A Coupled Pressure-Based Finite-Volume Solver for Incompressible Two-Phase Flow

This article reports on the formulation and testing of a coupled pressure-based algorithm for the solution of steady incompressible disperse two-phase-flow problems. The method is formulated within a Eulerian-Eulerian framework in the context of a collocated finite-volume scheme. An equation for pressure is derived from overall mass conservation following the segregated mass conservation–based algorithm (MCBA) approach and using an extended two-phase flow form of the Rhie-Chow interpolation technique. The newly developed pressure-based coupled solver differs from pressure-based segregated solvers in that it accounts implicitly for the pressure–velocity and the interphase drag couplings that are present in disperse multiphase flows to yield a system of coupled equations linking the velocity and pressure fields. The performance and accuracy of the coupled multiphase algorithm are assessed by solving eight one-dimensional two-phase flow problems spanning the spectrum from dilute bubbly to dense gas–solid flows. Each problem is solved over three grid systems with sizes of 10,000, 30,000, and 50,000 control volumes, respectively. Results are compared in terms of iterations and CPU time with similar ones generated using the segregated MCBA-SIMPLE algorithm. The newly developed coupled solver is shown to yield substantial decrease in the required number of iterations and CPU time, with the rate of solution acceleration varying between 1.3 and 4.6.     

Compressible large eddy simulation of turbulent combustion in complex geometry on unstructured meshes

Large-eddy simulations (LESs) of an industrial gas turbine burner are carried out for both nonreacting and reacting flow using a compressible unstructured solver. Results are compared with experimental data in terms of axial and azimuthal velocities (mean and RMS), averaged temperature, and existence of natural instabilities such as precessing vortex core (PVC). The LES is performed with a reduced two-step mechanism for methane-air combustion and a thickened flame model. The regime of combustion is partially premixed and the computation includes part of the swirler vanes. For this very complex geometry, results demonstrate the capacity of the LES to predict the mean flow, with and without combustion, as well as its main unstable modes: it is shown, for example, that the PVC mode is very strong for the cold flow but disappears with combustion.     

CONTROLLED VARIATION SCHEME IN A SEQUENTIAL SOLVER FOR RECIRCULATING FLOWS,PART I: THEORY AND FORMULATION

The formalism of the total variation diminishing (TVD) schemes is utilized to design a convection scheme for incompressible recirculating flows. The scheme has been named the controlled variation scheme (CVS). Even though the CVS does not possess the TVD property for sequential solution algorithms, due to the appearance of source terms, the concept of controlled variation fluxes can be effective in suppressing spurious oscillations that commonly occur in convection-dominated viscous flows, by injecting a nonlinear numerical diffusion, similar to the original TVD schemes, into the central difference scheme. This is demonstrated by using the one-dimensional linear convection-diffusion equation with and without a source term as model problems. The formulation and an efficient implementation of the CVS in a sequential pressure-based solver for incompressible steady-state Navier-Stokes equations is presented in this work. The applications of the CVS for two-dimensional laminar and turbulent flows is presented in Part II of the present work.     

An OpenFOAM pressure-based coupled CFD solver for turbulent and compressible flows in turbomachinery applications

ABSTRACT

In this article, a recently developed pressure-based, fully coupled solver capable of predicting fluid flow at all speeds is extended to deal with turbulent flows in a rotating frame of reference and emphasizing turbomachinery applications. The pressure–velocity coupling at the heart of the Navier-Stokes equations is resolved by deriving a pressure equation in a similar fashion to a segregated SIMPLE algorithm but with implicit treatment of the velocity and pressure fields. The resulting system of coupled equations is solved using an algebraic multigrid solver. The above numerical procedures have been implemented within OpenFOAM®, which is an open-source code framework capable of dealing with industrial-scale flow problems. The OpenFOAM-based coupled solver is validated using experimental and numerical data available from reference literature test cases as well as with a segregated solver based on the SIMPLE algorithm. This is done in addition to evaluating its performance by solving an industrial problem. In comparison with the segregated solver, the coupled solver results indicate substantial reduction in computational cost with increased robustness.     

Hybrid central–WENO scheme for the large eddy simulation of turbulent flows with shocks

To provide an effective numerical method for the large eddy simulation (LES) of turbulent flows with shocks, a hybrid scheme is developed in a finite volume framework based on the fourth-order central scheme and the third-order weighted essentially non-oscillatory (WENO) scheme. A total of six easy-to-implement and promising switch functions (SFs) are examined in the hybrid central–WENO scheme for the LES of compressible turbulent flows. Both the dissipation and dispersion of the developed hybrid central–WENO scheme are theoretically confirmed using the Fourier technique. Then, the effectiveness and accuracy of this scheme and the SFs are numerically tested by three problems: decaying compressible isotropic turbulence, inviscid, and turbulent transonic flow over a bump. The numerical results show the developed hybrid scheme, coupled with the SF based on local velocity divergence and pressure gradient, has excellent capabilities of capturing shocks and resolving turbulence.