Development of a parallel implicit solver of fluid modeling equations for gas discharges

A parallel fully implicit PETSc-based fluid modeling equations solver for simulating gas discharges is developed. Fluid modeling equations include: the neutral species continuity equation, the charged species continuity equation with drift-diffusion approximation for mass fluxes, the electron energy density equation, and Poisson's equation for electrostatic potential. Except for Poisson's equation, all model equations are discretized by the fully implicit backward Euler method as a time integrator, and finite differences with the Scharfetter–Gummel scheme for mass fluxes on the spatial domain. At each time step, the resulting large sparse algebraic nonlinear system is solved by the Newton–Krylov–Schwarz algorithm. A 2D-GEC RF discharge is used as a benchmark to validate our solver by comparing the numerical results with both the published experimental data and the theoretical prediction. The parallel performance of the solver is investigated.     

Numerical solution of potential flow equations with a predictor-corrector finite difference method

We develop a numerical solution algorithm of the nonlinear potential flow equations with the nonlinear free surface boundary condition.A finite difference method with a predictor-corrector method is applied to solve the nonlinear potential flow equations in a two-dimensional (2D) tank.The irregular tank is mapped onto a fixed square domain with rectangular cells through a proper mapping function.A staggered mesh system is adopted in a 2D tank to capture the wave elevation of the transient fluid.The finite difference method with a predictor-corrector scheme is applied to discretize the nonlinear dynamic boundary condition and nonlinear kinematic boundary condition.We present the numerical results of wave elevations from small to large amplitude waves with free oscillation motion,and the numerical solutions of wave elevation with horizontal excited motion.The beating period and the nonlinear phenomenon are very clear.The numerical solutions agree well with the analytical solutions and previously published results.     

Fluid Simulation with Articulated Bodies

We present an algorithm for creating realistic animations of characters that are swimming through fluids. Our approach combines dynamic simulation with data-driven kinematic motions (motion capture data) to produce realistic animation in a fluid. The interaction of the articulated body with the fluid is performed by incorporating joint constraints with rigid animation and by extending a solid/fluid coupling method to handle articulated chains. Our solver takes as input the current state of the simulation and calculates the angular and linear accelerations of the connected bodies needed to match a particular motion sequence for the articulated body. These accelerations are used to estimate the forces and torques that are then applied to each joint. Based on this approach, we demonstrate simulated swimming results for a variety of different strokes, including crawl, backstroke, breaststroke, and butterfly. The ability to have articulated bodies interact with fluids also allows us to generate simulations of simple water creatures that are driven by simple controllers.     

Multi-solver algorithms for the partitioned simulation of fluid–structure interaction

In partitioned fluid–structure interaction simulations, the flow equations and the structural equations are solved separately. As a result, a coupling algorithm is needed to enforce the equilibrium on the fluid–structure interface in cases with strong interaction. This coupling algorithm performs coupling iterations between the solver of the flow equations and the solver of the structural equations. Current coupling algorithms couple one flow solver with one structural solver. Here, a new class of multi-solver quasi-Newton coupling algorithms for unsteady fluid–structure interaction simulations is presented. More than one flow solver and more than one structural solver are used for a single simulation. The numerical experiments demonstrate that the duration of a simulation decreases as the number of solvers is increased.     

A fixed-grid model for simulation of a moving body in free surface flows

A two-dimensional computer model is developed to simulate free surface flow interaction with a moving body. The model is based on the cut-cell technique in a fixed-grid system. In this model, a body is approximated by the partial cell treatment (PCT), in which an irregular body is represented by the volumetric fraction of solid in Cartesian cells. The body motion is tracked by Lagrangian method whereas the fluid motion around the body is solved by Eulerian method. The concept of “locally relative stationary (LRS)” is introduced in this study. In the LRS method, a source term is added locally to the conventional continuity equation on body surfaces to take account of body motions, which subsequently affects the computational results of fluid pressure and flow velocity around the body. The LRS method is incorporated into an earlier Reynolds averaged Navier-Stokes (RANS) equations model developed by Lin and Liu [A numerical study of breaking waves in the surf zone. J Fluid Mech 1998;359:239-64]. The new model is capable of simulating generic turbulent free surface flows and their interaction with a moving body or multiple moving bodies. A series of numerical experiments have been conducted to verify the accuracy of the model for simulation of moving body interaction with a free surface flow. These tests include the generation of a solitary wave with the prescribed wave paddle movements, water exit and water impact and entry of a horizontal circular cylinder, fluid sloshing in a horizontally excited tank, and the acceleration/deceleration of an elliptical cylinder near a water surface. Excellent agreements are obtained when numerical results are compared to available analytical, experimental, and other numerical results. The model is a simple-to-implement computational tool for simulating a moving body in turbulent free surface flows.     

An efficient solver for the RANS equations and a one-equation turbulence model

A three-stage Runge-Kutta (RK) scheme with multigrid and an implicit preconditioner has been shown to be an effective solver for the fluid dynamic equations. Using the algebraic turbulence model of Baldwin and Lomax, this scheme has been used to solve the compressible Reynolds-averaged Navier–Stokes (RANS) equations for transonic and low-speed flows. In this paper we focus on the convergence of the RK/Implicit scheme when the effects of turbulence are represented by the one-equation model of Spalart and Allmaras. With the present scheme the RANS equations and the partial differential equation of the turbulence model are solved in a loosely coupled manner. This approach allows the convergence behavior of each system to be examined. Point symmetric Gauss-Seidel supplemented with local line relaxation is used to approximate the inverse of the implicit operator of the RANS solver. To solve the turbulence equation we consider three alternative methods: diagonally dominant alternating direction implicit (DDADI), symmetric line Gauss-Seidel (SLGS), and a two-stage RK scheme with implicit preconditioning. Computational results are presented for airfoil flows, and comparisons are made with experimental data. We demonstrate that the two-dimensional RANS equations and a transport-type equation for turbulence modeling can be efficiently solved with an indirectly coupled algorithm that uses RK/Implicit schemes.     

Local penalty methods for flows interacting with moving solids at high Reynolds numbers

An original numerical modelling of multiphase flows interacting with solids in unsteady regimes is presented. Based on the generalized Navier-Stokes equations for multiphase flows and Volume of Fluid (VOF) formulations, an Uzawa minimization algorithm is implemented for the treatment of incompressibility and solid constraints. Augmented Lagrangian terms are added in the momentum equations to speed the convergence of the iterative solver. Defining a priori the penalty parameters which are dedicated to incompressibility and solid constraints is difficult, or impossible, as soon as the flow involves more than one phase and inertia becomes predominant compared to viscous and gravity forces. The Lagrangian penalty terms are calculated automatically according to an original local estimate of the various physical contributions. Numerical validations have been carried out for single particle settling in confined media and viscous flow through a fixed Cubic Faced Centered array. A very good agreement is obtained between experimental, theoretical and numerical results. Extension to unsteady free surface flow interacting with particles is illustrated with the simulation of a dam break flow over moving obstacles.     

Photorealistic visualization and fluid animation: coupling of Maya with a two-phase Navier-Stokes fluid solver

We have coupled the three-dimensional solver for the two-phase incompressible Navier-Stokes equations NaSt3DGPF with Autodesk Maya. Maya is the industry standard software framework for the creation of three-dimensional animations. The parallel level-set-based fluid solver NaSt3DGPF simulates the interaction of two fluids like air and water. It uses high-order finite difference discretization methods that are designed for physics applications. By coupling both applications, we are now able to set up scientific fluid simulations in an easy-to-use user interface. Moreover, the rendering techniques provided by Maya allow us to create photorealistic visualizations for computational fluid dynamics problems and support the creation of highly visually realistic fluid simulations for animation movies. Altogether, we obtain an easy usable and fully coupled fluid animation toolkit for two-phase fluid simulations. These are the first published results of the full integration of a physics-oriented, high-order grid-based parallel two-phase fluid solver in Maya, at least to our knowledge.     

A multigrid solver for phase field simulation of microstructure evolution

This paper presents a semi-implicit numerical method for the simulation of grain growth in two dimensions with a multi-phase field model. To avoid the strong stability condition of traditional explicit methods, a first-order, semi-implicit discretisation scheme is employed, which offers a good compromise with regard to memory intensity and computational requirements. A nonlinear multigrid solver based on the Full Approximation Scheme is implemented to solve the equations resulting from this discretisation. Simulations with the multigrid solver show that the solver has grid size independent convergence properties and is faster than a standard first-order explicit solver. As such, the multigrid solver promises to be a reliable additional computational tool for the simulation of microstructural evolution. A comparison with existing alternatives remains, however, subject of further investigation. To validate the implementation, the results of specific test cases are studied.     

An ADI-based adaptive mesh Poisson solver for the MHD code NIRVANA

I describe a Poisson solver for the adaptive mesh magnetohydrodynamics (MHD) code NIRVANA using ADI techniques (ADI: Alternative Direction Implicit). The solver is fit to the mesh refinement framework of the code and utilizes its special block-structured design. The key part of the method is an algorithm for the intelligent clustering of subgrids which permits the application of numerical methods based on dimensional operator splitting like ADI. Test problems show the convergence of this ansatz.     

A block LU-SGS implicit unsteady incompressible flow solver on hybrid dynamic grids for 2D external bio-fluid simulations

A hybrid dynamic grid generation technique for two-dimensional (2D) morphing bodies and a block lower-upper symmetric Gauss-Seidel (BLU-SGS) implicit dual-time-stepping method for unsteady incompressible flows are presented for external bio-fluid simulations. To discretize the complicated computational domain around 2D morphing configurations such as fishes and insect/bird wings, the initial grids are generated by a hybrid grid strategy firstly. Body-fitted quadrilateral (quad) grids are generated first near solid bodies. An adaptive Cartesian mesh is then generated to cover the entire computational domain. Cartesian cells which overlap the quad grids are removed from the computational domain, and a gap is produced between the quad grids and the adaptive Cartesian grid. Finally triangular grids are used to fill this gap. During the unsteady movement of morphing bodies, the dynamic grids are generated by a coupling strategy of the interpolation method based on ‘Delaunay graph’ and local remeshing technique. With the motion of moving/morphing bodies, the grids are deformed according to the motion of morphing body boundaries firstly with the interpolation strategy based on ‘Delaunay graph’ proposed by Liu and Qin. Then the quality of deformed grids is checked. If the grids become too skewed, or even intersect each other, the grids are regenerated locally. After the local remeshing, the flow solution is interpolated from the old to the new grid. Based on the hybrid dynamic grid technique, an efficient implicit finite volume solver is set up also to solve the unsteady incompressible flows for external bio-fluid dynamics. The fully implicit equation is solved using a dual-time-stepping approach, coupling with the artificial compressibility method (ACM) for incompressible flows. In order to accelerate the convergence history in each sub-iteration, a block lower-upper symmetric Gauss-Seidel implicit method is introduced also into the solver. The hybrid dynamic grid generator is tested by a group of cases of morphing bodies, while the implicit unsteady solver is validated by typical unsteady incompressible flow case, and the results demonstrate the accuracy and efficiency of present solver. Finally, some applications for fish swimming and insect wing flapping are carried out to demonstrate the ability for 2D external bio-fluid simulations.     

Interactive visualization of volumetric white matter connectivity in DT-MRI using a parallel-hardware Hamilton-Jacobi solver

In this paper we present a method to compute and visualize volumetric white matter connectivity in diffusion tensor magnetic resonance imaging (DT-MRI) using a Hamilton-Jacobi (H-J) solver on the GPU (Graphics Processing Unit). Paths through the volume are assigned costs that are lower if they are consistent with the preferred diffusion directions. The proposed method finds a set of voxels in the DTI volume that contain paths between two regions whose costs are within a threshold of the optimal path. The result is a volumetric optimal path analysis, which is driven by clinical and scientific questions relating to the connectivity between various known anatomical regions of the brain. To solve the minimal path problem quickly, we introduce a novel numerical algorithm for solving H-J equations, which we call the Fast Iterative Method (FIM). This algorithm is well-adapted to parallel architectures, and we present a GPU-based implementation, which runs roughly 50-100 times faster than traditional CPU-based solvers for anisotropic H-J equations. The proposed system allows users to freely change the endpoints of interesting pathways and to visualize the optimal volumetric path between them at an interactive rate. We demonstrate the proposed method on some synthetic and real DT-MRI datasets and compare the performance with existing methods.     

On the improvement of a scalable sparse direct solver for unsymmetrical linear equations

This paper focuses on the application level improvements in a sparse direct solver specifically used for large-scale unsymmetrical linear equations resulting from unstructured mesh discretization of coupled elliptic/hyperbolic PDEs. Existing sparse direct solvers are designed for distributed server systems taking advantage of both distributed memory and processing units. We conducted extensive numerical experiments with three state-of-the-art direct linear solvers that can work on distributed-memory parallel architectures; namely, MUMPS (MUMPS solver website, http://graal.ens-lyon.fr/MUMPS), WSMP (Technical Report TR RC-21886, IBM, Watson Research Center, Yorktown Heights, 2000), and SUPERLU_DIST (ACM Trans Math Softw 29(2):110–140, 2003). The performance of these solvers was analyzed in detail, using advanced analysis tools such as Tuning and Analysis Utilities (TAU) and Performance Application Programming Interface (PAPI). The performance is evaluated with respect to robustness, speed, scalability, and efficiency in CPU and memory usage. We have determined application level issues that we believe they can improve the performance of a distributed-shared memory hybrid variant of this solver, which is proposed as an alternative solver [SuperLU_MCDT (Many-Core Distributed)] in this paper. The new solver utilizing the MPI/OpenMP hybrid programming is specifically tuned to handle large unsymmetrical systems arising in reservoir simulations so that higher performance and better scalability can be achieved for a large distributed computing system with many nodes of multicore processors. Two main tasks are accomplished during this study: (i) comparisons of public domain solver algorithms; existing state-of-the-art direct sparse linear system solvers are investigated and their performance and weaknesses based on test cases are analyzed, (ii) improvement of direct sparse solver algorithm (SuperLU_MCDT) for many-core distributed systems is achieved. We provided results of numerical tests that were run on up to 16,384 cores, and used many sets of test matrices for reservoir simulations with unstructured meshes. The numerical results showed that SuperLU_MCDT can outperform SuperLU_DIST 3.3 in terms of both speed and robustness.     

Coupling of a nonlinear finite element structural method with a Navier-Stokes solver

A new three-dimensional viscous aeroelastic solver is developed in the present work. A well validated full Navier-Stokes code is coupled with a nonlinear finite element plate model. Implicit coupling between the computational fluid dynamics and structural solvers is achieved using a subiteration approach. Computations of several benchmark static and dynamic plate problems are used to validate the finite element portion of the code. This coupled aeroelastic scheme is then applied to the problem of three-dimensional panel flutter. Inviscid and viscous supersonic results match previous computations using the same aerodynamic method coupled with a finite difference structural solver. For the case of subsonic flow, multiple solutions consisting of static, upward and downward deflections of the panel are discussed. The particular solution obtained is shown to be sensitive to the cavity pressure specified underneath the panel.     

A parallel boundary fitted dynamic mesh solver for applications to flapping flight

A parallel numerical solution procedure for unsteady incompressible flow is developed for simulating the dynamics of flapping flight. A collocated finite volume multiblock approach in a general curvilinear coordinate is used with Cartesian velocities and pressure as dependent variables. The Navier-Stokes equations are solved using a fractional-step algorithm. The dynamic grid algorithm is implemented by satisfying the space conservation law by computing the grid velocities in terms of the volume swept by the faces. The dynamic movement of grid in a multiblock approach is achieved by using a combination of spring analogy and Trans-Finite Interpolation. The spring analogy is used to compute the displacement of block corners, after which Trans-Finite Interpolation is applied independently on each computational block. The performance of the code is validated in forced transverse oscillations of a cylinder in cross-flow, a heaving airfoil, and hovering of a fruitfly. Finally, the unsteady aerodynamics of flapping flight at Re = 10,000 relevant to the development of Micro Air Vehicles is analyzed for forward flight. The results show the capability of the solver in predicting unsteady aerodynamics characterized by complex boundary movements.     

A discontinuous Galerkin method with block cyclic reduction solver for simulating compressible flows on GPUs

An optimized implementation of a block tridiagonal solver based on the block cyclic reduction (BCR) algorithm is introduced and its portability to graphics processing units (GPUs) is explored. The computations are performed on the NVIDIA GTX480 GPU. The results are compared with those obtained on a single core of Intel Core i7-920 (2.67 GHz) in terms of calculation runtime. The BCR linear solver achieves the maximum speedup of 5.84x with block size of 32 over the CPU Thomas algorithm in double precision. The proposed BCR solver is applied to discontinuous Galerkin (DG) simulations on structured grids via alternating direction implicit (ADI) scheme. The GPU performance of the entire computational fluid dynamics (CFD) code is studied for different compressible inviscid flow test cases. For a general mesh with quadrilateral elements, the ADI-DG solver achieves the maximum total speedup of 7.45x for the piecewise quadratic solution over the CPU platform in double precision.     

交替方向隐式CFD解法器的GPU并行计算及其优化

交替方向隐格式(ADI)是常见的偏微分方程离散格式之一,目前对ADI格式在计算流体力学（CFD）实际应用中的GPU并行工作开展较少。从一个有限体积CFD应用出发,通过分析ADI解法器的特点和计算流程,基于统一计算架构(CUDA)编程模型设计了基于网格点与网格线的两类细粒度GPU并行算法,讨论了若干性能优化方法。在天河-1A系统上,采用128×128×128网格规模的单区结构网格算例,无粘项、粘性项及ADI迭代计算的GPU并行性能相对于单CPU核,分别取得了100.1、40.1和10.3倍的加速比,整体ADI CFD解法器的GPU并行加速比为17.3     

On the application of the single-phase level set method to naval hydrodynamic flows

The application of the single-phase level set approach to the numerical simulations of three-dimensional free surface flows around complex geometries, at both non-breaking and breaking regimes is presented. In this approach only the liquid phase is simulated and the level set function is used as tracking device to locate the free surface position. The extrapolation of the solution in the dummy points in the gaseous phase is such that second-order accuracy is maintained also in the points adjacent to the free surface; the time evolution of the level set function and the re-initialization step have been merged so to get a function which is a distance function everywhere, and satisfies, at the same time, the kinematic condition on the free surface. The implementation of this technique into a general purpose Reynolds averaged Navier-Stokes (RANS) equations solver developed at INSEAN [Di Mascio A, Broglia R, Favini B. A Second Order Godunov-type Scheme for Naval Hydrodynamics. Kluwer Academic/Plenum Publishers; 2001, p. 253-61], is described in details; capabilities of the algorithm in dealing with non-breaking and breaking flows in the naval hydrodynamic context will be demonstrated by using a submerged hydrofoil and two different ship hulls in straight course as test cases. Comparisons with both experimental data and numerical surface fitting computations are presented; convergence properties of the algorithm, as well as validation and verification assessment will be also discussed.     

Implementation of a fully balanced periodic tridiagonal solver on a parallel distributed memory architecture

While parallel computers offer significant computational performance, it is generally necessary to evaluate several programming strategies. Two programming strategies for a fairly common problem—a periodic tridiagonal solver—are developed and evaluated. Simple model calculations as well as timing results are presented to evaluate these strategies. The particular tridiagonal solver evaluated is used in many computational fluid dynamic simulation codes. The feature that makes this algorithm unique is that these simulation codes usually require simultaneous solution for multiple right-hand-sides (RHS) of the system of equations. Each RHS solutions is independent and thus can be computed in parallel. Thus, a Gaussian-elimination-type algorithm can be used in a parallel computation and more complicated approaches such as cyclic reduction are not required. The two strategies are a transpose strategy and a distributed solver strategy. For the transpose strategy, the data are moved so that a subset of all the RHS problems is solved on each of the several processors. This usually requires significant data movement between processor memories across a network. The second strategy attempts to have the algorithm follow the data across processor boundaries in a chained manner. This usually requires significantly less data movement. An approach to accomplish this second strategy in a near-perfect load-balanced manner is developed. In addition, an algorithm will be shown to directly transform a sequential Gaussian-elimination-type algorithm into the parallel, chained, load-balanced algorithm.