并行多块结构重叠网格装配算法及应用

针对多块结构重叠网格并行装配的问题,设计了支持初始网格系统细分的多块结构重叠网格框架,并在此框架基础上提出了基于局部洞映射的并行挖洞算法、格心网格下可跨块寻点的并行搜索算法,使之可适应大规模并行数值模拟时的分布式计算环境。此算法被模块化的集成到了自主研发的大规模多块结构网格数值求解器（CCFD-MGMB）中,可支持大规模并行非定常多体分离数值模拟。并行测试结果表明,本文发展的算法具有良好的局部数据结构组织,数据可扩展性强。数值应用模拟结果表明了该算法的有效性及正确性,千核并行非定常数值计算效率（相对于64核）可达58%。     

Low Reynolds number turbulent flows around a dynamically shaped airfoil

A computational investigation for flows surrounding a dynamically shaped airfoil, at a chord Reynolds number of 78,800, is conducted along with a parallel experimental effort. A piezo-actuated flap on the upper surface of a fixed airfoil is adopted for active control. The actuation frequency focused on is 500 Hz. The computational framework consists of a multi-block, moving grid technique, the eⁿ-based laminar-turbulent transition model, the two-equation turbulence closure, and a pressure-based flow solver. The moving grid technique, which handles the geometric variations in time, employs the transfinite interpolation scheme with a spring network approach. Comparing the experimental and computational results for pressure and velocity fields, implications of the detailed flap geometry, the flapping amplitude, turbulence modeling, and grid distributions on the flow structure are assessed. The effect of the flap movement on the separation location and vortex dynamics is also investigated.     

Parallel multigrid solution of the Navier-Stokes equations on general 2D domains

Multigrid methods are distinguished by their optimal (sequential) efficiency and by the fact that all their algorithmical components are fully parallelizable. For this reason, this class of numerical methods is especially attractive for use on parallel (MIMD, local memory) computers. In this paper, we describe a parallel multigrid solver for steady-state incompressible Navier-Stokes equations on general domains which is currently being developed at the GMD. Due to the geometrical generality of the problem, our approach is based on a non-staggered (nodal-point) finite volume scheme on multi-block boundary fitted grids. The typical instability of non-staggered schemes is overcome by suitably modifying the discrete continuity equation without affecting the overall order of consistency.

Starting from the most simple Cartesian case, we discuss several possible multigrid approaches to the general 2D-problem. This motivates the basic design decisions of our multigrid solver in regard to both the discretization and the choice of multigrid components (smoothing schemes). Furthermore, the principal technique of parallelization (grid partitioning) is described as well as some fundamental aspects of the implementation (communication library).     

Parallel computation of unsteady three-dimensional incompressible viscous flow using an unstructured multigrid method

The development and validation of a parallel unstructured tetrahedral non-nested multigrid (MG) method for simulation of unsteady 3D incompressible viscous flow is presented. The Navier-Stokes solver is based on the artificial compressibility method (ACM) and a higher-order characteristics-based finite-volume scheme on unstructured MG. Unsteady flow is calculated with an implicit dual time stepping scheme. The parallelization of the solver is achieved by a MG domain decomposition approach (MG-DD), using the Single Program Multiple Data (SPMD) programming paradigm. The Message-Passing Interface (MPI) Library is used for communication of data and loop arrays are decomposed using the OpenMP standard. The parallel codes using single grid and MG are used to simulate steady and unsteady incompressible viscous flows for a 3D lid-driven cavity flow for validation and performance evaluation purposes. The speedups and efficiencies obtained by both the parallel single grid and MG solvers are reasonably good for all test cases, using up to 32 processors on the SGI Origin 3400. The parallel results obtained agree well with those of serial solvers and with numerical solutions obtained by other researchers, as well as experimental measurements.     

A parallel viscous flow solver on multi-block overset grids

A multi-block overset grid method is presented to accurately simulate viscous flows around complex configurations. A combination of multi-block and overlapping grids is used to discretize the flow domain. A hierarchical grid system with layers of grids of varying resolution is developed to ensure inter-grid connectivity within a framework suitable for multi-grid and parallel computations. At each stage of the numerical computation, information is exchanged between neighboring blocks across either or both matched block boundaries and overlapping boundaries. Coarse-grain parallel processing is facilitated by the multi-block system. Numerical results of flows over multi-element airfoils and three-dimensional turbulent flows around wing–body aerodynamic configurations demonstrate the utility and efficiency of the method.     

A parallel computational framework to solve flow and transport in integrated surface–subsurface hydrologic systems

Hydrologic modeling requires the handling of a wide range of highly nonlinear processes from the scale of a hill slope to the continental scale, and thus the computational efficiency of the model becomes a critical issue for water resource management. This work is aimed at implementing and evaluating a flexible parallel computing framework for hydrologic simulations by applying OpenMP in the HydroGeoSphere (HGS) model. HGS is a 3D control-volume finite element model that solves the nonlinear coupled equations describing surface–subsurface water flow, solute migration and energy transport. The computing efficiency of HGS is improved by three parallel computing schemes: 1) parallelization of Jacobian matrix assembly, 2) multi-block node reordering for performing LU solve efficiently, and 3) parameter privatization for reducing memory access latency. Regarding to the accuracy and consistency of the simulation solutions obtained with parallel computing, differences in the solutions are entirely due to use of a finite linear solver iteration tolerance, which produces slightly different solutions which satisfy the convergence tolerance. The maximum difference in the head solution between the serial and parallel simulations is less than 10⁻³ m, using typical convergence tolerances. Using the parallel schemes developed in this work, three key achievements can be summarized: (1) parallelization of a physically-based hydrologic simulator can be performed in a manner that allows the same code to be executed on various shared memory platforms with minimal maintenance; (2) a general, flexible and robust parallel iterative sparse-matrix solver can be implemented in a wide range of numerical models employing either structured or unstructured mesh; and (3) the methodology is flexible, especially for the efficient construction of the coefficient and Jacobian matrices, compared to other parallelized hydrologic models which use parallel library packages.     

Incompressible flow computations over moving boundary using a novel upwind method

In this paper a novel method for simulating unsteady incompressible viscous flow over a moving boundary is described. The numerical model is based on a 2D Navier–Stokes incompressible flow in artificial compressibility formulation with Arbitrary Lagrangian Eulerian approach for moving grid and dual time stepping approach for time accurate discretization. A higher order unstructured finite volume scheme, based on a Harten Lax and van Leer with Contact (HLLC) type Riemann solver for convective fluxes, developed for steady incompressible flow in artificial compressibility formulation by Mandal and Iyer (AIAA paper 2009-3541), is extended to solve unsteady flows over moving boundary. Viscous fluxes are discretized in a central differencing manner based on Coirier’s diamond path. An algorithm based on interpolation with radial basis functions is used for grid movements. The present numerical scheme is validated for an unsteady channel flow with a moving indentation. The present numerical results are found to agree well with experimental results reported in literature.     

A block LU-SGS implicit unsteady incompressible flow solver on hybrid dynamic grids for 2D external bio-fluid simulations

A hybrid dynamic grid generation technique for two-dimensional (2D) morphing bodies and a block lower-upper symmetric Gauss-Seidel (BLU-SGS) implicit dual-time-stepping method for unsteady incompressible flows are presented for external bio-fluid simulations. To discretize the complicated computational domain around 2D morphing configurations such as fishes and insect/bird wings, the initial grids are generated by a hybrid grid strategy firstly. Body-fitted quadrilateral (quad) grids are generated first near solid bodies. An adaptive Cartesian mesh is then generated to cover the entire computational domain. Cartesian cells which overlap the quad grids are removed from the computational domain, and a gap is produced between the quad grids and the adaptive Cartesian grid. Finally triangular grids are used to fill this gap. During the unsteady movement of morphing bodies, the dynamic grids are generated by a coupling strategy of the interpolation method based on ‘Delaunay graph’ and local remeshing technique. With the motion of moving/morphing bodies, the grids are deformed according to the motion of morphing body boundaries firstly with the interpolation strategy based on ‘Delaunay graph’ proposed by Liu and Qin. Then the quality of deformed grids is checked. If the grids become too skewed, or even intersect each other, the grids are regenerated locally. After the local remeshing, the flow solution is interpolated from the old to the new grid. Based on the hybrid dynamic grid technique, an efficient implicit finite volume solver is set up also to solve the unsteady incompressible flows for external bio-fluid dynamics. The fully implicit equation is solved using a dual-time-stepping approach, coupling with the artificial compressibility method (ACM) for incompressible flows. In order to accelerate the convergence history in each sub-iteration, a block lower-upper symmetric Gauss-Seidel implicit method is introduced also into the solver. The hybrid dynamic grid generator is tested by a group of cases of morphing bodies, while the implicit unsteady solver is validated by typical unsteady incompressible flow case, and the results demonstrate the accuracy and efficiency of present solver. Finally, some applications for fish swimming and insect wing flapping are carried out to demonstrate the ability for 2D external bio-fluid simulations.     

An integrated media, integrated processes watershed model

The motivation of this work is to carry out parallel simulations of incompressible flows on block-structured meshes. A new partitioning method is proposed. The quality of rectangular partitions is checked and compared with other methods, as regards load balance, edge-cut and block numbers. The partitioner is coupled with the massively parallel Hypre solver library and efficiency of the coupling is measured. Finally, the code is applied to study laminar flows (steady and unsteady) on three non-rectangular geometries. Very fine grids are used to compute reference solutions of a Z-shaped channel flow and the L-shaped and double lid driven cavities.     

A parallel multigrid solver for incompressible flows on computing architectures with accelerators

An efficient parallel multigrid pressure correction algorithm is proposed for the solution of the incompressible Navier–Stokes equations on computing architectures with acceleration devices. The pressure correction procedure is based on the numerical solution of a Poisson-type problem, which is discretized using a fourth-order finite difference compact scheme. Since this is the most time-consuming part of the solver, we propose a parallel pressure correction algorithm using an iterative method based on a block cyclic reduction solution method combined with a multigrid technique. The grid points are numbered with respect to the red–black ordering scheme for the parallel Gauss–Seidel smoother. These parallelization techniques allow the execution of the entire simulation computations on the acceleration device, minimizing memory communication costs. The realization is developed using the OpenACC API, and the numerical method is demonstrated for the solution of two classical incompressible flow test problems. The first is the two-dimensional lid-driven cavity problem over equal mesh sizes while the other is the Stokes boundary layer, which is a decent benchmark problem for unequal mesh spacing. The effect of several multigrid components on modern and legacy acceleration architectures is examined. Eventually the performance investigation demonstrates that the proposed parallel multigrid solver achieves an acceleration of more than 10\(\times \) over the sequential solver and more than 4\(\times \) over multi-core CPU only realizations for all tested accelerators.     

基于多块结构网格大规模并行计算的负载均衡设计及实现

结构网格具有网格生成速度快、质量高、数据结构相对简单、较适用于流体表面应力集中的运算等优点。在大规模 CFD（Computational Fluid Dynamics）并行计算中,需要将网格区域划分为多块网格,而多块网格之间的数据通信会制约并行计算能力的提高,因此对结构网格的负载平衡优化是提高并行计算能力的重点。本文提出了一个采用多层次 k-way 多约束条件图剖分算法来改进负载平衡的方案,并对 M6 翼型和 CRM 模型的多种规模进行了实际计算,结果证明多层次 k-way图剖分算法能够有效地优化负载平衡,在此基础上得到了最优节点间的计算负载平衡和通信负载平衡,最终达到了理想的并行效率。     

Implementation of density-based solver for all speeds in the framework of OpenFOAM

In the framework of open source CFD code OpenFOAM, a density-based solver for all speeds flow field is developed. In this solver the preconditioned all speeds AUSM+(P) scheme is adopted and the dual time scheme is implemented to complete the unsteady process. Parallel computation could be implemented to accelerate the solving process. Different interface reconstruction algorithms are implemented, and their accuracy with respect to convection is compared. Three benchmark tests of lid-driven cavity flow, flow crossing over a bump, and flow over a forward-facing step are presented to show the accuracy of the AUSM+(P) solver for low-speed incompressible flow, transonic flow, and supersonic/hypersonic flow. Firstly, for the lid driven cavity flow, the computational results obtained by different interface reconstruction algorithms are compared. It is indicated that the one dimensional reconstruction scheme adopted in this solver possesses high accuracy and the solver developed in this paper can effectively catch the features of low incompressible flow. Then via the test cases regarding the flow crossing over bump and over forward step, the ability to capture characteristics of the transonic and supersonic/hypersonic flows are confirmed. The forward-facing step proves to be the most challenging for the preconditioned solvers with and without the dual time scheme. Nonetheless, the solvers described in this paper reproduce the main features of this flow, including the evolution of the initial transient.     

大规模并行 CFD 软件的负载平衡设计

基于多区结构网格的计算流体力学方法,在并行处理的难点是多个网格数据块在计算资源上的高效合理分配,以实现大规模并行环境下的负载平衡。本文围绕负载平衡问题,介绍了 CCFD 软件开展的一些工作,包括：1. 面向结构网格的双层图剖分策略,通过细层图剖分环节考虑计算量和通信量的负载平衡;2. 建立可细分的重叠网格体系,并基于该体系建立了重叠网格系统的双级负载平衡模型。算例验证表明,所采用的负载平衡策略在大规模并行环境下能获得较高并行效率。     

Parallelization of a multi-blocked CFD code via three strategies for fluid flow and heat transfer analysis

This paper reports on a parallel implementation of a general 3D multi-block CFD code. The parallelization is achieved by using three strategies. Firstly, it is done on dual-processor PC-clusters where Windows NT systems are running. A multi-thread programming model is adopted for the multi-block code, where one thread corresponds to a block. Shared-memory is used for the exchange of inner-boundaries between neighboring blocks (threads) on the same node, while WinSockets are employed for those on different nodes. Secondly, the parallelization is extended to UNIX operating system. MPI is applied for all the message passing between different processors, including those on the same node. Thirdly, Pthreads (POSIX threads), a standardized application interface for threads, are adopted to take the advantage of the shared-memory feature of the SMP nodes, while MPI is only applied for the message passing between processors on different nodes. In all the strategies, a static load-balancing method is employed for equitable distribution of computational work to specified nodes. The parameters of the present code is studied in detail to facilitate the explanation of the speedup results. Two examples are provided to show the speedup and load balancing of the parallel calculation. Detailed comparison is made to evaluate the efficiency of different strategies.     

混合重叠网格通信优化算法

重叠网格技术广泛应用在复杂外型和运动边界问题的流场数值模拟中.本文在并行重叠网格隐式挖洞算法实现的基础上,提出了笛卡尔辅助网格和多块结构网格的混合重叠网格方法.通过笛卡尔辅助网格实现重叠网格洞边界和网格插值关系的快速建立.通过定义重叠区域网格权重、部件网格与背景网格绑定的方法,建立了混合网格的并行分配模式,有效减少重叠插值信息在各进程间的通信,实现计算负载和通信负载在各个进程的均匀分配.测试表明该方法可应用于数千万量级的重叠网格系统,可扩展至千核规模,高效的实现多个物体构成的复杂网格系统的重叠关系建立.     

High-performance computing of wind turbine aerodynamics using isogeometric analysis

In this article we present a high-performance computing framework for advanced flow simulation and its application to wind energy based on the residual-based variational multiscale (RBVMS) method and isogeometric analysis. The RBVMS formulation and its suitability and accuracy for turbulent flow in a moving domain are presented. Particular emphasis is placed on the parallel implementation of the methodology and its scalability. Two challenging flow cases were considered: the turbulent Taylor–Couette flow and the NREL 5 MW offshore baseline wind turbine rotor at full scale. In both cases, flow quantities of interest from the simulation results compare favorably with the reference data and near-perfect linear parallel scaling is achieved.     

基于结构网格的高超声速流动大规模并行数值模拟研究

本文采用MPI消息传递模式自主开发出适用于高超声速流动数值模拟的并行计算软件,该软件以三维Navier-Stokes方程为基本控制方程来求解层流问题,应用基于结构网格的有限体积法对计算域进行离散,采用AUSMPW+格式求解对流通量,利用MUSCL插值方法获得高阶精度,时间格式上采用LU-SGS方法进行时间迭代以加快求解定常流动的收敛过程。在高性能计算机上针对不同高超声速流动进行大规模并行计算的结果表明,所开发的CFD并行计算软件具有较高的并行计算效率,为高超声速飞行器气动力/热的准确预测提供了高效工具。     

Parallel computation for natural convection

Parallel computation for two-dimensional convective flows in cavities with adiabatic horizontal boundaries and driven by differential heating of the two vertical end walls are investigated using the Intel Paragon, Intel Touchstone Delta, Cray T3D and IBM SP2. The numerical scheme, including a parallel multigrid solver, and domain decomposition techniques for parallel computing are discussed in detail. Performance comparisons are made for the different parallel systems, and numerical results using various numbers of processors are discussed. © 1997 John Wiley & Sons, Ltd.     

A unifying grid approach for solving potential flows applicable to structured and unstructured grid configurations

In this study, an efficient numerical method is proposed for unifying the structured and unstructured grid approaches for solving the potential flows. The new method, named as the “alternating cell directions implicit - ACDI”, solves for the structured and unstructured grid configurations equally well. The new method in effect applies a line implicit method similar to the Line Gauss Seidel scheme for complex unstructured grids including mixed type quadrilateral and triangle cells. To this end, designated alternating directions are taken along chains of contiguous cells, i.e. ‘cell directions’, and an ADI-like sweeping is made to update these cells using a Line Gauss Seidel like scheme. The algorithm makes sure that the entire flow field is updated by traversing each cell twice at each time step for unstructured quadrilateral grids that may contain triangular cells. In this study, a cell-centered finite volume formulation of the ACDI method is demonstrated. The solutions are obtained for incompressible potential flows around a circular cylinder and a forward step. The results are compared with the analytical solutions and numerical solutions using the implicit ADI and the explicit Runge-Kutta methods on single-and multi-block structured and unstructured grids. The results demonstrate that the present ACDI method is unconditionally stable, easy to use and has the same computational performance in terms of convergence, accuracy and run times for both the structured and unstructured grids.