One-dimensional electromagnetic relativistic PIC-hydrodynamic hybrid simulation code H-VLPL (hybrid virtual laser plasma lab)

In this paper we present a new one-dimensional full electromagnetic relativistic hybrid plasma model. The full kinetic particle-in-cell (PIC) and hydrodynamic model have been combined in the single hybrid plasma code H-VLPL (hybrid virtual laser plasma laboratory). The semi-implicit algorithm allows to simulate plasmas of arbitrary densities via automatic reduction of the highest plasma frequencies down to the numerically stable range. At the same time, the model keeps the correct spatial scales like the plasma skin depth. We discuss the numerically efficient implementation of this model. Further, we carefully test the hybrid model validity by applying it to a series of physical examples. The new mathematical method allows to overcome the typical time step restrictions of explicit PIC codes.     

BOUT++: A framework for parallel plasma fluid simulations

A new modular code called BOUT++ is presented, which simulates 3D fluid equations in curvilinear coordinates. Although aimed at simulating Edge Localised Modes (ELMs) in tokamak x-point geometry, the code is able to simulate a wide range of fluid models (magnetised and unmagnetised) involving an arbitrary number of scalar and vector fields, in a wide range of geometries. Time evolution is fully implicit, and 3rd-order WENO schemes are implemented. Benchmarks are presented for linear and non-linear problems (the Orszag-Tang vortex) showing good agreement. Performance of the code is tested by scaling with problem size and processor number, showing efficient scaling to thousands of processors.Linear initial-value simulations of ELMs using reduced ideal MHD are presented, and the results compared to the ELITE linear MHD eigenvalue code. The resulting mode-structures and growth-rate are found to be in good agreement (γBOUT++=0.245ωA, γELITE=0.239ωA, with Alfvénic timescale 1/ωA=R/VA). To our knowledge, this is the first time dissipationless, initial-value simulations of ELMs have been successfully demonstrated.     

A parallel algorithm for global magnetic reconnection studies

A new algorithm is presented for the computation of two-dimensional magnetic reconnection in plasmas. Both resistive magnetohydrodynamic (MHD) and two-fluid models are considered. It has been implemented on several parallel platforms and shows good scalability up to 32 CPUs for reasonable problem sizes. A fixed, non-uniform rectangular mesh is used to resolve the different spatial scales in the reconnection problem. The resistive MHD version uses an implicit/explicit hybrid method, while the two-fluid version uses an alternating-direction implicit (ADI) method with high-order artificial dissipation. The technique has proven useful for comparing several different theories of collisional and collisionless reconnection.     

A load-balancing algorithm for a parallel electromagnetic particle-in-cell code

Particle-in-cell simulations often suffer from load-imbalance on parallel machines due to the competing requirements of the field-solve and particle-push computations. We propose a new algorithm that balances the two computations independently. The grid for the field-solve computation is statically partitioned. The particles within a processor's sub-domain(s) are dynamically balanced by migrating spatially-compact groups of particles from heavily loaded processors to lightly loaded ones as needed. The algorithm has been implemented in the quicksilver electromagnetic particle-in-cell code. We provide details of the implementation and present performance results for quicksilver running models with up to a billion grid cells and particles on thousands of processors of a large distributed-memory parallel machine.     

On a semi-implicit scheme for spectral simulations of dispersive magnetohydrodynamics

The effectiveness of a semi-implicit (SI) temporal scheme is discussed in the context of the dispersive magnetohydrodynamics where, due to the whistler modes, stability of explicit algorithms requires a time step decreasing quadratically as the resolution is linearly increased. After analyzing the effects of this scheme on the Alfvén-wave dispersion relation, spectral simulations of nonlinear initial value problems where small-scale dispersion has a main effect on the global dynamics are presented. Permitting a moderate, albeit significant, increase of the time step for a minor additional cost relatively to explicit schemes, the SI algorithm provides an efficient tool in situations, such as turbulent regimes, where the time steps making fully implicit schemes efficient are too large to ensure a satisfactory accuracy.     

A revised δf algorithm for nonlinear PIC simulation

An equation for the evolution of δf is shown to be redundant in the δf particle-in-cell (PIC) simulation scheme. Having eliminated this equation, an adaptive f₀ construction is shown to follow intuitively and to be straight forward to implement.     

A semi-Lagrangian code for nonlinear global simulations of electrostatic drift-kinetic ITG modes

A semi-Lagrangian code for the solution of the electrostatic drift-kinetic equations in straight cylinder configuration is presented. The code, CYGNE, is part of a project with the long term aim of studying microturbulence in fusion devices. The code has been constructed in such a way as to preserve a good control of the constants of motion, possessed by the drift-kinetic equations, until the nonlinear saturation of the ion-temperature-gradient modes occurs. Studies of convergence with phase space resolution and time-step are presented and discussed. The code is benchmarked against electrostatic Particle-in-Cell codes.     

FLY. A parallel tree N-body code for cosmological simulations

FLY is a parallel treecode which makes heavy use of the one-sided communication paradigm to handle the management of the tree structure. In its public version the code implements the equations for cosmological evolution, and can be run for different cosmological models.This reference guide describes the actual implementation of the algorithms of the public version of FLY, and suggests how to modify them to implement other types of equations (for instance, the Newtonian ones).

Program summary

Title of program:FLYCatalogue identifier: ADSCProgram summary URL:http://cpc.cs.qub.ac.uk/summaries/ADSCProgram obtainable from: CPC Program Library, Queen's University of Belfast, N. IrelandComputer for which the program is designed and others on which it has been tested: Cray T3E, Sgi Origin 3000, IBM SPOperating systems or monitors under which the program has been tested: Unicos 2.0.5.40, Irix 6.5.14, Aix 4.3.3Programming language used: Fortran 90, CMemory required to execute with typical data: about 100 Mwords with 2 million-particlesNumber of bits in a word: 32Number of processors used: parallel program. The user can select the number of processors ?1Has the code been vectorized or parallelized?: parallelizedNumber of bytes in distributed program, including test data, etc.: 4 615 604Distribution format: tar gzip fileKeywords: Parallel tree N-body code for cosmological simulationsNature of physical problem:FLY is a parallel collisionless N-body code for the calculation of the gravitational force.Method of solution: It is based on the hierarchical oct-tree domain decomposition introduced by Barnes and Hut (1986).Restrictions on the complexity of the program: The program uses the leapfrog integrator schema, but could be changed by the user.Typical running time: 50 seconds for each time-step, running a 2-million-particles simulation on an Sgi Origin 3800 system with 8 processors having 512 Mbytes RAM for each processor.Unusual features of the program:FLY uses the one-side communications libraries: the SHMEM library on the Cray T3E system and Sgi Origin system, and the LAPI library on IBM SP system     

A transport simulation code for inertial confinement fusion relevant laser-plasma interaction

We present a code for the simulation of laser-plasma interaction processes relevant for applications in inertial confinement fusion. The code consists of a fully nonlinear hydrodynamics in two spatial dimensions using a Lagrangian, discontinuous Galerkin-type approach, a paraxial treatment of the laser field and a spectral treatment of the dominant non-local transport terms. The code is fully parallelized using MPI in order to be able to simulate macroscopic plasmas.One example of a fully nonlinear evolution of a laser beam in an underdense plasma is presented for the conditions previewed for the future MegaJoule laser project.     

A massively parallel,multi-disciplinary Barnes–Hut tree code for extreme-scale N-body simulations

The efficient parallelization of fast multipole-based algorithms for the N-body problem is one of the most challenging topics in high performance scientific computing. The emergence of non-local, irregular communication patterns generated by these algorithms can easily create an insurmountable bottleneck on supercomputers with hundreds of thousands of cores. To overcome this obstacle we have developed an innovative parallelization strategy for Barnes–Hut tree codes on present and upcoming HPC multicore architectures. This scheme, based on a combined MPI–Pthreads approach, permits an efficient overlap of computation and data exchange. We highlight the capabilities of this method on the full IBM Blue Gene/P system JUGENE at Jülich Supercomputing Centre and demonstrate scaling across 294,912 cores with up to 2,048,000,000 particles. Applying our implementation pepc to laser–plasma interaction and vortex particle methods close to the continuum limit, we demonstrate its potential for ground-breaking advances in large-scale particle simulations.     

A new simulation code for particle diffusion in anisotropic, large-scale and turbulent magnetic fields

A new Monte Carlo code is presented that simulates the scattering processes of energetic particles in turbulent magnetic fields. The growing number of analytical models for anisotropic turbulence geometries gives rise to the need of fast and adaptable simulation codes that, in order to be able to judge the accuracy of the results, calculate the estimated mean errors of the transport parameters. Furthermore, the need to understand the interplay of scattering in anisotropic large-scale (such as the solar magnetic field) and turbulent (such as the Maltese cross-like structured solar wind turbulence) magnetic fields is accounted for with the calculation of the off-diagonal elements of the diffusion tensor.     

Partitioning 3D space for parallel many-particle simulations

In a common approach for parallel processing applied to simulations of many-particle systems with short-ranged interactions and uniform density, the cubic simulation box is partitioned into domains of equal shape and size, each of which is assigned to one processor. We compare the commonly used simple-cubic (SC) domain shape to domain shapes chosen as the Voronoi cells of BCC, FCC, and HCP sphere packings. The latter three are found to result in superior partitionings with respect to communication overhead. Scaling of the domain shape is used to extend the range of applicability of these partitionings to a large set of processor numbers. The higher efficiency with BCC and FCC partitionings is demonstrated in simulations of the sillium model for amorphous silicon.     

MUPHY: A parallel MUlti PHYsics/scale code for high performance bio-fluidic simulations

We present a parallel version of MUPHY, a multi-physics/scale code based upon the combination of microscopic Molecular Dynamics (MD) with a hydro-kinetic Lattice Boltzmann (LB) method. The features of the parallel version of MUPHY are hereby demonstrated for the case of translocation of biopolymers through nanometer-sized, multi-pore configurations, taking into explicit account the hydrodynamic interactions of the translocating molecules with the surrounding fluid. The parallel implementation exhibits excellent scalability on the IBM BlueGene platform and includes techniques which may improve the flexibility and efficiency of other complex multi-physics parallel applications, such as hemodynamics, targeted-drug delivery and others.     

A nonuniform nested grid method for simulations of RF induced ionospheric turbulence

We present a numerical scheme to simulate radio-frequency (RF) induced ionospheric turbulence, in which an electromagnetic wave is injected into the overhead ionospheric plasma. At the turning point of the ordinary mode, the electromagnetic wave undergoes linear mode-conversion to electrostatic Langmuir and upper hybrid waves that can have a much shorter wavelength than the electromagnetic wave. In order to resolve both the electromagnetic and electrostatic waves, avoiding severe restrictions on the time step due to the Courant-Friedrich-Lewy (CFL) condition, the equation of motion for the plasma particles is solved on a denser grid than that for the Maxwell equations near the mode-conversion region. An interpolation scheme is employed to calculate the electromagnetic field in the equation of motion of the plasma particles, and an averaging scheme is used to calculate the current density acting as a source in the Maxwell equation. Special care has to be taken to reduce numerical recurrence effects when the wavelength of the electrostatic wave is of the same order or shorter than the coarse grid spacing of the electromagnetic wave.     

JuNoLo - Jülich nonlocal code for parallel post-processing evaluation of vdW-DF correlation energy

Nowadays the state of the art Density Functional Theory (DFT) codes are based on local (LDA) or semilocal (GGA) energy functionals. Recently the theory of a truly nonlocal energy functional has been developed. It has been used mostly as a post-DFT calculation approach, i.e. by applying the functional to the charge density calculated using any standard DFT code, thus obtaining a new improved value for the total energy of the system. Nonlocal calculation is computationally quite expensive and scales as N² where N is the number of points in which the density is defined, and a massively parallel calculation is welcome for a wider applicability of the new approach. In this article we present a code which accomplishes this goal.

Program summary

Program title: JuNoLoCatalogue identifier: AEFM_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEFM_v1_0.htmlProgram obtainable from: CPC Program Library, Queen's University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 176 980No. of bytes in distributed program, including test data, etc.: 2 126 072Distribution format: tar.gzProgramming language: Fortran 90Computer: any architecture with a Fortran 90 compilerOperating system: Linux, AIXHas the code been vectorised or parallelized?: Yes, from 1 to 65536 processors may be used.RAM: depends strongly on the problem's size.Classification: 7.3External routines:• FFTW (http://www.tw.org/)• MPI (http://www.mcs.anl.gov/research/projects/mpich2/ or http://www.lam-mpi.org/)Nature of problem: Obtaining the value of the nonlocal vdW-DF energy based on the charge density distribution obtained from some Density Functional Theory code.Solution method: Numerical calculation of the double sum is implemented in a parallel F90 code. Calculation of this sum yields the required nonlocal vdW-DF energy.Unusual features: Binds to virtually any DFT program.Additional comments: Excellent parallelization features.Running time: Depends strongly on the size of the problem and the number of CPUs used.     

A massively parallel hybrid scheme for direct numerical simulation of turbulent viscoelastic channel flow

This paper describes in detail a numerical scheme designed for direct numerical simulation (DNS) of turbulent drag reduction. The hybrid spatial scheme includes Fourier spectral accuracy in two directions and sixth-order compact finite differences for first and second-order wall-normal derivatives, while time marching can be up to fourth-order accurate. High-resolution and high-drag reduction viscoelastic DNS are made possible through domain decomposition with a two-dimensional MPI Cartesian grid alternatively splitting two directions of space (‘pencil’ decomposition). The resulting algorithm has been shown to scale properly up to 16384 cores on the Blue Gene/P at IDRIS–CNRS, France.Drag reduction is modeled for the three-dimensional wall-bounded channel flow of a FENE-P dilute polymer solution which mimics injection of heavy-weight flexible polymers in a Newtonian solvent. We present results for four high-drag reduction viscoelastic flows with friction Reynolds numbers Re_τ0 = 180, 395, 590 and 1000, all of them sharing the same friction Weissenberg number We_τ0 = 115 and the same rheological parameters. A primary analysis of the DNS database indicates that turbulence modification by the presence of polymers is Reynolds-number dependent. This translates into a smaller percent drag reduction with increasing Reynolds number, from 64% at Re_τ0 = 180 down to 59% at Re_τ0 = 1000, and a steeper mean current at small Reynolds number. The Reynolds number dependence is also visible in second-order statistics and in the vortex structures visualized with iso-surfaces of the Q-criterion.     

A parallel hierarchical-element method for contour dynamics simulations

Many two-dimensional incompressible inviscid vortex flows can be simulated very efficiently by means of the contour dynamics method. Several applications require the use of a hierarchical-element method (HEM), which is a modified version of the classical contour dynamics scheme based on the fast multipole method. The HEM can be used, for example, to study the large-scale motion of coherent structures in idealized geophysical fluid dynamics where the flow can be modelled as the motion in a thin layer of fluid in the presence of a non-uniform background rotation. Nevertheless, such simulations require a substantial computational effort, even when the HEM is used. In this article it is shown that the acceleration of contour dynamics simulations can be increased further by parallelizing the HEM algorithm. Speed-up, load balance and scalability are parallel performance features which are studied for several representative cases. The HEM has been parallelized using the OpenMP interface and is tested with up to 16 processors on an Origin 3800 CC-NUMA computer.     

FLY: MPI-2 high resolution code for LSS cosmological simulations

Cosmological simulations of structures and galaxies formations have played a fundamental role in the study of the origin, formation and evolution of the Universe. These studies improved enormously with the use of supercomputers and parallel systems and, recently, grid based systems and Linux clusters. Now we present the new version of the tree N-body parallel code FLY that runs on a PC Linux Cluster using the one side communication paradigm MPI-2 and we show the performances obtained. FLY is included in the Computer Physics Communication Program Library. This new version was developed using the Linux Cluster of CINECA, an IBM Cluster with 1024 Intel Xeon Pentium IV 3.0 GHz. The results show that it is possible to run a 64 million particle simulation in less than 15 minutes for each time-step, and the code scalability with the number of processors is achieved. This leads us to propose FLY as a code to run very large N-body simulations with more than 10⁹ particles with the higher resolution of a pure tree code. The FLY new version is available at the CPC Program Library, http://cpc.cs.qub.ac.uk/summaries/ADSC_v2_0.html [U. Becciani, M. Comparato, V. Antonuccio-Delogu, Comput Phys. Comm. 174 (2006) 605].