Parallel computational steering for HPC applications using HDF5 files in distributed shared memory

Interfacing a GUI driven visualization/analysis package to an HPC application enables a supercomputer to be used as an interactive instrument. We achieve this by replacing the IO layer in the HDF5 library with a custom driver which transfers data in parallel between simulation and analysis. Our implementation using ParaView as the interface, allows a flexible combination of parallel simulation, concurrent parallel analysis, and GUI client, either on the same or separate machines. Each MPI job may use different core counts or hardware configurations, allowing fine tuning of the amount of resources dedicated to each part of the workload. By making use of a distributed shared memory file, one may read data from the simulation, modify it using ParaView pipelines, write it back, to be reused by the simulation (or vice versa). This allows not only simple parameter changes, but complete remeshing of grids, or operations involving regeneration of field values over the entire domain. To avoid the problem of manually customizing the GUI for each application that is to be steered, we make use of XML templates that describe outputs from the simulation (and inputs back to it) to automatically generate GUI controls for manipulation of the simulation.     

Asymmetric tensor field visualization for surfaces

Asymmetric tensor field visualization can provide important insight into fluid flows and solid deformations. Existing techniques for asymmetric tensor fields focus on the analysis, and simply use evenly-spaced hyperstreamlines on surfaces following eigenvectors and dual-eigenvectors in the tensor field. In this paper, we describe a hybrid visualization technique in which hyperstreamlines and elliptical glyphs are used in real and complex domains, respectively. This enables a more faithful representation of flow behaviors inside complex domains. In addition, we encode tensor magnitude, an important quantity in tensor field analysis, using the density of hyperstreamlines and sizes of glyphs. This allows colors to be used to encode other important tensor quantities. To facilitate quick visual exploration of the data from different viewpoints and at different resolutions, we employ an efficient image-space approach in which hyperstreamlines and glyphs are generated quickly in the image plane. The combination of these techniques leads to an efficient tensor field visualization system for domain scientists. We demonstrate the effectiveness of our visualization technique through applications to complex simulated engine fluid flow and earthquake deformation data. Feedback from domain expert scientists, who are also co-authors, is provided.     

Fluid–structure interaction for non-conforming interfaces based on a dual mortar formulation

In the present work the problem of fluid–structure interaction (FSI) with independently space discretized fluid and structure fields is addressed in the context of finite elements. To be able to deal with non-conforming meshes at the fluid–structure interface, we propose the integration of a dual mortar method into the general FSI framework. This method has lately been used successfully to impose interface constraints in other contexts such as finite deformation contact. The main focus is set on monolithic coupling algorithms for FSI here. In these cases the dual mortar approach allows for the elimination of the additional Lagrange multiplier degrees of freedom from the global system by condensation. The resulting system matrices have the same block structure as their counterparts for the conforming case and permit the same numerical treatment. Partitioned Dirichlet–Neumann coupling is also considered briefly and it is shown that the dual mortar approach permits a numerically efficient mapping between fluid and structure quantities at the interface.Numerical examples demonstrate the efficiency and robustness of the proposed method. We present results for a variety of different element formulations for the fluid and the structure field, indicating that the proposed method is not limited to any specific formulation. Furthermore, the applicability of state-of-the-art iterative solvers is considered and the convergence behavior is shown to be comparable to standard simulations with conforming discretizations at the interface.     

A numerical study on the fluid compressibility effects in strongly coupled fluid–solid interaction problems

Interactions between an incompressible fluid passing through a flexible tube and the elastic wall is one of the strongly coupled fluid–solid interaction (FSI) problems frequently studied in the literature due to its research importance and wide range of applications. Although incompressible fluid is a prevalent model in many simulation studies, the assumption of incompressibility may not be appropriate in strongly coupled FSI problems. This paper narrowly aims to study the effect of the fluid compressibility on the wave propagation and fluid–solid interactions in a flexible tube. A partitioned FSI solver is used which employs a finite volume-based fluid solver. For the sake of comparison, both traditional incompressible (ico) and weakly compressible (wco) fluid models are used in an Arbitrary Lagrangian–Eulerian (ALE) formulation and a PISO-like algorithm is used to solve the unsteady flow equations on a collocated mesh. The solid part is modeled as a simple hyperelastic material obeying the St-Venant constitutive relation. Computational results show that not only use of the weakly compressible fluid model makes the FSI solver in this case more efficient, but also the incompressible fluid model may produce largely unrealistic computational results. Therefore, the use of the weakly compressible fluid model is suggested for strongly coupled FSI problems involving seemingly incompressible fluids such as water especially in cases where wave propagation in the solid plays an important role.

     

A mesh adaptivity procedure for CFD and fluid-structure interactions

We present a procedure to adapt and repair meshes in the general solution of Navier–Stokes incompressible and compressible fluid flows, including structural interactions. For fluid-structure interactions, FSI, the fluid is described by an arbitrary-Lagrangian–Eulerian formulation fully coupled to general solids and structures described by Lagrangian formulations. The solids and structures can undergo highly nonlinear response due to large deformations, nonlinear material behavior, contact and temperature. We focus on the need to adapt the fluid mesh in pure CFD solutions when high gradients are present or boundary layer effects are important, and FSI solutions when large structural deformations take place. The procedure is a practical scheme to solve complex problems. We illustrate the proposed scheme in various example solutions.     

Three-dimensional fluid-structure coupling in transient analysis

The numerical simulation of nonlinear, transient fluid-structure interactions (FSI) is a current area of concern by researchers in various fields, including the field of nuclear reactor safety. This paper primarily discusses the formulation used in an algorithm that couples three-dimensional hydrodynamic and structural domains. Here, both the fluid and structure are discretized using finite elements. The semi-discretized equations of motion are solved using an explicit temporal integrator.Coupling is accomplished by satisfying interface mechanics. The structure imposes kinematic constraints to the moving fluid boundary, and the fluid in turn provides an external loading on the structure. At each interface node, normals are computed from the nodal basis functions of only the hydrodynamic nodes. By defining the interface normal in this manner, it becomes independent of the type of structural boundary (i.e. shell, plate, continuum, etc.) and thus makes this aspect of the coupling independent of the structure type. A penalty type gap-impact element is developed to model the impact region between the fluid and structure.Results for several problems are presented and these include a comparison between analytical results for a FSI problem and numerical predictions.     

Object-oriented visualization

Feature based techniques incorporated into standard visualization algorithms can greatly enhance the quantification and visualization of observed phenomena, as described in the article. The methods to isolate and recognize coherent 3D structures are analogous to 2D vision techniques. The overall goals are the same in both fields, namely, to interpret an image (data) and construct a model to describe it. Although the article uses data sets from numerical simulations of fluid flow, the concepts are applicable to other domains where scientists study the evolution and morphologies of 4D space time vector and scalar fields. More work is needed to explore complex features based upon domain specific knowledge and to define the parameters for classification and tracking. Sophisticated databases for storage and retrieval of feature based data sets are also an interesting area of study. The ultimate goal of visualization is to aid in the understanding and analysis of data. With faster parallel computers and more sophisticated laboratory equipment, information is being produced in ever greater amounts. This information must be presented to the scientist in a form suitable for cogent assimilation and manipulation. The article presents issues and algorithms for an object oriented approach to this problem and demonstrates its usefulness for visualization     

Time dependent processing in a parallel pipeline architecture

Pipeline architectures provide a versatile and efficient mechanism for constructing visualizations, and they have been implemented in numerous libraries and applications over the past two decades. In addition to allowing developers and users to freely combine algorithms, visualization pipelines have proven to work well when streaming data and scale well on parallel distributed-memory computers. However, current pipeline visualization frameworks have a critical flaw: they are unable to manage time varying data. As data flows through the pipeline, each algorithm has access to only a single snapshot in time of the data. This prevents the implementation of algorithms that do any temporal processing such as particle tracing; plotting over time; or interpolation, fitting, or smoothing of time series data. As data acquisition technology improves, as simulation time-integration techniques become more complex, and as simulations save less frequently and regularly, the ability to analyze the time-behavior of data becomes more important. This paper describes a modification to the traditional pipeline architecture that allows it to accommodate temporal algorithms. Furthermore, the architecture allows temporal algorithms to be used in conjunction with algorithms expecting a single time snapshot, thus simplifying software design and allowing adoption into existing pipeline frameworks. Our architecture also continues to work well in parallel distributed-memory environments. We demonstrate our architecture by modifying the popular VTK framework and exposing the functionality to the ParaView application. We use this framework to apply time-dependent algorithms on large data with a parallel cluster computer and thereby exercise a functionality that previously did not exist.     

Interactive volume visualization of general polyhedral grids

This paper presents a novel framework for visualizing volumetric data specified on complex polyhedral grids, without the need to perform any kind of a priori tetrahedralization. These grids are composed of polyhedra that often are non-convex and have an arbitrary number of faces, where the faces can be non-planar with an arbitrary number of vertices. The importance of such grids in state-of-the-art simulation packages is increasing rapidly. We propose a very compact, face-based data structure for representing such meshes for visualization, called two-sided face sequence lists (TSFSL), as well as an algorithm for direct GPU-based ray-casting using this representation. The TSFSL data structure is able to represent the entire mesh topology in a 1D TSFSL data array of face records, which facilitates the use of efficient 1D texture accesses for visualization. In order to scale to large data sizes, we employ a mesh decomposition into bricks that can be handled independently, where each brick is then composed of its own TSFSL array. This bricking enables memory savings and performance improvements for large meshes. We illustrate the feasibility of our approach with real-world application results, by visualizing highly complex polyhedral data from commercial state-of-the-art simulation packages.     

A large scale parallel fluid-structure interaction computing platform for simulating structural responses to a detonation shock

Due to the intrinsic nature of multi-physics, it is prohibitively complex to design and implement a simulation software platform for study of structural responses to a detonation shock. In this article, a partitioned fluid-structure interaction computing platform is designed for parallel simulating structural responses to a detonation shock. The detonation and wave propagation are modeled in an open-source multi-component solver based on OpenFOAM and blastFoam, and the structural responses are simulated through the finite element library deal.II. To capture the interaction dynamics between the fluid and the structure, both solvers are adapted to preCICE. For improving the parallel performance of the computing platform, the inter-solver data is exchanged by peer-to-peer communications and the intermediate server in conventional multi-physics software is eliminated. Furthermore, the coupled solver with detonation support has been deployed on a computing cluster after considering the distributed data storage and load-balancing between solvers. The 3D numerical result of structural responses to a detonation shock is presented and analyzed. On 256 processor cores, the speedup ratio of the simulations for a detonation shock reach 178.0 with 5.1 million of mesh cells and the parallel efficiency achieve 69.5%. The results demonstrate good potential of massively parallel simulations. Overall, a general-purpose fluid-structure interaction software platform with detonation support is proposed by integrating open source codes. And this work has important practical significance for engineering application in fields of construction blasting, mining, and so forth.     

MUPHY: A parallel MUlti PHYsics/scale code for high performance bio-fluidic simulations

We present a parallel version of MUPHY, a multi-physics/scale code based upon the combination of microscopic Molecular Dynamics (MD) with a hydro-kinetic Lattice Boltzmann (LB) method. The features of the parallel version of MUPHY are hereby demonstrated for the case of translocation of biopolymers through nanometer-sized, multi-pore configurations, taking into explicit account the hydrodynamic interactions of the translocating molecules with the surrounding fluid. The parallel implementation exhibits excellent scalability on the IBM BlueGene platform and includes techniques which may improve the flexibility and efficiency of other complex multi-physics parallel applications, such as hemodynamics, targeted-drug delivery and others.     

并行分布可视化系统JaVis中的I/O优化机制

针对目前的TB级大规模数据场,可视化过程中数据场的I/O是比较花费时间的,并行分布可视化系统如何设计其数据结构管理机制,才能减少数据场的I/O处理、提高可视化处理速度,这是目前可视化系统必须解决的重要问题之一.详细阐述了JaVis系统采用的合约机制、元数据管理机制、子集管理机制以及数据组织格式等多种优化方式大大减少了数据I/O处理,提高了数据可视化的处理速度.     

Visualizing temporal patterns in large multivariate data using textual pattern matching

Extracting and visualizing temporal patterns in large scientific data is an open problem in visualization research. First, there are few proven methods to flexibly and concisely define general temporal patterns for visualization. Second, with large time-dependent data sets, as typical with today's large-scale simulations, scalable and general solutions for handling the data are still not widely available. In this work, we have developed a textual pattern matching approach for specifying and identifying general temporal patterns. Besides defining the formalism of the language, we also provide a working implementation with sufficient efficiency and scalability to handle large data sets. Using recent large-scale simulation data from multiple application domains, we demonstrate that our visualization approach is one of the first to empower a concept driven exploration of large-scale time-varying multivariate data.     

Performance modeling and analysis of heterogeneous lattice Boltzmann simulations on CPU–GPU clusters

Computational fluid dynamic simulations are in general very compute intensive. Only by parallel simulations on modern supercomputers the computational demands of complex simulation tasks can be satisfied. Facing these computational demands GPUs offer high performance, as they provide the high floating point performance and memory to processor chip bandwidth. To successfully utilize GPU clusters for the daily business of a large community, usable software frameworks must be established on these clusters. The development of such software frameworks is only feasible with maintainable software designs that consider performance as a design objective right from the start. For this work we extend the software design concepts to achieve more efficient and highly scalable multi-GPU parallelization within our software framework waLBerla for multi-physics simulations centered around the lattice Boltzmann method. Our software designs now also support a pure-MPI and a hybrid parallelization approach capable of heterogeneous simulations using CPUs and GPUs in parallel. For the first time weak and strong scaling performance results obtained on the Tsubame 2.0 cluster for more than 1000 GPUs are presented using waLBerla. With the help of a new communication model the parallel efficiency of our implementation is investigated and analyzed in a detailed and structured performance analysis. The suitability of the waLBerla framework for production runs on large GPU clusters is demonstrated. As one possible application we show results of strong scaling experiments for flows through a porous medium.     

Highly scalable computational algorithms on emerging parallel machine multicore architectures II: Development and implementation in the CSD and FSI contexts

In this paper, the second in a series, the authors have extended and implemented their computational algorithms for improving the scalability of CSD (Computational Structural Dynamics) and FSI (Fluid–Structure Interaction) simulations on emerging architectures like multicore High Performance Computing (HPC) platforms. These algorithmic developments and extensions are classified into two categories: (i) enhanced scalability for CSD simulations on multicore platforms, (ii) newer ideas for running FSI simulations. In the first category, the authors employed the ideas developed in the first paper of this series including the multilevel partitioning strategy, next generation optimized communication procedure and better memory management to get enhanced scalability for CSD simulations. In the second category, the authors came up with a novel solver specific multicore-FSI optimal partitioning so as to improve the overall FSI scalability. After implementing the new “intelligent partitioning” algorithm, a speedup ratio of nearly 2.5x was obtained for the total time. The intelligent partitioning algorithm optimizes the number of solid domains relative to the number of fluid domains to optimize the overall FSI solution, irrespective of the type of the flow solver. In general, the authors have demonstrated (i) good, almost linear scalability for aeroelastic applications with several millions of cells on multicore platforms with thousands of cores, (ii) significant improvement in the scalability for smaller FSI problems using the intelligent partitioning.     

VizCluster and its Application on Classifying Gene Expression Data

Visualization enables us to find structures, features, patterns, and relationships in a dataset by presenting the data in various graphical forms with possible interactions. A visualization can provide a qualitative overview of large and complex datasets, can summarize data, and can assist in identifying regions of interest and appropriate parameters focused on quantitative analysis. Recently, DNA microarray technology provides a broad snapshot of the state of the cell, by measuring the expression levels of thousands of genes simultaneously. Such information can thus be used to analyze different samples by gene expression profiles. It has already had a significant impact on the field of bioinformatics, requiring innovative techniques to efficiently and effectively extract, analyze, and visualize these fast growing data.In this paper, we present a dynamic interactive visualization environment, VizCluster, and its application on classifyinggene expression data. VizCluster takes advantage of graphical visualization methods to reveal underlining data patterns. It combines the merits of both high dimensional projection scatter-plot and parallel coordinate plot. In its core lies a nonlinear projection which maps the n-dimensional vectors onto two-dimensional points. To preserve the information at different scales and yet reduce the typical problem of parallel coordinate plots being messy caused by overlapping lines, a zip zooming viewing method is proposed. Integrated with other features, VizCluster is developed to give a simple, fast, intuitive, and yet powerful view of the data set. Its primary applications are on the classification of samples and evaluation of gene clusters for microarray datasets. Three gene expression datasets are used to illustrate the approach. We demonstrate that VizCluster approach is promising to be used for analyzing and visualizing microarray data sets and further development is worthwhile.     

Numerical study of a membrane-type micro check-valve for microfluidic applications

In this paper, we present a method to simulate membrane-type micro check-valve using finite element method with fluid structure interaction formulation. The designed micro check-valve which is analogue of semiconductor device, diode, allows the fluid to pass in forward-mode and blocks it in reverse mode. To have a better understanding on the valve design we also studied the effect of design dimensions on the valve performance. Our study shows the valve flow rate-pressure curve is similar to diode current–voltage characterization curve. A series of simulations were carried out by using two-way fully-coupled Fluid Structure Interaction (FSI) analysis. Using Arbitrary Lagrangian–Eulerian (ALE) method in combination with high viscosity region instead of solid valve-seat enables the designer to overcome simulations difficulties and reduces the amount of calculation time by not considering the contact phenomenon of the membrane and the valve seat. The results show the valve can withstand pressures of up to 11 kPa in reverse-mode regardless of membrane’s hole-size and length of chamber-inlet. Also, chamber-inlet size has high effect on valve opening threshold point in a way that increment of chamber-inlet area will reduce opening threshold point and vice versa.

     

Scaling up multi-touch selection and querying: Interfaces and applications for combining mobile multi-touch input with large-scale visualization displays

We present a mobile multi-touch interface for selecting, querying, and visually exploring data visualized on large, high-resolution displays. Although emerging large (e.g., ～10 m wide), high-resolution displays provide great potential for visualizing dense, complex datasets, their utility is often limited by a fundamental interaction problem – the need to interact with data from multiple positions around a large room. Our solution is a selection and querying interface that combines a hand-held multi-touch device with 6 degree-of-freedom tracking in the physical space that surrounds the large display. The interface leverages context from both the user's physical position in the room and the current data being visualized in order to interpret multi-touch gestures. It also utilizes progressive refinement, favoring several quick approximate gestures as opposed to a single complex input in order to most effectively map the small mobile multi-touch input space to the large display wall. The approach is evaluated through two interdisciplinary visualization applications: a multi-variate data visualization for social scientists, and a visual database querying tool for biochemistry. The interface was effective in both scenarios, leading to new domain-specific insights and suggesting valuable guidance for future developers.     

Molecular dynamics-based prediction of boundary slip of fluids in nanochannels

Computational modeling and simulation can provide an effective predictive capability for flow properties of the confined fluids in micro/nanoscales. In this paper, considering the boundary slip at the fluid–solid interface, the motion property of fluids confined in parallel-plate nanochannels are investigated to couple the atomistic regime to continuum. The corrected second-order slip boundary condition is used to solve the Navier–Stokes equations for confined fluids. Molecular dynamics simulations for Poiseuille flows are performed to study the influences of the strength of the solid–fluid coupling, the fluid temperature, and the density of the solid wall on the velocity slip at the fluid boundary. For weak solid–fluid coupling strength, high temperature of the confined fluid and high density of the solid wall, the large velocity slip at the fluid boundary can be obviously observed. The effectiveness of the corrected second-order slip boundary condition is demonstrated by comparing the velocity profiles of Poiseuille flows from MD simulations with that from continuum.