高性能计算环境下地理计算服务流程并行处理方法

地理计算是地理信息科学中重要的高级计算模式,是新一代地理信息系统(Geographic Information System, GIS)中不可缺少的组成部分。地理计算过程涉及数据处理、空间分析、过程模拟等多个流程,过程繁复且耗时,计算复 杂且运算量大。高性能计算(High Performance Computing, HPC)是解决复杂地理计算问题的有效方法。针对高性能 计算环境中地理计算服务流程并行执行问题,提出一种简洁的地理计算服务流程建模方法和基于多线程的并发流程 处理策略,设计并实现了高性能计算环境下地理计算服务流程执行引擎原型系统,实现了并发条件下地理计算服务流 程的自动执行与负载均衡。通过在真实高性能计算环境中的实验,验证了所提出的方法能够有效解决地理计算服务 流程建模、执行优化问题,具有良好的性能、准确性及应用前景。     

Approaching parallel computing to simulating population dynamics in demography

Agent-based modelling and simulation is a promising methodology that can be applied in the study of population dynamics. The main advantage of this technique is that it allows representing the particularities of the individuals that are modeled along with the interactions that take place among them and their environment. Hence, classical numerical simulation approaches are less adequate for reproducing complex dynamics. Nowadays, there is a rise of interest on using distributed computing to perform large-scale simulation of social systems. However, the inherent complexity of this type of applications is challenging and requires the study of possible solutions from the parallel computing perspective (e.g., how to deal with fine grain or irregular workload). In this paper, we discuss the particularities of simulating populating dynamics by using parallel discrete event simulation methodologies. To illustrate our approach, we present a possible solution to make transparent the use of parallel simulation for modeling demographic systems: Yades tool. In Yades, modelers can easily define models that describe different demographic processes with a web user interface and transparently run them on any computer architecture environment thanks to its demographic simulation library and code generator. Therefore, transparency is provided by two means: the provision of a web user interface where modelers and policy makers can specify their agent-based models with the tools they are familiar with, and the automatic generation of the simulation code that can be executed in any platform (cluster or supercomputer). A study is conducted to evaluate the performance of our solution in a High Performance Computing environment. The main benefit of this outline is that our findings can be generalized to problems with similar characteristics to our demographic simulation model.     

Current issues in high performance computing I/O architectures and systems

The abundance of parallel and distributed computing platforms, such as MPP, SMP, and the Beowulf clusters, to name just a few, has added many more possibilities and challenges to high performance computing (HPC), parallel I/O, mass data storage, scalable architectures, and large-scale simulations, which traditionally belong to the realm of custom-tailored parallel systems. The intent of this special issue is to discuss problems and solutions, to identify new issues, and to help shape future research directions in these areas. From these perspectives, this special issue addresses the problems encountered at the hardware, architectural, and application levels, while providing conceptual as well as empirical treatments to the current issues in high performance computing, and the I/O architectures and systems utilized therein.     

New trends in parallel and distributed simulation: From many-cores to Cloud Computing

Recent advances in computing architectures and networking are bringing parallel computing systems to the masses so increasing the number of potential users of these kinds of systems. In particular, two important technological evolutions are happening at the ends of the computing spectrum: at the “small” scale, processors now include an increasing number of independent execution units (cores), at the point that a mere CPU can be considered a parallel shared-memory computer; at the “large” scale, the Cloud Computing paradigm allows applications to scale by offering resources from a large pool on a pay-as-you-go model. Multi-core processors and Clouds both require applications to be suitably modified to take advantage of the features they provide. Despite laying at the extreme of the computing architecture spectrum – multi-core processors being at the small scale, and Clouds being at the large scale – they share an important common trait: both are specific forms of parallel/distributed architectures. As such, they present to the developers well known problems of synchronization, communication, workload distribution, and so on. Is parallel and distributed simulation ready for these challenges? In this paper, we analyze the state of the art of parallel and distributed simulation techniques, and assess their applicability to multi-core architectures or Clouds. It turns out that most of the current approaches exhibit limitations in terms of usability and adaptivity which may hinder their application to these new computing architectures. We propose an adaptive simulation mechanism, based on the multi-agent system paradigm, to partially address some of those limitations. While it is unlikely that a single approach will work well on both settings above, we argue that the proposed adaptive mechanism has useful features which make it attractive both in a multi-core processor and in a Cloud system. These features include the ability to reduce communication costs by migrating simulation components, and the support for adding (or removing) nodes to the execution architecture at runtime. We will also show that, with the help of an additional support layer, parallel and distributed simulations can be executed on top of unreliable resources.     

A case study on expressiveness and performance of component-oriented parallel programming

Component-oriented programming has been applied to address the requirements of large-scale applications from computational sciences and engineering that present high performance computing (HPC) requirements. However, parallelism continues to be a challenging requirement in the design of CBHPC (Component-Based High Performance Computing) platforms. This paper presents strong evidence about the efficacy and the efficiency of HPE (Hash Programming Environment), a CBHPC platform that provides full support for parallel programming, on the development, deployment and execution of numerical simulation code onto cluster computing platforms.     

High performance computing for three-dimensional agent-based molecular models

Agent-based simulations are increasingly popular in exploring and understanding cellular systems, but the natural complexity of these systems and the desire to grasp different modelling levels demand cost-effective simulation strategies and tools.In this context, the present paper introduces novel sequential and distributed approaches for the three-dimensional agent-based simulation of individual molecules in cellular events. These approaches are able to describe the dimensions and position of the molecules with high accuracy and thus, study the critical effect of spatial distribution on cellular events. Moreover, two of the approaches allow multi-thread high performance simulations, distributing the three-dimensional model in a platform independent and computationally efficient way.Evaluation addressed the reproduction of molecular scenarios and different scalability aspects of agent creation and agent interaction. The three approaches simulate common biophysical and biochemical laws faithfully. The distributed approaches show improved performance when dealing with large agent populations while the sequential approach is better suited for small to medium size agent populations.Overall, the main new contribution of the approaches is the ability to simulate three-dimensional agent-based models at the molecular level with reduced implementation effort and moderate-level computational capacity. Since these approaches have a generic design, they have the major potential of being used in any event-driven agent-based tool.     

Agent-based simulation of electricity markets: a survey of tools

Agent-based simulation has been a popular technique in modeling and analyzing electricity markets in recent years. The main objective of this paper is to study existing agent-based simulation packages for electricity markets. We first provide an overview of electricity markets and briefly introduce the agent-based simulation technique. We then investigate several general-purpose agent-based simulation tools. Next, we review four popular agent-based simulation packages developed for electricity markets and several agent-based simulation models reported in the literature. We compare all the reviewed packages and models and identify their common features and design issues. Based on the study, we describe an agent-based simulation framework for electricity markets to facilitate the development of future models for electricity markets.     

面向HPC的函数计算冷启动优化

High performance computing problems usually have the characteristics of parallelization of subtasks, and a lot of computing resources are consumed in the process of execution. It has been proved that traditional cloud computing based on virtual machine can deal with such problems, but the management of distributed environment and the distributed design of solutions make the processing more complex. Function computing is a new type of serverless cloud computing paradigm, its automatic expansion and considerable computing resources can be well combined with HPC problems. However, the cold start delay is an unavoidable problem on the public cloud function computing platform, especially in the task of HPC problems having high concurrent jobs of which delay will be further magnified. In this paper, we first analyze the completion time of a simple HPC task under cold start and hot start conditions, and analyze the causes of additional delay. According to these analyses, we combine the time series ana lysis tools and the platform's automatic expansion mechanism to propose an effective preheating method, which can effectively reduce the cold start delay of HPC tasks on the function computing platform.     

A Direct Execution Approach to Simulating Mobile Agent Algorithms

Mobile agent technology has been applied to develop the solutions for various kinds of parallel and distributed computing problems. However, performance evaluation of mobile agent algorithms remains a difficult task, mainly due to the characteristics of mobile agents such as distributed and asynchronous execution, autonomy and mobility. This paper proposes a general approach based on direct execution simulation for evaluating the performance of mobile agent algorithms by collecting and analyzing the information about the agents during their execution. We describe the proposed generic simulation model, named MADES, the architecture of a software environment based on MADES, and a prototype implementation. A mobile agent-based distributed load balancing algorithm has been used for experiments with the prototype.     

New techniques for simulating high performance MPI applications on large storage networks

In this work, we propose new techniques to analyze the behavior, the performance, and specially the scalability of High Performance Computing (in short, HPC) applications on different computing architectures. Our final objective is to test applications using a wide range of architectures (real or merely designed) and scaling it to any number of nodes or components. This paper presents a new simulation framework, called SIMCAN, for HPC architectures. The main characteristic of the proposed simulation framework is the ability to be configured for simulating a wide range of possible architectures that involve any number of components. SIMCAN is developed to simulate complete HPC architectures, but putting special emphasis on the storage and network subsystems. The SIMCAN framework can handle complete components (nodes, racks, switches, routers, etc.), but also key elements of the storage and network subsystems (disks, caches, sockets, file systems, schedulers, etc.). We also propose several methods to implement the behavior of HPC applications. Each method has its own advantages and drawbacks. In order to evaluate the possibilities and the accuracy of the SIMCAN framework, we have tested it by executing a HPC application called BIPS3D on a hardware-based computing cluster and on a modeled environment that represent the real cluster. We also checked the scalability of the application using this kind of architecture by simulating the same application with an increased number of computing nodes.     

ArchSim: A System-Level Parallel Simulation Platform for the Architecture Design of High Performance Computer

High performance computer (HPC) is a complex huge system, of which the architecture design meets increasing difficulties and risks. Traditional methods, such as theoretical analysis, component-level simulation and sequential simulation, are not applicable to system-level simulations of HPC systems. Even the parallel simulation using large-scale parallel machines also have many difficulties in scalability, reliability, generality, as well as efficiency. According to the current needs of HPC architecture design, this paper proposes a system-level parallel simulation platform: ArchSim. We first introduce the architecture of ArchSim simulation platform which is composed of a global server (GS), local server agents (LSA) and entities. Secondly, we emphasize some key techniques of ArchSim, including the synchronization protocol, the communication mechanism and the distributed checkpointing/restart mechanism. We then make a synthesized test of some main performance indices of ArchSim with the phold benchmark and analyze the extra overhead generated by ArchSim. Finally, based on ArchSim, we construct a parallel event-driven interconnection network simulator and a system-level simulator for a small scale HPC system with 256 processors. The results of the performance test and HPC system simulations demonstrate that ArchSim can achieve high speedup ratio and high scalability on parallel host machine and support system-level simulations for the architecture design of HPC systems.     

Towards highly available and scalable high performance clusters

In recent years, we have witnessed a growing interest in high performance computing (HPC) using a cluster of workstations. This growth made it affordable to individuals to have exclusive access to their own supercomputers. However, one of the challenges in a clustered environment is to keep system failure to the minimum and to achieve the highest possible level of system availability. High-Availability (HA) computing attempts to avoid the problems of unexpected failures through active redundancy and preemptive measures. Since the price of hardware components are significantly dropping, we propose to combine both HPC and HA concepts and layout the design of a HA-HPC cluster, considering all possible measures. In particular, we explore the hardware and the management layers of the HA-HPC cluster design, as well as a more focused study on the parallel-applications layer (i.e. FT-MPI implementations). Our findings show that combining HPC and HA architectures is feasible, in order to achieve HA cluster that is used for High Performance Computing.     

一种提高高性能服务器资源利用率的新方法

目前高性能服务器能够支持多种应用,但各种应用在同一段时间内对资源需求是不均衡的,致使高性能服务器的资源利用率较低。该文所提出的动态部署系统是一种提高基于InfiniBand 和SAN的高性能服务器资源利用的新方法。该动态部署系统通过构建虚拟服务器,改变服务器结点的计算特性,在各种应用中移动计算节点,从而合理分配服务器中闲置资源,提高资源利用率和应用的性能。对动态部署系统的基本概念、设计、功能实现、性能测试及其理论证明等进行了研究。     

Highly interactive computational steering for coupled 3D flow problems utilizing multiple GPUs

Most computational fluid dynamics (CFD) simulations require massive computational power which is usually provided by traditional High Performance Computing (HPC) environments. Although interactivity of the simulation process is highly appreciated by scientists and engineers, due to limitations of typical HPC environments, present CFD simulations are usually executed non interactively. A recent trend is to harness the parallel computational power of graphics processing units (GPUs) for general purpose applications. As an alternative to traditional massively parallel computing, GPU computing has also gained popularity in the CFD community, especially for its application to the lattice Boltzmann method (LBM). For instance, Tölke and others presented very efficient implementations of the LBM for 2D as well as 3D space (Toelke J, in Comput Visual Sci. (2008); Toelke J and Krafczk M, in Int J Comput Fluid Dyn 22(7): 443–456 (2008)). In this work we motivate the use of GPU computing to facilitate interactive CFD simulations. In our approach, the simulation is executed on multiple GPUs instead of traditional HPC environments, which allows the integration of the complete simulation process into a single desktop application. To demonstrate the feasibility of our approach, we show a fully bidirectional fluid-structure-interaction for self induced membrane oscillations in a turbulent flow. The efficiency of the approach allows a 3D simulation close to realtime.     

HSWAP:适用于高性能计算环境的数值模拟工作流管理平台

针对高性能计算（HPC）环境中的"建模、计算、分析、优化"一体化应用构建的问题，设计了支持数值模拟软件封装和数值模拟工作流交互设计的数值模拟工作流管理平台——HSWAP。首先，基于对数值模拟活动的运行特征共性建模构建组件模型；然后，利用工作流表达数值模拟活动间的控制、数据依赖关系，建立形式化的数值模拟工作流模型，所形成的工作流模型可在平台中自动解析并适配高性能计算资源，从而实现批量关联数值模拟任务的自动生成与调度，为领域用户屏蔽高性能计算资源的使用细节。平台提供Web Portal服务，支持图形数值模拟程序的交互界面推送。目前该平台已在超算中心实际生产环境得到部署应用，可在2人月内完成包含10个以下数值模拟软件、20个以内计算任务节点的数值模拟工作流的集成。     

基于Agent仿真模型的校核与验证研究

首先对国内外基于Agent仿真模型的校核与验证方面的发展情况进行了述评;然后,提出了一个完整的基于Agent仿真模型的校核与验证框架,该方案中包括表面验证、参数灵敏度分析、模型校准与运行时验证;最后以环境经济政策仿真模型中的校核与验证为例对该框架进行了简单说明。     

基于Agent的复杂系统分布仿真建模方法的研究

基于Agent的分布仿真是研究大型复杂系统的一种有效的、重要的方法。为了减小复杂系统仿真的复杂度,增加仿真模型的重用和可维护性,需要研究基于Agent分布仿真的建模方法。首先对复杂系统及其特性进行了分析,对基于Agent的仿真进行了全面的论述,然后对基于Agent的复杂系统仿真中的复杂系统建模分析、Agent建模分析以及Agent的分布进行了分析,给出了基于Agent的复杂系统分布仿真的建模步骤,最后给出了在此建模思想指导下的金融证券市场的建模过程。     

Enhancing integrated earthquake simulation with high performance computing

Integrated earthquake simulation (IES) is a seamless simulation of the three earthquake processes, namely, the earthquake hazard process, the earthquake disaster process and the anti-disaster action process. High performance computing (HPC) is essential if IES, or particularly, the simulation of the earthquake disaster process is applied to an urban area in which 10^4∼6 structures are located. IES is enhanced with parallel computation, and its performance is examined, so that virtual earthquake disaster simulation will be made for a model of an actual city by inputting observed strong ground motion. It is shown that parallel IES has fairly good scalability even when advanced non-linear seismic structure analysis is used.     

A technique to automatically determine Ad-hoc communication patterns at runtime

Current High Performance Computing (HPC) systems are typically built as interconnected clusters of shared-memory multicore computers. Several techniques to automatically generate parallel programs from high-level parallel languages or sequential codes have been proposed. To properly exploit the scalability of HPC clusters, these techniques should take into account the combination of data communication across distributed memory, and the exploitation of shared-memory models.In this paper, we present a new communication calculation technique to be applied across different SPMD (Single Program Multiple Data) code blocks, containing several uniform data access expressions. We have implemented this technique in Trasgo, a programming model and compilation framework that transforms parallel programs from a high-level parallel specification that deals with parallelism in a unified, abstract, and portable way.The proposed technique computes at runtime exact coarse-grained communications for distributed message-passing processes. Applying this technique at runtime has the advantage of being independent of compile-time decisions, such as the tile size chosen for each process. Our approach allows the automatic generation of pre-compiled multi-level parallel routines, libraries, or programs that can adapt their communication, synchronization, and optimization structures to the target system, even when computing nodes have different capabilities.Our experimental results show that, despite our runtime calculation, our approach can automatically produce efficient programs compared with MPI reference codes, and with codes generated with auto-parallelizing compilers.     

Design of efficient Java message-passing collectives on multi-core clusters

This paper presents a scalable and efficient Message-Passing in Java (MPJ) collective communication library for parallel computing on multi-core architectures. The continuous increase in the number of cores per processor underscores the need for scalable parallel solutions. Moreover, current system deployments are usually multi-core clusters, a hybrid shared/distributed memory architecture which increases the complexity of communication protocols. Here, Java represents an attractive choice for the development of communication middleware for these systems, as it provides built-in networking and multithreading support. As the gap between Java and compiled languages performance has been narrowing for the last years, Java is an emerging option for High Performance Computing (HPC).