国产异构架构系统上HPL的优化与分析

随着异构系统成为建造超级计算机的重要选择,如何让CPU与加速器协调工作以充分发挥异构系统的计算性能具有重要意义.HPL是高性能计算领域最重要的基准测试程序,传统面向纯CPU系统的HPL算法通过利用加速器加速矩阵乘法的做法已经无法取得很好的性能.针对这一问题,本文基于新的国产处理器-国产加速器异构系统提出了一个新的HPL性能模型,设计了一种全新的多线程细粒度异构HPL算法.我们完成了一个轻量级跨平台异构加速框架HPCX用来实现跨平台的HPL算法.我们的性能模型能够准确的预测类似异构系统的HPL性能,我们的多线程细粒度异构HPL算法在NVIDIA GPU平台上性能超过目前NVIDIA平台上性能最好的NVIDIA官方闭源nvhpl程序9%.在国产处理器-国产加速器平台512节点的规模上,我们的新HPL算法实现了2.3PFLOPS实测峰值性能和71.1%的浮点效率.     

国产异构系统上HPL的优化与分析

随着异构系统成为建造超级计算机的重要选择,如何让CPU与加速器协调工作以充分发挥异构系统的计算性能具有重要意义.HPL是高性能计算领域最重要的基准测试程序,传统面向纯CPU系统的HPL算法通过加速器加速矩阵乘法的做法已经无法取得很好的性能.针对这一问题,提出了基于国产处理器-国产加速器异构系统的HPL性能模型和多线程细粒度流水HPL算法.完成了一个轻量级跨平台异构加速框架HPCX,以实现跨平台的HPL算法.该性能模型能够准确地预测类似异构系统的HPL性能.该HPL算法在NVIDIA GPU平台上性能超过了NVIDIA官方闭源nvhpl程序9%.在国产处理器-国产加速器平台512个节点的规模上,优化的HPL算法实现了2.3 PFLOPS实测峰值性能和71.1%的浮点效率.     

Parallel application performance on shared high performance reconfigurable computing resources

The use of a network of shared, heterogeneous workstations each harboring a reconfigurable computing (RC) system offers high performance users an inexpensive platform for a wide range of computationally demanding problems. However, effectively using the full potential of these systems can be challenging without the knowledge of the system's performance characteristics. While some performance models exist for shared, heterogeneous workstations, none thus far account for the addition of RC systems. Our analytic performance model includes the effects of the reconfigurable device, application load imbalance, background user load, basic message passing communication, and processor heterogeneity. The methodology proves to be accurate in characterizing these effects for applications running on shared, homogeneous, and heterogeneous HPRC resources. The model error in all cases was found to be less than 5% for application runtimes greater than 30 s, and less than 15% for runtimes less than 30 s.     

Using accurate AIC-based performance models to improve the scheduling of parallel applications

Predictions based on analytical performance models can be used on efficient scheduling policies in order to select adequate resources for an optimal execution in terms of throughput and response time. However, developing accurate analytical models of parallel applications is a hard issue. The TIA (Tools for Instrumenting and Analysis) modeling framework provides an easy to use modeling method for obtaining analytical models of MPI applications. This method is based on modeling selection techniques and, in particular, on Akaike’s information criterion (AIC). In this paper, first the AIC-based performance model of the HPL benchmark is obtained using the TIA modeling framework. Then the use of this model for assessing the runtime estimation on different backfilling policies is analyzed in the GridSim simulator. The behavior of these simulations is compared with the equivalent simulations based on the theoretical model of the HPL provided by its developers.     

面向异构计算机平台的HPL方案

HPL（high performance Linpack）是一套被广泛用于测评计算机性能的测试程序,几十年来学术界及产业界十分关注对HPL测试程序的定制化优化工作,以充分反应同时代新兴计算机平台的性能.面向当今主流多设备异构计算平台,尝试为HPL的优化工作提供一种解决方案：Hetero-HPL.在Hetero-HPL中,进程与协处理器的对应关系可被改变,因此HPL算法在单节点独立运行情况下可以完全避免进程间数据传输开销.算法各个重要步骤有能力完全利用物理节点的所有资源,如内存容量、CPU核心、协处理器、PCI-e总线等.Hetero-HPL并不引入冗余计算量及通信量,并在任意设备数量下妥善应对锁页内存分配限制,确保多设备负载均衡和设备内高效的大规模同质运算.在实验平台上,Hetero-HPL效率可以达到平台峰值性能的76.5%（其中,dgemm函数效率为84%）.进一步的实验结果表明,Hetero-HPL在多节点联机运行情况下也是一种可行的方案.     

复杂异构计算系统HPL的优化

当今世界的主流超级计算机越来越多地使用带有加速器的异构系统.随着加速器的浮点性能不断提高,超级计算机内计算节点的CPU、内存、总线、网络以及系统架构都要与之相适应.HPL（high performance Linpack）是高性能计算机评测的传统基准测试程序,复杂异构系统给HPL评测带来很多机遇与挑战.针对带有GPU的异构超级计算机系统,提出一套新的CPU与加速器计算任务分配方式,提出平衡点理论指导HPL性能优化.为了优化HPL程序,提出了使用CPU与加速器协同工作的look-ahead算法和行交换连续流水算法,实现了加速器、CPU、网络等部件的高度并行.此外,为带有加速器的系统设计了新的panel分解和行交换的实现方法,提高了加速器的利用率.在每个节点带有4个GPU的系统上,单节点HPL效率达到了79.51%.     

Complex version of high performance computing LINPACK benchmark (HPL)

This paper describes our effort to enhance the performance of the AORSA fusion energy simulation program through the use of high‐performance LINPACK (HPL) benchmark, commonly used in ranking the top 500 supercomputers. The algorithm used by HPL, enhanced by a set of tuning options, is more effective than that found in the ScaLAPACK library. Retrofitting these algorithms, such as look‐ahead processing of pivot elements, into ScaLAPACK is considered as a major undertaking. Moreover, HPL is configured as a benchmark, but only for real‐valued coefficients. We therefore developed software to convert HPL for use within an application program that generates complex coefficient linear systems. Although HPL is not normally perceived as a part of an application, our results show that the modified HPL software brings a significant increase in the performance of the solver when simulating the highest resolution experiments thus far configured, achieving 87.5 TFLOPS on over 20 000 processors on the Cray XT4. Copyright © 2009 John Wiley & Sons, Ltd.     

面向国产异构系统的HPL异构协同设计

HPL是高性能计算广泛采用的Linpack测试软件包,传统HPL算法中,求解矩阵将以块为单位循环分布到所有处理器,由于国产加速器(China Accelerator)的底层矩阵乘接口仅支持定制接口,传统HPL算法已不适合CPU+China Accelerator异构系统,因此,必须基于定制接口完成矩阵分布细致划分与封装dPEM,以提供一个通用的HPL测试配置环境;同时,为了充分发挥国产异构系统的效率,设计了异构协同矩阵乘调度算法OA4MM,以提高国产异构系统的效率。实验验证了dPEM的有效性和OA4MM算法的高效性,OA4MM较传统的异构HPL调度算法性能提升近10%。     

复杂异构计算系统HPL优化研究

当今世界的主流超级计算机越来越多地使用带有加速器的异构系统.随着加速器的浮点性能不断提高,超级计算机内计算节点的CPU、内存、总线、网络以及系统架构都要与之相适应.HPL（High Performance Linpack）是高性能计算机评测的传统基准测试程序,复杂异构系统给HPL评测带来很多机遇与挑战.针对带有GPU的异构超级计算机系统,提出一套新的CPU与加速器计算任务分配方式,提出平衡点理论指导HPL性能优化.为了优化HPL程序,提出了使用CPU与加速器协同工作的look-ahead算法和行交换连续流水算法,实现了加速器、CPU、网络等部件的高度并行.此外,为带有加速器的系统设计了新的panel分解和行交换的实现方法,提高加速器的利用率.在每个节点带有4个GPU的系统上,单节点HPL效率达到79.51%,14884节点效率达到62.22%.     

一种分层P2P网络中的负载均衡算法

负载失衡是影响P2P系统应用服务性能的关键因素之一。目前,已有的研究集中在基于flat DHT(Distributed Hashing Table)的P2P模型上。分层拓扑结构由于其诸多优点而受到重视。将分层的思想引入虚拟服务器技术中,结合其优势,提出一种层次化的负载均衡算法。仿真实验表明,该算法可以依据节点能力的不同,保证负载在各个节点上公平分布。     

Jcluster: an efficient Java parallel environment on a large‐scale heterogeneous cluster

In this paper, we present Jcluster, an efficient Java parallel environment that provides some critical services, in particular automatic load balancing and high‐performance communication, for developing parallel applications in Java on a large‐scale heterogeneous cluster. In the Jcluster environment, we implement a task scheduler based on a transitive random stealing (TRS) algorithm. Performance evaluations show that the scheduler based on TRS can make any idle node obtain a task from another node with much fewer stealing times than random stealing (RS), which is a well‐known dynamic load‐balancing algorithm, on a large‐scale cluster. In the performance aspects of communication, with the method of asynchronously multithreaded transmission, we implement a high‐performance PVM‐like and MPI‐like message‐passing interface in pure Java. The evaluation of the communication performance is conducted among the Jcluster environment, LAM‐MPI and mpiJava on LAM‐MPI based on the Java Grande Forum's pingpong benchmark. Copyright © 2005 John Wiley & Sons, Ltd.     

Towards the optimal synchronization granularity for dynamic scheduling of pipelined computations on heterogeneous computing systems

Loops are the richest source of parallelism in scientific applications. A large number of loop scheduling schemes have therefore been devised for loops with and without data dependencies (modeled as dependence distance vectors) on heterogeneous clusters. The loops with data dependencies require synchronization via cross‐node communication. Synchronization requires fine‐tuning to overcome the communication overhead and to yield the best possible overall performance. In this paper, a theoretical model is presented to determine the granularity of synchronization that minimizes the parallel execution time of loops with data dependencies when these are parallelized on heterogeneous systems using dynamic self‐scheduling algorithms. New formulas are proposed for estimating the total number of scheduling steps when a threshold for the minimum work assigned to a processor is assumed. The proposed model uses these formulas to determine the synchronization granularity that minimizes the estimated parallel execution time. The accuracy of the proposed model is verified and validated via extensive experiments on a heterogeneous computing system. The results show that the theoretically optimal synchronization granularity, as determined by the proposed model, is very close to the experimentally observed optimal synchronization granularity, with no deviation in the best case, and within 38.4% in the worst case. Copyright © 2012 John Wiley & Sons, Ltd.     

大规模CFD多区结构网格任务负载平衡算法

针对现有负载平衡算法的适应度低、可扩展性差、通信开销度量不准确的缺陷, 提出一种大规模CFD多区结构网格任务负载平衡算法。通过对网格块的分割、网格块之间的组合映射、进程上网格计算量的调整来实现并行CFD任务负载平衡。实验结果表明, 该算法既适应同构平台也适应异构平台, 既适应网格块数多于进程数的情况也适应网格块数少于进程数的情况, 该算法可使得整个计算空间分配到各进程上的计算量负载平衡, 同时使得各进程间的最大通信开销最小。     

HPCC:面向存储访问模型的基准测试--一种可能替代TOP500 HPL的测试方法

高性能计算机系统的性能评价历来是本领域所关注的重要问题．TOP500排名所采用的标准测试HPL（High Performance Linpack）并不能真实的反映系统各方面的性能，尤其是存储访问方面．HPC Challenge基准测试则着重于各种存储访问模型，在HPL的基础之上又整合了多个有代表性的核心测试程序，很有可能在未来取代现在TOP500采用的的HPL测试．本文首先简单介绍HPC Challenge诞生的背景，解释基准测试的基本概念和原理，从存储访问模型的角度对各项测试进行了描述，并根据实际的测试结果进行比较和分析．最后给出结论以及将来的工作．     

Hetero-HPL:面向异构高性能计算机的HPL测试程序

HPL（High Performance Linpack）是一套被广泛用于测评计算机性能的测试程序,几十年来学术界及产业界十分关注对HPL测试程序的定制化优化工作,以充分反应同时代新兴计算机平台的性能.面向当今主流多设备异构计算平台,本文尝试为HPL的优化工作提供一种新的解决方案：Hetero-HPL.在Hetero-HPL中,进程不再要求与（协）处理器一一对应,因此HPL算法在单节点独立运行情况下可以完全避免进程间数据传输开销,算法各个重要步骤有能力完全利用物理节点的所有资源,如内存容量,CPU核心,协处理器,PCI-e总线等.Hetero-HPL并不引入冗余计算量及通信量,并在任意设备数量下妥善应对锁页内存分配限制,确保多设备负载均衡和设备内的高效的大规模同质运算.在实验平台上,Hetero-HPL效率可以达到平台峰值性能的76.5%（其中矩阵乘函数效率为84%）;进一步的实验表明,Hetero-HPL在多节点联机运行情况下也是一种可行的方案.     

Tuning parallel applications in parallel

In this paper, we present and evaluate a parallel algorithm for parameter tuning of parallel applications. We discuss the impact of performance variability on the accuracy and efficiency of the optimization algorithm and propose a strategy to minimize the impact of this variability. We evaluate our algorithm within the Active Harmony system, an automated online/offline tuning framework. We study its performance on three benchmark codes: PSTSWM, HPL and POP. Compared to the Nelder–Mead algorithm, our algorithm finds better configurations up to seven times faster. For POP, we were able to improve the performance of a production sized run by 59%.     

On the effectiveness of runtime techniques to reduce memory sharing overheads in distributed Java implementations

Distributed Java virtual machine (dJVM) systems enable concurrent Java applications to transparently run on clusters of commodity computers. This is achieved by supporting Java's shared‐memory model over multiple JVMs distributed across the cluster's computer nodes. In this work, we describe and evaluate selective dynamic diffing and lazy home allocation, two new runtime techniques that enable dJVMs to efficiently support memory sharing across the cluster. Specifically, the two proposed techniques can contribute to reduce the overheads due to message traffic, extra memory space, and high latency of remote memory accesses that such dJVM systems require for implementing their memory‐coherence protocol either in isolation or in combination. In order to evaluate the performance‐related benefits of dynamic diffing and lazy home allocation, we implemented both techniques in Cooperative JVM (CoJVM), a basic dJVM system we developed in previous work. In subsequent work, we carried out performance comparisons between the basic CoJVM and modified CoJVM versions for five representative concurrent Java applications (matrix multiply, LU, Radix, fast Fourier transform, and SOR) using our proposed techniques. Our experimental results showed that dynamic diffing and lazy home allocation significantly reduced memory sharing overheads. The reduction resulted in considerable gains in CoJVM system's performance, ranging from 9% up to 20%, in four out of the five applications, with resulting speedups varying from 6.5 up to 8.1 for an 8‐node cluster of computers. Copyright © 2007 John Wiley & Sons, Ltd.     

VCluster: a thread‐based Java middleware for SMP and heterogeneous clusters with thread migration support

Clusters, composed of symmetric multiprocessor (SMP) machines and heterogeneous machines, have become increasingly popular for high‐performance computing. Message‐passing libraries, such as message‐passing interface (MPI) and parallel virtual machine (PVM), are de facto parallel programming libraries for clusters that usually consist of homogeneous and uni‐processor machines. For SMP machines, MPI is combined with multithreading libraries like POSIX Thread and OpenMP to take advantage of the architecture. In addition to existing parallel programming libraries that are in C/C++ and FORTRAN programming languages, the Java programming language presents itself as another alternative with its object‐oriented framework, platform neutral byte code, and ever‐increasing performance. This paper presents a new parallel programming model and a library, VCluster, which implements this model. VCluster is based on migrating virtual threads instead of processes to support clusters of SMP machines more efficiently. The implementation uses thread migration, which can be used in dynamic load balancing. VCluster was developed in pure Java, utilizing the portability of Java to support clusters of heterogeneous machines. Several applications are developed to illustrate the use of this library and compare the usability and performance of VCluster with other approaches. Copyright © 2007 John Wiley & Sons, Ltd.     

Performance and energy aware scheduling simulator for HPC: evaluating different resource selection methods

Today, in an energy‐aware society, job scheduling is becoming an important task for computer engineers and system analysts that may lead to a performance per Watt trade‐off of computing infrastructures. Thus, new algorithms, and a simulator of computing environments, may help information and communications technology and data center managers to make decisions with a solid experimental basis. There are several simulators that try to address performance and, somehow, estimate energy consumption, but there are none in which the energy model is based on benchmark data that have been countersigned by independent bodies such as the Standard Performance Evaluation Corporation. This is the reason why we have implemented a performance and energy‐aware scheduling (PEAS) simulator for high‐performance computing. Furthermore, to evaluate the simulator, we propose an implementation of the non‐dominated sorting genetic algorithm‐II (NSGA‐II) algorithm, a fast and elitist multiobjective genetic algorithm, for the resource selection. With the help of the PEAS simulator, we have studied if it is possible to provide an intelligent job allocation policy that may be able to save energy and time without compromising performance. The results of our simulations show a great improvement in response time and power consumption. In most of the cases, NSGA‐II performs better than other ‘intelligent’ algorithms like multiobjective heterogeneous earliest finish time and clearly outperforms the first‐fit algorithm. We demonstrate the usefulness of the simulator for this type of studies and conclude that the superior behavior of multiobjective algorithms makes them recommended for use in modern scheduling systems. Copyright © 2015 John Wiley & Sons, Ltd.     

Exploiting heterogeneous parallelism with the Heterogeneous Programming Library

While recognition of the advantages of heterogeneous computing is steadily growing, the issues of programmability and portability hinder its exploitation. The introduction of the OpenCL standard was a major step forward in that it provides code portability, but its interface is even more complex than that of other approaches. In this paper, we present the Heterogeneous Programming Library (HPL), which permits the development of heterogeneous applications addressing both portability and programmability while not sacrificing high performance. This is achieved by means of an embedded language and data types provided by the library with which generic computations to be run in heterogeneous devices can be expressed. A comparison in terms of programmability and performance with OpenCL shows that both approaches offer very similar performance, while outlining the programmability advantages of HPL.