异构大数据编程环境 Hadoop+

互联网和物联网技术的飞速发展开启了“大数据”时代。目前，硬件的高速发展催生了许多异构芯片，它们越来越多地出现在大规模数据中心里，支持不同的应用程序，在提升性能的同时降低整体功耗。文章重点介绍了基于 MapReduce编程模型的 Hadoop+框架的设计与实现，它允许用户在单个任务中调用 CUDA/OpenCL的并行实现，并能通过异构任务模型帮助用户。在我们的实验平台上，五种常见机器学习算法使用 Hadoop+框架相对于 Hadoop能达到1.4×～16.1×的加速比，在 Hadoop+框架中使用异构任务模型指导其资源分配策略，对单个应用负载上最高达到36.0%的性能提升；对多应用的混合负载，最多能减少36.9%，平均17.6%的应用执行时间。

Hadoop+: A Big-data Programming Framework for Heterogeneous Computing Environments

The rapid development of Internet and Internet of Things opens the era of big data. Currently, heterogeneous architectures are being widely adopted in large-scale datacenters, for the sake of performance improvement and reduction of energy consumption. This paper presents the design and implementation of Hadoop+, a programming framework that implements MapReduce and enables invocation of parallelized CUDA/OpenCL within a map/reduce task, and helps the user by taking advantage of a heterogeneous task model. Experimental result shows that Hadoop+ attains 1.4× to 16.1× speedups over Hadoop for five commonly used machine learning algorithms. Coupled with a heterogeneous task model that helps allocate computing resouces, Hadoop+ brings a 36.0% improvement in data processing speed for single-application workloads, and for mixed workloads of multiple applications, the execution time is reduced by up to 36.9%with an average 17.6%.