RGraph:基于RDMA的高效分布式图数据处理系统

图是描述实体间关系的重要数据结构,被广泛地应用于信息科学、物理学、生物学、环境生态学等重要的科学领域.现如今,随着图数据规模的不断增大,利用分布式系统来处理大图数据已经成为主流,出现了形如Pregel、GraphX、PowerGraph和Gemini等经典的分布式大图数据处理系统.然而,与当前先进的基于单机的图处理系统...

RGraph: Effective Distributed Graph Data Processing System Based on RDMA

Graph is a significant data structure which describes the relationship between entries, and it is widely used in information science, physics, biology, environmental ecology and other scientific fields. Nowadays, with the growing magnitude of graph data, processing large-scale graph data using distributed system has become the popular, many specialized distributed system, including Pregel, GraphX, PowerGraph and Gemini have been proposed. However, compared with the current state-of-the-art shared-memory graph processing systems, these specialized distributed graph processing systems do not deliver satisfactory or stable performance advantages in processing real-world graph datasets. We analyze several representative distributed graph processing systems and summarize the major challenges that affect their performance. After study, we propose RGraph, an effective distributed graph processing system based on RDMA. The key idea of RGraph is improving performance on top of making full use of the advantages of RDMA. For graph partition, RGraph adopts chunk-based partition to avoid destroying the native locality of the real-world graph, so as to ensure the locality-preserving vertex accesses. For workload, RGraph proposes a task migration mechanism based on RDMA one-side READ and a fine-grained task preemption method among threads to ensure the dynamic load balance for inter-node and intra-node, so that all computing resources can be fully utilized. For communication, RGraph effectively encapsulates IB verbs and implements a concurrent RDMA communication stack satisfied graph computing semantics. Compared with traditional MPI, RGraph''s communication stack can reduce the latency up to 2.1x for servers'' communication. Finally, we use five real-world large-scale graph datasets and one synthetic dataset to evaluation RGraph on a HPC cluster with eight servers, and the experiment shows that RGraph has obvious performance advantages. Compared with Powergraph, RGraph has 10.1x-16.8x performance improvement. And compared with the existing state-of-the-art CPU-based distributed graph processing system, RGraph still has 2.89x-5.12x performance improvement. Meanwhile, RGraph can still guarantee stable performance advantage on extremely skewed power-law graph.