基于内存云的大块数据对象并行存取策略

由于内存云(RAMCloud)只支持最大1 MB的小块数据对象存储,因此当大于1 MB的对象需要存储在内存云集群中就会受到对象大小的限制,无法在集群中进行存储。为了解决内存云存储限制的问题,提出了基于内存云的大块数据对象并行存取策略。该存储策略首先将大块数据对象分割成若干个1 MB的小块数据对象,然后在客户端生成数据摘要,最后使用并行存储算法将客户端分割成的小块数据对象存储在内存云集群中。读取时首先读取数据摘要,然后根据数据摘要从内存云集群中并行读取小块数据对象,并将小块数据对象合并生成大块数据对象。实验结果表明:大块数据对象的并行存取策略在不破坏内存云集群体系结构的前提下存储时间为16~18 μs,读取时间为6~7 μs。在InfiniBand网络架构下,所提并行算法的加速比呈现类似线性的增长,它使大块数据对象也能够像小块数据对象一样在微秒级别下快速、高效地进行存取。

Parallel access strategy for big data objects based on RAMCloud

RAMCloud only supports the small object storage which is not larger than 1 MB. When the object which is larger than 1 MB needs to be stored in the RAMCloud cluster, it will be constrained by the object's size. So the big data objects can not be stored in the RAMCloud cluster. In order to resolve the storage limitation problem in RAMCloud, a parallel access strategy for big data objects based on RAMCloud was proposed. Firstly, the big data object was divided into several small data objects within 1 MB. Then the data summary was created in the client. The small data objects which were divided in the client were stored in RAMCloud cluster by the parallel access strategy. On the stage of reading, the data summary was firstly read, and then the small data objects were read in parallel from the RAMCloud cluster according to the data summary. Then the small data objects were merged into the big data object. The experimental results show that, the storage time of the proposed parallel access strategy for big data objects can reach 16 to 18 μs and the reading time can reach 6 to 7 μs without destroying the architecture of RAMCloud cluster. Under the InfiniBand network framework, the speedup of the proposed paralled strategy almost increases linearly, which can make the big data objects access rapidly and efficiently in microsecond level just like small data objects.