基于海量数据存储系统多级存储介质的热点数据区分方法

海量数据的应用导致文件读写压力变大,必然需要考虑文件I/O带来的系统性能瓶颈.考虑到不同存储设备性能和成本不同,而且数据访问具有时间和空间局部性,因此需要进行分级存储.考虑到数据存在着周期性的变化规律,数据访问的热度是变化的,海量存储系统中相当大比例的数据静止不动,而且高性能存储设备有限,因此基于分级存储技术进行数据迁移,并且对常规价值评定模型进行改进后提出它所适用的基于文件级的数据迁移算法.针对并发访问读写压力变大之后对数据进行迁移的场景,用Web并发访问测试工具来对数据迁移之后的硬件读写进行评估,经过对Web服务器并发访问响应时间的测试验证,与常规迁移方法相比该算法的确改进了系统的性能.

A Learning Method of Hot-Spot Extent in Multi-Tiered Storage Medium Based on Huge Data Storage File System

The application of massive data result in larger pressure in the process of read and write, however, attention must be paid to the performance bottleneck of input and output of system. With the difference of performance and cost of different storage devices, temporal locality and spatial locality of data access, multi-tiered storage is needed. For the frequently change rule of data, variety of data accessed frequency, the big proportion of static data in storage system for cloud data and limitation of high performance storage equipments, migrate data based on multi-tiered storage technology. Define some factors evaluating the value judgement and modify the existing model of value judgement based on file level. With analysis of existing application environment and modified value judgement model, a data migration algorithm based on file level is proposed. Migrate data with heavy access resulting from simultaneously data access, and use the test tool of web simultaneously data access to measure the hardware access. Test the responsing time of simultaneously data access of a web server, which shows that comparing to the normal data migration, the algorithm improves the performance of system.