一种面向数据可用性和存储可靠性动态要求的自适应纠删码存储策略设计

为了满足指数级增长的大数据存储需求,现代的分布式存储系统需要提供大容量的存储空间以及快速的存储服务.因此在主流的分布式存储系统中,均应用了纠删码技术以节约数据中心的磁盘成本,保证数据的可靠性,并且满足应用程序和客户端的快速存储需求.在实际应用中数据往往重要程度并不相同,对数据可用性要求不一,且不同磁盘的故障率和可靠性动...

Design of an Adaptive Erasure Code Storage Strategy Oriented to Dynamic Requirements of Data Availability and Storage Reliability

In order to meet the exponentially growing demand for large data storage,modern distributed storage systems need to provide large-capacity storage space and fast storage services.Therefore,in mainstream distributed storage systems,erasure coding technology is applied to reduce disk costs in the data center,ensure data reliability,and meet the fast storage requirements of applications and clients.In practice,data often has different levels of importance and different requirements for data availability.Also,different disks have different failure rates and reliability.These pose new challenges for traditional RAID storage systems including erasure code-based storage systems.This article proposes On-demand ARECS(On-demand Availability and Reliability Oriented Adaptive Erasure Coded Storage System),which is designed according to multiple dimensions of storage back-end data availability and disk reliability,and comprehensively determines erasure coding strategy and storage node selection.This reduces storage redundancy and storage latency,while improving data availability and storage reliability.We conducted experiments in the Tahoe-LAFS open source distributed file system,and the experiment results verified our theoretical analysis.Under the premise of ensuring data availability and disk reliability with diverse requirements,data redundancy and storage delay were significantly reduced.