面向海量非结构化数据的非关系型存储管理机制

针对传统的关系数据存储系统性能不足、容错性差,无法适应海量非结构化数据管理的问题,提出一种高性能、高可用非关系型存储管理机制。首先,设计了良好的用户访问服务接口,通过高效的一致性哈希算法支持数据分发到多个存储节点;其次,采用可配置的数据副本机制改善存储系统的可用性;最后,提出查询故障处理机制,用以提升存储系统的容错性,避免节点失效导致服务中断问题。实验结果表明,在不同规模用户负载下,新的存储系统的并发访问请求能力和传统的文件系统、关系数据库相比,分别提升了30%和50%;同时,在合理响应时间内,故障状态下的存储系统的可用性损失小于14%。因此,该机制适用于海量非结构化数据的高效存储管理。

Non-relational data storage management mechanism for massive unstructured data

Traditional relational data storage systems have been criticized by poor performance and lacking of fault tolerance, therefore it cannot satisfy the efficiency requirement of the massive unstructured data management. A non-relational storage management mechanism with high-performance and high-availability was proposed. First, a user-friendly application interface was designed, and data could be distributed to multiple storage nodes through efficient consistent hashing algorithm. Second, a configurable data replication mechanism was presented to enhance availability of the storage system. Finally, a query fault handling mechanism was proposed to improve the storage system's fault-tolerance and avoid service outages, which were caused by the node failure. The experimental results show that the concurrent access capacity of the proposed storage system increases by 30% and 50% respectively compared to traditional file system and relational database under different user workloads; meanwhile, the availability loss of the storage system under the fault state is less than 14% in a reasonable response time. Therefore, it is applicable for efficient storage management of massive unstructured data.