RAID小数据随机访问性能分析与优化

RAID采用条纹结构,使多块磁盘可并行访问,提高了带宽,适合于大块数据顺序访问,而对小块数据随机访问影响不大。针对Stripe条纹大小对RAID的读写性能进行分析,探讨多用户小数据访问模式下的IOPS(IOs per second)问题,提出粗粒度条纹布局模型。仿真实验表明：该模型的性能优于现有布局方式,显著提高小块数据随机访问性能。     

异构磁盘阵列RAID5结构数据布局研究

由于应用需求的快速发展以及网络存储系统的出现,因此异构磁盘阵列的变得越来越常见。RAID5由于较高的性能和可靠性以及较低的代价,是应用最为广泛的RAID结构。目前对异构磁盘阵列RAID5结构的研究,重点主要放在充分利磁盘存储空间以及对性能的定性研究。论文提出了一种异构磁盘阵列RAID5结构数据布局优化方法,该方法充分考虑异构磁盘的相对容量和性能,以及校验单元的散布对RAID5小数据写性能的影响,可以生成负载均匀分布或接近均匀分布的布局。仿真实验结果表明,对于多用户小数据访问模式,优化布局的性能明显优于简单RAID5布局,且具有更高的伸缩性。     

高效的自适应文件系统的原型设计

为了改进文件系统的性能,提出的文件系统既具有自适应的RAlD3和RAID5混合结构,又能根据缓存数据在缓冲区的活动性不同而分类缓存,数据被延迟刷新到磁盘来减少、合并写磁盘的次数,进一步采用自适应的文件条带策略来优化文件访问性能.研究结果表明,该文件系统能按需自适应地读写多媒体数据和事务处理数据,当文件系统负载低时,可以尽量积极地将文件分布到所有磁盘来最小化I/O的反馈时间;反之,在系统负载高时,将文件分布的范围小一些以便最大化文件系统整体的吞吐量.     

一种新的RAID结构快速扩展方法

提出了一种新的RAID结构扩展方法,其扩展速度大大优于平凡方法。通过仿真实验表明,与标准RAID结构相比,新的扩展方法产生的布局结构只在某些特定的访问模式下性能稍差,而对于常见的多用户并发访问模式,两者性能几乎没有差距。     

一种高可用对象存储系统的数据组织研究

通过对现有不同存储系统的容错能力进行认真分析,提出了一种新的高可用对象存储系统架构HAOSS(High Availability Object Storage System).HAOSS系统分两层:上层存储的对象采用在多个设备之间的数据备份的方式来实现高可靠性,多个备份能够同时对外提供服务,保证系统的高性能,但是磁盘利用率比较低.下层采用RAID5,RAID6以及RAID_Blaurn等不同的容错编码来实现多盘容错功能,磁盘利用率较高,但是由于编码越来越复杂,容错编码需要大量的计算,性能受到很大的影响.在数据组织上,新对象和热点对象放上层,这样大部分请求都能够在上层命中,从而保证系统的性能.下层主要用来放不常用的数据.磁盘利用率比较高.在1000Mbps以太网环境下对HAOSS性能进行了测试,结果表明,HAOSS顺序读写性能都比较好,最大可以达到104MB/s,达到1000Mb以太网的理论最大物理带宽.     

一种新型瓦记录磁盘的高可靠数据存储方法

近年来,传统磁记录的存储密度增长已经达到极限,为了满足快速增长的数据容量需求,多种新型存储技术不断涌现,其中瓦记录（shingled magnetic recording,SMR）技术已实现商业化,在企业实际应用.由于瓦记录磁盘的叠瓦式结构,磁盘在随机写入时会引起写放大,造成磁盘性能下降.这一问题在部署传统的高可靠存储方案（如RAID5）时会变得更加严重,原因在于校验数据更新频率很高,磁盘内出现大量的随机写请求.研究发现瓦记录内部其实存在具有原位更新能力的“可覆盖写磁道（free track）”,基于“可覆盖写磁道”,提出了一种专门针对瓦记录盘的高可靠数据存储方法——FT-RAID,以替代经典的RAID5方法,实现一种廉价、大容量、高可靠的存储系统.FT-RAID包含两个部分：“可覆盖写磁道映射（FT-mapping）”和“可覆盖写磁道缓冲区（FT-buffer）”.FT-mapping实现了一种瓦记录友好的RAID映射方式,将频繁更新的校验块数据映射至“可覆盖写磁道”;FT-buffer实现了一种瓦记录友好的两层缓冲区结构,上层确保了热数据能够原位更新,下层提高了缓冲区的容量.基于真实企业I/O访问记录的实验结果表明,与传统RAID 5相比,FT-RAID能够减少80.4%的写放大率,显著提高存储系统整体性能.     

阵列矩阵RAID50研究——容量聚合与全并发策略

提出了一种跨多阵列通道的海量存储RAID50模型,通过采取多阵列卡的RAID0分条和阵列卡上多磁盘RAID5分条和校验的二级并发的数据组织与分块方式,以扩展块(大小等于阵列卡上的一个RAID5校验组)作为Cache和阵列之间数据交换的单位,实现了将阵列矩阵中所有磁盘的容量聚合及全并发访问。设计了该模型逻辑卷管理的最佳适配算法及二级地址映射算法。理论分析与实验结果表明:该策略将I/O响应时间降到了最低,且获得了与阵列通道数线性相关的逻辑卷容量和I/O性能。     

Redbud并行文件系统的可扩展存储管理机制

数据规模和并发访问的需求日益增长,可扩展能力成为并行文件系统的重要需求之一.文中提出了一种基于非对称并行文件系统Redbud的高可扩展资源管理机制.该管理机制根据数据的访问特征,使用不同的树形结构管理不同类型的数据,满足了文件数据和元数据的并发检索需求;该管理机制还使用文件级的数据分布机制,允许用户利用各种策略进行目录和文件的管理,能满足文件级的数据访问性能、目录级数据可靠性等实际应用需求.多个基准测试程序和实际应用程序的测试结果表明,文件的独占访问能达到磁盘95％的性能;同时,随着设备和应用节点的增加,数据和元数据的并发访问性能线性增长.     

自修复磁盘阵列技术研究

以数据为中心的计算模式对数据的可靠性提出新的需求,高可靠存储设备成为高性能计算的基础.自从Xiotech公司提出ISE结构并取得成功后,关注磁盘失效之前的征兆而不是失效后的具体处理的"自修复"概念已成为磁盘阵列设计新的指导思想之一.提出存储系统可靠性系数的概念,分析单个磁盘可靠性与磁盘阵列可靠性关系,选择合适的磁盘S.M.A.R.T属性结合磁盘当前利用率和历史信息建立评定磁盘状态的T~2US算法,接着介绍RAID 5结合T2US监测的高可靠、自修复的RAID 5T结构,说明RAID 5T结构的运行方式和磁盘不符合T~2US监测标准时数据迁移策略,最后对结构的可靠性利用存储系统可靠性系数和MTTF分别进行了分析.     

混合S-RAID:一种适于连续数据存储的节能数据布局

存储系统规模的日益增大,其高能耗成为一个无法忽视的问题,因此对存储系统的节能研究十分重要.提出一种由SSD固态盘与普通磁盘组成的混合S-RAID结构,通过关闭部分处于空闲状态的磁盘,达到节能效果.混合S-RAID将包括超级块在内的少量随机读写数据放在由SSD组成的RAID1中,将连续数据放在由普通磁盘组成的S-RAID中,S-RAID对磁盘分组,连续数据访问模式下只有一个组处于活动状态,关闭处于空闲状态的磁盘组.在仅增加少量成本的前提下,提高了存储系统的节能效果.混合S-RAID适用于各种以连续数据访问为主要特征的应用环境.实验表明,由12块普通磁盘和两块SSD固态盘组成的混合S-RAID 5与同级别RAID 5相比,能耗仅为RAID 5的28％.     

Storage allocation policies for time-dependent multimedia data

Multimedia computing requires support for heterogeneous data types with differing storage, communication and delivery requirements. Continuous media data types, such as audio and video, impose delivery requirements that are not satisfied by conventional physical storage organizations. In this paper, we describe a physical organization for multimedia data based on the need to support the delivery of multiple playout sessions from a single rotating-disk storage device. Our model relates disk characteristics to the different media recording and playback rates and derives their storage pattern. This storage organization guarantees that as long as a multimedia delivery process is running, starvation will never occur. Furthermore, we derive bandwidth and buffer constraints for disk access and present an approach to minimize latencies for non-continuous media stored on the same device. The analysis and numerical results indicate the feasibility of using conventional rotating magnetic disk storage devices to support multiple sessions for video-on-demand applications     

负载自适应的存储池数据重构策略

大数据时代各应用领域对计算机存储系统的性能和可靠性需求与日俱增。新型存储介质为计算机存储系统的性能提升提供了良好的机遇,基于固态盘的存储阵列（RAIS）已在各种存储系统中广泛使用。传统RAIS系统中当一块固态盘出现故障时,通过数据重构操作恢复故障盘的数据,重构时间长,且影响对上层应用提供I/O访问服务的能力。针对该问题,设计实现了基于多线程并发处理的存储池架构,该架构能够并发处理存储池中的I/O请求,提高用户I/O和数据重构I/O的访问性能。提出了一种负载自适应的I/O调度策略,能够在保证用户I/O服务质量的同时,提升数据重构效率。实验结果表明,基于存储池的多线程并发I/O处理架构能够提升数据重构性能,负载自适应的I/O调度策略能够根据用户I/O的负载情况动态调整用户I/O和数据重构I/O的调度比例,在保证用户I/O服务质量的同时,提升数据重构效率。     

Inter-Object Layer Clustering for scalable video streaming

In this work, we develop an efficient storage technique to support real-time streaming of layer encoded video in a single hard disk. The size of a single hard disk drive will soon be able to hold multi-tera bytes and is going to handle relatively larger number of files. We expect that disk layout in a single disk will be rather critical issue in determining the efficiency of the storage system. We propose a novel storage technique, Inter-Object Layer Clustering for layer encoded video objects. In Inter-Object Layer Clustering, storage is partitioned into two regions: lower layer partition and upper layer partition. Lower and upper layer partition harbor the lower layer and upper layer data blocks across all video objects and cluster them together. We develop an elaborate performance model for this placement scheme. We examine the performance of the proposed technique using analytical formulation as well as a physical experiment. We found that clustering the layers across all objects brings 100% increase in the number of concurrent sessions compared to the case where file is stored in temporal order when the clients’ access bandwidth is narrow. Inter-Object Layer Clustering shows 15% performance improvement compared to the clustering of layers within the objects.     

Efficient video file allocation schemes for video-on-demand services

A video-on-demand (VOD) server needs to store hundreds of movie titles and to support thousands of concurrent accesses. This, technically and economically, imposes a great challenge on the design of the disk storage subsystem of a VOD server. Due to different demands for different movie titles, the numbers of concurrent accesses to each movie can differ a lot. We define access profile as the number of concurrent accesses to each movie title that should be supported by a VOD server. The access profile is derived based on the popularity of each movie title and thus serves as a major design goal for the disk storage subsystem. Since some popular (hot) movie titles may be concurrently accessed by hundreds of users and a current high-end magnetic disk array (disk) can only support tens of concurrent accesses, it is necessary to replicate and/or stripe the hot movie files over multiple disk arrays. The consequence of replication and striping of hot movie titles is the potential increase on the required number of disk arrays. Therefore, how to replicate, stripe, and place the movie files over a minimum number of magnetic disk arrays such that a given access profile can be supported is an important problem. In this paper, we formulate the problem of the video file allocation over disk arrays, demonstrate that it is a NP-hard problem, and present some heuristic algorithms to find the near-optimal solutions. The result of this study can be applied to the design of the storage subsystem of a VOD server to economically minimize the cost or to maximize the utilization of disk arrays.     

一种新型网络视频点播的存储层次

针对多媒体应用中快速存储资源上对匮乏和磁盘速度受限问题,实现了使用网络空闲内存资源作为快速设备缓冲媒体的机制,扩展了多媒体环境中的存储层次,并给出了一种简单快速的cache替换算法,初步结果表明,该机制可以加速多媒体环境中存储数据的访问速度。     

T-Archive:基于分级存储的信息生命周期管理系统

基于策略的信息生命周期管理对于存储和数据的高效使用至关重要,数据分类和策略管理的方便性以及策略执行的高性能成为该领域的突出问题.T-Archive是一个基于分级存储系统架构的策略控制的生命周期管理系统.该系统提供数据分类、策略定制管理、数据自动迁移以及客户端和应用对数据的完全透明访问.实验结果表明,迁移文件的透明访问机制对系统性能影响很小,极大地减少了客户端的管理开销,增强了系统的可扩展性.研究了策略管理在策略缓存方面的性能,使其能够很好地适应大规模的文件系统.     

基于口令锁生成动态组密钥的整盘加密方案

针对整盘加密模式在实际应用中系统较难维护和密钥管理困难的问题,采用基于口令锁的双层密钥获取结构,应用动态组密钥和安全参数的秘密拆分与门限接入等技术,提出一个基于逻辑口令锁生成组密钥的整盘加密方案。该方案可以实现多用户独立接入,且无需保存磁盘密钥。理论分析和实验结果证明其具有较高的安全性,性能优于同类方案。     

Improving multimedia systems performance using constant-density recording disks

Multimedia systems store and retrieve large amounts of data which require extremely high disk bandwidth and their performance critically depends on the efficiency of disk storage. However, existing magnetic disks are designed for small amounts of data retrievals geared to traditional operations; with speed improvements mainly focused on how to reduce seek time and rotational latency. When the same mechanism is applied to multimedia systems, overheads in disk I/O can result in dramatic deterioration in system performance. In this paper, we present a mathematical model to evaluate the performance of constant-density recording disks, and use this model to analyze quantitatively the performance of multimedia data request streams. We show that high disk throughput may be achieved by suitably adjusting the relevant parameters. In addition to demonstrating quantitatively that constant-density recording disks perform significantly better than traditional disks for multimedia data storage, a novel disk-partitioning scheme which places data according to their bandwidths is presented.     

Scheduling and data layout policies for a near-line multimedia storage architecture

Recent advances in computer technologies have made it feasible to provide multimedia services, such as news distribution and entertainment, via high-bandwidth networks. The storage and retrieval of large multimedia objects (e.g., video) becomes a major design issue of the multimedia information system. While most other works on multimedia storage servers assume an on-line disk storage system, we consider a two-tier storage architecture with a robotic tape library as the vast near-line storage and an on-line disk system as the front-line storage. Magnetic tapes are cheaper, more robust, and have a larger capacity; hence, they are more cost effective for large scale storage systems (e.g., video-on-demand (VOD) systems may store tens of thousands of videos). We study in detail the design issues of the tape subsystem and propose some novel tape-scheduling algorithms which give faster response and require less disk buffer space. We also study the disk-striping policy and the data layout on the tape cartridge in order to fully utilize the throughput of the robotic tape system and to minimize the on-line disk storage space.     

Design and evaluation of a multimedia storage server for mixed traffic

The relative simplicity of access to digital communications nowadays and the simultaneous increase in the available bandwidth are leading to the definition of new telematic services, mainly oriented towards multimedia applications and interactivity with the user. In the near future, a decisive role will be played in this scenario by the providers of interactive multimedia services of the on-demand type, which will guarantee the end user a high degree of flexibility, speed and efficiency. In this paper, some of the technical aspects regarding these service providers are dealt with, paying particular attention to the problems of storing information and managing service requests. More specifically, the paper presents and evaluates a new storage technique based on the use of disk array technology, which can manage both typical multimedia connections and traditional requests. The proposed architecture is based on the joint use of the partial dynamic declustering and the information dispersal algorithm, which are employed for the allocation and retrieval of the data stored on the disk array. We also define efficient strategies for request management in such a way as to meet the time constraints imposed by multimedia sessions and guarantee good response times for the rest of the traffic. The system proposed is then analyzed using a simulation approach.