Client-side straggler-aware I/O scheduler for object-based parallel file systems

Object-based parallel file systems have emerged as promising storage solutions for high-performance computing (HPC) systems. Despite the fact that object storage provides a flexible interface, scheduling highly concurrent I/O requests that access a large number of objects still remains as a challenging problem, especially in the case when stragglers (storage servers that are significantly slower than others) exist in the system. An efficient I/O scheduler needs to avoid possible stragglers to achieve low latency and high throughput. In this paper, we introduce a log-assisted straggler-aware I/O scheduling to mitigate the impact of storage server stragglers. The contribution of this study is threefold. First, we introduce a client-side, log-assisted, straggler-aware I/O scheduler architecture to tackle the storage straggler issue in HPC systems. Second, we present three scheduling algorithms that can make efficient decision for scheduling I/Os while avoiding stragglers based on such an architecture. Third, we evaluate the proposed I/O scheduler using simulations, and the simulation results have confirmed the promise of the newly introduced straggler-aware I/O scheduler.     

Techniques for increasing the stream capacity of a high-performancemultimedia server

High-performance servers and high-speed networks will form the backbone of the infrastructure required for distributed multimedia information systems. A server for an interactive distributed multimedia system may require thousands of gigabytes of storage space and a high I/O bandwidth. In order to maximize the system utilization, and thus minimize the cost, it is essential that the load be balanced among each of the server's components, viz. the disks, the interconnection network and the scheduler. Many algorithms for maximizing retrieval capacity from the storage system have been proposed in the literature. This paper presents techniques for improving the server capacity by assigning media requests to the nodes of a server so as to balance the load on the interconnection network and the scheduling nodes. Five policies for request assignment-round-robin (RR), minimum link allocation (MLA), minimum contention allocation (MCA), weighted minimum link allocation (WMLA) and weighted minimum contention allocation (WMCA)-are developed. The performance of these policies on a server model developed by the authors (1995) is presented. We also consider the issue of file replication, and develop two schemes for storing the replicas: the parent group-based round-robin placement (PGBRRP) scheme, and the group-wide round-robin placement (GWRRP) scheme. The performance of the request assignment policies in the presence of file replication is presented     

Filter Cache:一种提高Lustre I/O性能的方法

高性能计算系统需要一个可靠高效的并行文件系统.Lustre集群文件系统是典型的基于对象存储的集群文件系统,它适合大数据量聚合I/O操作.大文件I/O操作能够达到很高的带宽,但是小文件I/O性能低下.针对导致Lustre的设计中不利于小文件I/O操作的两个方面,提出了Filter Cache方法.在Lustre的OST组件中设计一个存放小文件I/O数据的Cache,让OST端的小文件I/O操作异步进行,以此来减少用户感知的小文件I/O操作完成的时间,提高小文件I/O操作的性能.     

Storage technique for real-time streaming of layered video

Scalable streaming technology has been proposed to effectively support heterogeneous devices with dynamically varying bandwidth. From the file system’s point of view, scalable streaming introduces another dimension of complexity in disk scheduling. Most of the existing efforts on multimedia file systems are dedicated to I/O scheduling algorithm and data placement scheme that efficiently guarantee I/O bandwidth. The important underlying assumption in these efforts is that most of the multimedia file accesses are simple playback operations and therefore are sequential. However, this workload characteristic is not valid in scalable streaming environment. In a scalable streaming environment, i.e., when only a subset of imagery is retrieved, the playback does not necessarily coincide with the sequential access on the file. The current file structure and the file system organization leaves much to be desired for supporting scalable streaming service. In this work, we propose a file system scheme, Harmonic Placement to efficiently support scalable streaming. The basic idea of Harmonic placement is to cluster the frequently accessed layers together to avoid unnecessary disk seeks. The data blocks are partitioned into two sets with respect to the layers: lower layers and upper layers. In Harmonic placement, the data blocks in the lower layers are placed with respect to their frame sequence and the data blocks in the upper layers are clustered according to the layers they belong to. We develop elaborate performance models for three different file system schemes: Progressive placement, Interleaved Placement and Harmonic Placement. We investigate the performance of the file server with different file system schemes. It was found that file system performance is very sensitive to the file organization scheme. When most of the service requests are for low-quality video (e.g., 128 Kbits/s ISDN), Progressive placement scheme supports twice as many sessions as the Interleaved placement scheme. When most of the service requests are for high-quality video (e.g., 1.5 Mbits/s MPEG-2 DVD quality), Interleaved placement can support twice as many requests as Progressive placement. In both cases, Harmonic placement scheme yields the most promising performance. Primitive version of this work has appeared on Proceedings of NOSSDAV ’06, Providence, Rhode Island, USA. This work is in part funded by KOSEF throught National Research Lab (ROA-2007-000-200114-0) and by HY-SDR center at Hanyang University.     

基于QoS的磁盘调度策略

随着视频点播、视频会议、视频监控、数字图书馆等流媒体应用的普及，流媒体服务器存储资源管理成为制约服务质量的瓶颈之一。根据多媒体服务器的性能要求，提出了一种支持QoS的磁盘调度策略。它由三个主要部分组成：探测模块、负载监测模块和自适应管理模块。探测模块，负责判断当前的资源情况能否满足服务请求；自适应模块，根据负载监删模块检测到的负载变化情况，动态调整服务周期在实时请求和尽力服务请求之间的分配。实验表明此磁盘调度策略能在保证实时请求无抖动执行的同时，明显减少了非实时请求的响应时间。     

Multi-bank闪存文件系统的一种I/O调度机制

闪存以其体积小、抗震性强、能耗低、读取速度快等特点,被广泛应用于存储系统中。NOOP是闪存上传统的调度方法,但是NOOP的I/O性能较低,不能满足很多应用程序的要求。根据闪存读取速度快、多个banks(chips)可以并行运行等特点,提出了一种基于闪存文件系统YAFFS的Multi-bank闪存调度方法(简称MBS)。MBS并行地执行请求,且给予读请求更高的优先级。MBS根据AVL-based-tree机制识别出的写请求属性动态地将其分配到合适的bank中。实验结果表明,相比NOOP,MBS调度具有更高的I/O吞吐率、更短的请求响应时间并具有均匀的bank擦除次数和利用率。     

A segmentation approach for file broadcast scheduling

We study the broadcast scheduling problem in which clients send their requests to a server in order to receive some files available on the server. The server may be scheduled in a way that several requests are satisfied in one broadcast. When files are transmitted over computer networks, broadcasting the files by fragmenting them provides flexibility in broadcast scheduling that allows the optimization of per user response time. The broadcast scheduling algorithm, then, is in charge of determining the number of segments of each file and their order of transmission in each round of transmission. In this paper, we obtain a closed form approximation formula which approximates the optimal number of segments for each file, aiming at minimizing the total response time of requests. The obtained formula is a function of different parameters including those of underlying network as well as those of requests arrived at the server. Based on the obtained approximation formula we propose an algorithm for file broadcast scheduling which leads to total response time which closely conforms to the optimum one. We use extensive simulation and numerical study in order to evaluate the proposed algorithm which reveals high accuracy of obtained analytical approximation. We also investigate the impact of various headers that different network protocols add to each file segment. Our segmentation approach is examined for scenarios with different file sizes at the range of 100 KB to 1 GB. Our results show that for this range of file sizes the segmentation approach shows on average 13% tolerance from that of optimum in terms of total response time and the accuracy of the proposed approach is growing by increasing file size. Besides, using proposed segmentation in this work leads to a high Goodput of the scheduling algorithm.     

视频服务器中多网络I/O的调度与接纳控制

在视频服务器中,由于网络I/O带宽比磁盘I/O带宽相对丰富而未得到研究者的重视,但当服务达到一定的规模、网络输出成为瓶颈时,在多网络I/O情况下,网络输出的量化计算则不可避免.着重探讨了视频服务器在多网络I/O并行输出的情况下,如何进行最大输出调度的问题,给出了3个调度算法及相应的接纳计算公式,特别是一个带缓冲的优化调度模型.实验结果显示,调度模型及规划性能是优越而有效的.该项工作对视频服务器的调度设计、资源的有效配置具有指导作用.     

Evaluation of Remote-I/O Support for a DSM-Based Computation Offloading Scheme

Computation offloading enables mobile devices to execute rich applications by using the abundant computing resources of powerful server systems. The distributed shared memory based (DSM-based) computation offloading approach is expected to be especially popular in the near future because it can dynamically migrate running threads to computing nodes and does not require any modifications of existing applications to do so. The current DSM-based computation offloading scheme, however, has focused on efficiently offloading computationally intensive applications and has not considered the significant performance degradation caused by processing the I/O requests issued by offloaded threads. Because most mobile applications are interactive and thus yield frequent I/O requests, efficient handling of I/O operations is critically important. In this paper, we quantitatively analyze the performance degradation caused by I/O processing in DSM-based computation offloading schemes using representative commodity applications. To remedy the performance degradation, we apply a remote I/O scheme based on remote device support to computation offloading. The proposed approach improves the execution time by up to 43.6% and saves up to 17.7% of energy consumption in comparison with the existing offloading schemes. Selective compression of the remote I/O scheme reduces the network traffic by up to 53.5%.     

存储区域网络中路径容错及负载均衡的研究

为利用存储区域网络的冗余路径,在主机上加入驱动程序对逻辑单元（LUN）的输入,输出请求进行截获。在故障路径上的I／O请求失败后。由驱动程序在正常路径上重新执行。路径失败时I／O请求直接发送给HBA驱动程序,避免了磁盘驱动程序中的无效重试过程,缩短了故障切换时间。I／O请求被分配到多条路径上,提高了主机的吞吐能力,缩短了I／O平均响应时间。     

一种基于iSCSI的附网存储服务器系统的设计与实现

随着应用的高速发展，容量大、响应快、高性能NAS正成为网络存储领域的研究热点。针对这种情况，设计了一种基于iSCSI协议的专用附网存储服务器(iSCSI-based NAS，iNAS)，在iNAS中，通过iSCSI软件实现模块，使得iNAS同时提供file I／O和block I／O服务，实现了NAS和SAN的融合；通过在RAID控制器和用户内存之间的直接数据传输(零拷贝)，极大地提高了iNAS的I／O响应速度．通过一个多级分流的设备驱动程序，将多个RAID整合成单一的存储池，从而实现了存储虚拟化，实验结果显示，iNAS对文件I／O请求和块I／O请求都具有极高的响应速度。     

一种资源优化的双最小均衡Web集群区分服务调度算法

在一种新的Web集群体系结构的基础上,提出了一种资源优化的双最小均衡区分服务调度算法：首先在前端调度器按资源均衡度将Web请求分配到各后台服务器．然后将Web请求的优先级与资源均衡度两个特征参数结合起来,综合设计后台服务器的Web请求调度顺序,为了评估该算法的性能,进行了大量的模拟实验．在与其他著名调度策略如分离式调度的对比结果显示：双最小均衡调度算法使Web请求的效率提高了11％,同时很好地实现了区分服务．证实了资源优化调度策略具有一定的普遍意义．     

并行网络文件系统PNFS性能评测与分析

传统的网络文件系统难以满足高性能计算系统的I/O 需求,并行网络文件系统——PNFS可以有效地解决传统网络文件系统在可扩展性、可用性和性能上存在的问题。首先对PNFS的体系结构进行了设计,实现了元数据服务器与存储服务器的分离,消除了由于集中服务器结构引发的I/O瓶颈问题。然后,对PNFS的原型系统进行了性能测试,并与相同环境下NFS的测试结果进行比较与分析,结果表明PNFS能够为客户端提供并行访问文件数据的能力,有着较高的I/O读写带宽和较低的访问延迟,同时实现了客户端I/O带宽与存储服务器规模之间的线性可扩展关系,能较好地满足高性能计算中的I/O需求。     

Queueing models for a single server LAN with processor sharing disciplines

A local area network (LAN) is a collection of autonomous workstations interconnected by a communication network. A key component of a local network is the file server which stores programs and data and makes them available to the workstations as needed. In practice, a workstation requests a large portion of the file (type 1 request) when a new application is launched. Following this, the workstation requests additional portions of the file as needed (type 2 requests). Clearly, the response time to these requests will depend strongly on the file server scheduling policy to service the two types of incoming requests. In an earlier paper [12] we studied this system when type 2 requests havepreemptive and non-preemptive priority over type 1 requests. From a practical point of view, it is more realistic to describe the file transfer by around-robin scheduling discipline. In this paper we study LANs where the file server followsprocessor sharingdiscipline, which is a limiting case of theround-robin discipline. Assuming that all relevant intervals are exponentially distributed we develop algorithms to analyze the performance of the system. Illustrative examples are presented to study the system behavior under various load conditions. The computational approach presented in this paper helps in resolving some of the analytical difficulties associated with the analysis ofprocessor sharing disciplines.     

一种新颖的并行程序配置优化算法

本文提出了一种新颖的并行程序配置优化算法。这种算法利用黑板系统将配置优化问题分解组织为不同层次的知识领域,并利用A^＊算法对决策树进行搜索。研究了并行程序的任务调度、存储服务器的数据分配、自适应分片、协同I／O和数据筛选五个知识领域。     

Compatible byte-addressable direct I/O for peripheral memory devices in Linux

Memory devices can be used as storage systems to provide a lower latency that can be achieved by disk and flash storage. However, traditional buffered input/output (I/O) and direct I/O are not optimized for memory-based storages. Traditional buffered I/O includes a redundant memory copy with a disk cache. Traditional direct I/O does not support byte addressing. Memory-mapped direct I/O, which optimizes file operations for byte-addressable persistent memory and appears to the CPU as a main memory. However, it has an interface that is not always compatible with existing applications. In addition, it cannot be used for peripheral memory devices (e.g., networked memory devices and hardware RAM drives) that are not interfaced with the memory bus. This paper presents a new Linux I/O layer, byte direct I/O (BDIO), that can process byte-addressable direct I/O using the standard application programming interface. It requires no modification of existing application programs and can be used not only for the memory but also for the peripheral memory devices that are not addressable by a memory management unit. The proposed BDIO layer allows file systems and device drivers to easily support BDIO. The new I/O achieved 18% to 102% performance improvements in the evaluation experiments conducted with online transaction processing, file server, and desktop virtualization storage.     

负载自适应的存储池数据重构策略

大数据时代各应用领域对计算机存储系统的性能和可靠性需求与日俱增.新型存储介质为计算机存储系统的性能提升提供了良好的机遇,基于固态盘的存储阵列(RAIS)已在各种存储系统中广泛使用.传统RA IS系统中当一块固态盘出现故障时,通过数据重构操作恢复故障盘的数据,重构时间长,且影响对上层应用提供I/O访问服务的能力.针对该问...     

Batching and dynamic allocation techniques for increasing the stream capacity of an on-demand media server

A server for an interactive distributed multimedia system may require thousands of gigabytes of storage space and high I/O bandwidth. In order to maximize system utilization, and thus minimize cost, the load must be balanced among the server's disks, interconnection network and scheduler. Many algorithms for maximizing retrieval capacity from the storage system have been proposed. This paper presents techniques for improving server capacity by assigning media requests to the nodes of a server so as to balance the load on the interconnection network and the scheduling nodes. Five policies for dynamic request assignment are developed. An important factor that affects data retrieval in a high-performance continuous media server is the degree of parallelism of data retrieval. The performance of the dynamic policies on an implementation of a server model developed earlier is presented for two values of the degree of parallelism.     

Issues in the design of a storage server for video-on-demand

We examine issues related to the design of a storage server for video-on-demand (VOD) applications. The storage medium considered is magnetic disks or arrays of disks. We investigate disk scheduling policies, buffer management policies and I/O bus protocol issues. We derive the number of sessions that can be supported from a single disk or an array of disks and determine the amount of buffering required to support a given number of users. Furthermore, we propose a scheduling mechanism for disk accesses that significantly lowers the buffer-size requirements in the case of disk arrays. The buffer size required under the proposed scheme is independent of the number of disks in the array. This property allows for striping video content over a large number of disks to achieve higher concurrency in access to a particular video object. This enables the server to satisfy hundreds of independent requests to the same video object or to hundreds of different objects while storing only one copy of each video object. The reliability implications of striping content over a large number of disks are addressed and two solutions are proposed. Finally, we examine various policies for dealing with disk thermal calibration and the placement of videos on disks and disk arrays.     

Operating system support for a video-on-demand file service

This paper describes the design and implementation of a continuous media file server intended for use in emerging video-on-demand applications. The main focus and contribution of the paper is in scheduling and admission-control algorithms for accessing the server's processor and storage resources. The scheduling algorithms support multiple classes of tasks with diverse performance requirements and allow for the co-existence of guaranteed real-time requests with sporadic, and unsolicited requests. The scheduler maintains performance guarantees for real-time streams in the presence of unpredictably varying non-real-time traffic while ensuring system stability even during overloads. A prototype video file server was implemented on an Intel 486 platform. Performance results show that a large number of streams can be supported, while maintaining efficient utilization of system resources.