Mitra: A Scalable Continuous Media Server

Mitra is a scalable storage manager that supports the display of continuous media data types, e.g., audio and video clips. It is a software based system that employs off-the-shelf hardware components. Its present hardware platform is a cluster of multi-disk workstations, connected using an ATM switch. Mitra supports the display of a mix of media types. To reduce the cost of storage, it supports a hierarchical organization of storage devices and stages the frequently accessed objects on the magnetic disks. For the number of displays to scale as a function of additional disks, Mitra employs staggered striping. It implements three strategies to maximize the number of simultaneous displays supported by each disk. First, the EVEREST file system allows different files (corresponding to objects of different media types) to be retrieved at different block size granularities. Second, the FIXB algorithm recognizes the different zones of a disk and guarantees a continuous display while harnessing the average disk transfer rate. Third, Mitra implements the Grouped Sweeping Scheme (GSS) to minimize the impact of disk seeks on the available disk bandwidth.In addition to reporting on implementation details of Mitra, we present performance results that demonstrate the scalability characteristics of the system. We compare the obtained results with theoretical expectations based on the bandwidth of participating disks. Mitra attains between 65% to 100% of the theoretical expectations.     

Performance evaluation for VBR Continuous Media File Server admission control

A file server for continuous media must provide resource guarantees and only admit requests that do not violate the resource availability. This paper addresses the admission performance of a server that explicitly considers the variable bit rate nature of the continuous media streams. A prototype version of the server has been implemented and evaluated in several heterogeneous environments. The two system resources for which admission control is evaluated are the disk bandwidth and the network bandwidth. Performance results from both measurement and simulation are shown with respect to different admission methods and varying scenarios of stream delivery patterns. We show that the vbrSim algorithm developed specifically for the server outperforms the other options for disk admission especially with request patterns that have staggered arrivals, while the network admission control algorithm is able to utilize a large percentage of the network bandwidth available. We also show the interactions between the limits of these two resources and how a system can be configured without wasted capacity on either one of the resources. Copyright © 2004 John Wiley & Sons, Ltd.     

Design and Implementation of the Stony Brook Video Server

The system architecture of the Stony Brook Video Server (SBVS), which guarantees end-to-end real-time video playback in a client-server setting, is presented. SBVS employs a real-time network access protocol, RETHER, to use existing Ethernet hardware as the underlying communications media. The video server tightly integrates the bandwidth guarantee mechanisms for network transport and disk I/O. SBVS's stream-by-stream disk scheduling scheme optimizes the effective disk bandwidth without incurring significant scheduling overhead. To demonstrate the feasibility of the proposed architecture, we have implemented a prototype called SBVS-1, which can support five concurrent MPEG-1 video streams on an Intel 486DX2/EISA PC. To our knowledge, this system is the first video server that provides an end-to-end performance guarantee from the server's disks to the each user's display over standard Ethernet. This paper describes the implementation details of integrating network and I/O bandwidth guarantee mechanisms, and the performance measurements that drive and/or validate our design decisions. © 1997 by John Wiley & Sons, Ltd.     

Operating system support for a video-on-demand file service

This paper describes the design and implementation of a continuous media file server intended for use in emerging video-on-demand applications. The main focus and contribution of the paper is in scheduling and admission-control algorithms for accessing the server's processor and storage resources. The scheduling algorithms support multiple classes of tasks with diverse performance requirements and allow for the co-existence of guaranteed real-time requests with sporadic, and unsolicited requests. The scheduler maintains performance guarantees for real-time streams in the presence of unpredictably varying non-real-time traffic while ensuring system stability even during overloads. A prototype video file server was implemented on an Intel 486 platform. Performance results show that a large number of streams can be supported, while maintaining efficient utilization of system resources.     

面向HDTV视频存储的专用文件系统

分析了HDTV视频存储的原理,讨论了实际应用中通用文件系统在连续媒体存储方面遇到的问题,归纳出多媒体文件系统应具备的功能和磁盘调度算法,提出了一种满足连续媒体存储策略的专用文件系统,提高了资源利用率和服务质量,并基于系统硬件平台,设计了适用于软件实现的算法逻辑.     

Storage and synchronization for distributed continuous media

Advances in local area ATM networking make it possible to integrate digital audio and video, or continuous media (CM), into distributed computing environments. The temporal nature of CM introduces new requirements for acceptable end-to-end delay and jitter. This and an increased demand for bandwidth affect the design of modern, general purpose, operating systems. This paper addresses distributed system support for CM applications, focusing on the design and implementation of facilities for secondary storage and synchronization. We describe the relevant issues, along with details of our research project and its relationship to existing work. We emphasize experimentation and present results obtained with a prototype in an ATM testbed. These provide fresh insights into the role of operating system support for multimedia.     

QoS-based evaluation of file systems and distributed system services for continuous media provisioning

This paper presents a QoS-based performance analysis of file systems and distributed object services for Continuous Media (CM) provisioning, as well as the details and implementation experiences of a continuous media file system and associated CM servers. For this we have implemented QoS-driven CM servers and the Presto continuous media file system (PFS) in the context of a distributed multimedia application development environment, and validate the performance of PFS against that of the conventional Unix file system through an experimental evaluation. Using our CM server prototype, we next examine the effect of continuous media data delivery on the three different kinds of network protocols such as CORBA, UDP/IP, and TCP/IP, with respect to QoS provisioning and throughput.     

Design and evaluation of a generic software architecture foron-demand video servers

Introduces the design, implementation and evaluation of a generic software architecture for on-demand video servers. We describe different key components for controlling the storage and network devices within the server. The interactive collaborations between these software components are also illustrated. The experimental results indicate a very promising direction in exploring the right combinations of these software components. The server is thus able to increase the number of concurrent video accesses with the same hardware configuration. For instance, with the right combinations, the system achieved about 80% of the storage system bandwidth of four disks, about 70% of the storage system bandwidth of six disks, and generally reached the maximal achieved SCSI bandwidth when eight disks are used over two SCSI buses. Our research and experimental results are based on video servers currently under construction across a variety of hardware platforms, including SMP, DMP and clusters of PCs or workstations. The most advanced prototype server is based on an SGI shared-memory multiprocessor with a mass storage system consisting of RAID-3 disk arrays. With all the enabling/management schemes, we were able to further investigate interesting research issues by considering the user's access profiles for taking advantage of popular video titles. The results were significant, with a range of 60% improvement given a 512 kByte block size. In addition to the experimental results, theoretical performance models were also developed that closely match to our collected experimental results     

A Statistical Admission Control Scheme for Continuous Media Servers Using Caching

In continuous media servers, disk load can be reduced by using buffer cache. In order to utilize the saved disk bandwidth by caching, a continuous media server must employ an admission control scheme to decide whether a new client can be admitted for service without violating the requirements of clients already being serviced. A scheme providing deterministic QoS guarantees in servers using caching has already been proposed. Since, however, deterministic admission control is based on the worst case assumption, it causes the wastage of the system resources. If we can exactly predict the future available disk bandwidth, both high disk utilization and hiccup-free service are achievable. However, as the caching effect is not analytically determined, it is difficult to predict the disk load without substantial computation overhead. In this paper, we propose a statistical admission control scheme for continuous media servers where caching is used to reduce disk load. This scheme improves disk utilization and allows more streams to be serviced while maintaining near-deterministic service. The scheme, called Shortsighted Prediction Admission Control (SPAC), combines exact prediction through on-line simulation and statistical estimation using a probabilistic model of future disk load in order to reduce computation overhead. It thereby exploits the variation in disk load induced by VBR-encoded objects and the decrease in client load by caching. Through trace-driven simulations, it is demonstrated that the scheme provides near-deterministic QoS and keeps disk utilization high.     

可扩展单一映象文件系统的设计、实现及评价

曙光超级服务器是典型的机群系统,ＣＯＳＭＯＳ是为其研制开发的可扩展的单一映象文件系统。文中主要描述了ＣＯＳＭＯＳ原型系统的设计、实现及评价。其中重点介绍了双粒度合作式缓存、分布式元数据管理及网络磁盘存储分组等关键技术,并利用Ｉ／Ｏ其准程序对原型文件系统进行了性能评价,测试结果表明了该原型系统在保证系统单一映象的基础上,具备良好的可扩展性。     

SDFS: A software-defined file system for multitenant cloud storage

Performance isolation is highly desirable in cloud platforms where the virtual disks of virtual machines are simply large files on the shared and networked storage servers. However, existing isolation techniques cannot deal with the implications of the file system used by the networked storage servers, such that underlying resource usage is unpredictable (eg, the delayed write-back mechanism could postpone writes, and the journaling mechanism could amplify writes). The lack of visibility on underlying resource usage leads to the predicament of being unable to meet isolation goals. In this paper, we present a software-defined file system (SDFS) that exploits the underlying file system to allocate resources at per-image-file granularity and provide tenants with guaranteed throughput. The SDFS comprises two components: control plane and data plane. At the control plane, we provide a set of system calls to document tenant performance requirements into the metadata of image files. At the data plane, we construct a file-based scheduler to manage memory and disk resources according to the tenant performance requirements. The SDFS design does not require a modification to guest operating systems, hypervisors, or file server protocols. Through a prototype implementation, we demonstrate that the SDFS can meet isolation goals and increase resource utilization with negligible overhead.     

基于节的分布式多媒体资源管理

分布式多媒体系统面临的最大挑战是支持稳定的连续媒体流和媒体流间同步，分布式多媒体资源管理模型是保证它们的关键，本文提出了一种基于节的资源分配和管理模型，在资源分配时，它通过建立媒体流的线程，Ｉ／Ｏ处理，缓冲区和网络处理的平衡方程，计算保证稳定的连续媒体所必需的资源，在资源管理时，它通过调整媒体流的速率达到媒体流间同步，本文还提出了一种实时的节资源请求方案，它能系统资源开销最小，最后给出了在交换式１     

Improving multimedia systems performance using constant-density recording disks

Multimedia systems store and retrieve large amounts of data which require extremely high disk bandwidth and their performance critically depends on the efficiency of disk storage. However, existing magnetic disks are designed for small amounts of data retrievals geared to traditional operations; with speed improvements mainly focused on how to reduce seek time and rotational latency. When the same mechanism is applied to multimedia systems, overheads in disk I/O can result in dramatic deterioration in system performance. In this paper, we present a mathematical model to evaluate the performance of constant-density recording disks, and use this model to analyze quantitatively the performance of multimedia data request streams. We show that high disk throughput may be achieved by suitably adjusting the relevant parameters. In addition to demonstrating quantitatively that constant-density recording disks perform significantly better than traditional disks for multimedia data storage, a novel disk-partitioning scheme which places data according to their bandwidths is presented.     

SPIFFI-a scalable parallel file system for the Intel Paragon

This paper presents the design and performance of SPIFFI, a scalable high-performance parallel file system intended for use by extremely I/O intensive applications including “Grand Challenge” scientific applications and multimedia systems. This paper contains experimental results from a SPIFFI prototype on a 64 node/64 disk Intel Paragon. The results show that SPIFFI provides high performance and linear scaleup on real hardware. The paper also explains how shared file pointers (i.e., file pointers that are shared by multiple processes) can simplify the design of a parallel application. By sequentializing I/O accesses and by providing dynamic I/O load balancing, a shared file pointer may even improve an application's performance. This paper also presents the predictions of a SPIFFI simulator that we validated using the prototype. The simulator results show that SPIFFI continues to provide high performance even when it is scaled to configurations with as many as 128 disks or 256 compute nodes     

海量视频数据分布式存储性能优化方法研究

针对文件系统存储效率低、多余副本导致空间浪费以及磁盘能源损耗严重的问题,提出一种新型分布式优化存储策略(distributed optimized storage strategy,DOSS).首先,引入Bcache混合存储技术在磁盘阵列前增设固态硬盘,作为高速缓冲区对多路视频流进行临时数据组织,变多线程并发写任务为单线程顺序写入任务,规避磁盘内部碎片产生,有效提高系统写入效率.其次,提出改进的liberation码对视频数据进行压缩存储,在保证系统可靠性的同时提高磁盘空间利用率.最后,基于ioctl系统调用编写盘片转速控制函数,实现磁盘多级休眠和低延迟唤醒,减低磁盘能耗,提高使用寿命.结果表明单台存储服务器在500路4 Mbps并发视频流下存储效率提高约36％,存储空间节省约40％,系统应对12000路并发视频流时仍存在休眠磁盘约10％.     

Highly available and heterogeneous continuous media storage systems

A number of recent technological trends have made data intensive applications such as continuous media (audio and video) servers a reality. These servers store and retrieve large volumes of data using magnetic disks. Servers consisting of multiple nodes and large arrays of heterogeneous disk drives have become a fact of life for several reasons. First, magnetic disks might fail. Failed disks are almost always replaced with newer disk models because the current technological trend for these devices is one of annual increase in both performance and storage capacity. Second, storage requirements are ever increasing, forcing servers to be scaled up progressively. In this study, we present a framework to enable parity-based data protection for heterogeneous storage systems and to compute their mean lifetime. We describe the tradeoffs associated with three alternative techniques: independent subservers, dependent subservers, and disk merging. The disk merging approach provides a solution for systems that require highly available secondary storage in environments that also necessitate maximum flexibility.     

GPT磁盘克隆成MBR磁盘后数据恢复的研究

用户在使用Ghost工具软件安装操作系统或者克隆磁盘时,有时会将"选择镜像文件到分区"误操作为"选择镜像文件到磁盘";克隆结束后,导致整个GPT磁盘成为一个大C盘,GPT磁盘中各逻辑盘丢失的现象。针对这种情况,以Windows7为平台,Ghost8.0为实验软件,WinHex15.08为数据分析与恢复软件,在虚拟GPT磁盘中建立5个卷,5个卷的文件系统都是NTFS。再制作一个镜像文件,在镜像文件中建立一个MBR分区,对应文件系统也是NTFS。将镜像文件克隆到GPT磁盘中,造成GPT磁盘中原来5个卷对应分区丢失。对克隆后的GPT磁盘结构进行分析,提出了恢复GPT磁盘各卷的基本思路、方法与步骤,即恢复克隆前的GPT头和GPT头备份。实验结果表明:除第1个分区中的部分数据被覆盖无法恢复外,GPT磁盘中剩余4个卷的数据均可完整恢复。     

Efficient striping techniques for variable bit rate continuous media file servers

The performance of striped disk arrays is governed by two parameters: the stripe unit size and the degree of striping. In this paper, we describe techniques for determining the stripe unit size and degree of striping for disk arrays storing variable bit rate continuous media data. We present an analytical model to determine the optimal stripe unit size in redundant and non-redundant disk arrays. We then use the model to study the effect of various system parameters on the optimal stripe unit size. To determine the degree of striping, we first demonstrate that striping a continuous media stream across all disks in the array causes the number of clients supported to increase sub-linearly with increase in the number of disks. To overcome this limitation, we propose a technique that partitions a disk array and stripes each media stream across a single partition. We then propose an analytical model to determine the optimal partition size and maximize the number of clients supported by the array.     

Spatio-temporal effects of multimedia objects storage and delivery for video-on-demand systems

As the number of video streams to be supported by a digital video delivery system (DVDS) increases, an improved understanding of the necessity for reliable and cost-efficient support for a considerable number of video streams (in the magnitude of tens of thousands), and the dependency largely on software capabilities emerges. Even in the presence of an optimal hardware configuration, or model, and associated costs, using software to exploit the underlying hardware capabilities is of paramount importance. Although a number of DVDSs have become operational, their ability to deliver the required services mainly depends on the small number of streams supported and the hardware trade-offs. It is imperative that current software developments account for the eventual scalability of the number of video streams without commensurate increase in hardware. In this paper, we present strategies for the management of video streams in order to maintain and satisfy their space and time requirements. We use a DVDS architectural model with functionally dichotomized nodes: a single-node partition is responsible for data retrieval, while the remaining partition of nodes accepts user requests, determines object locations, and routes requests through the network that connects both partitions. We present a detailed analysis of the issues related to queuing I/O requests and data buffering. The discussion includes the requirements for arranging and scheduling I/O requests and data buffers, with the objective of guaranteeing the required data availability rates for continuous media display.     

SAR数据存储系统中FAT32文件系统开发与实现

针对合成孔径雷达（SAR）技术中高速实时处理、实时存储大容量数据的要求,该文提出SAR数据存储系统中FAT32文件系统实现方案。该方案以SATA硬盘为硬件平台,以嵌入式实时操作系统为软件平台,能够实现实时处理存储功能。