Merging video streams in a multimedia storage server: complexity and heuristics

Due to recent advances in network, storage and data compression technologies, video-on-demand (VOD) service has become economically feasible. It is a challenging task to design a video storage server that can efficiently service a large number of concurrent requests on demand. One approach to accomplishing this task is to reduce the I/O demand to the VOD server through data- and resource-sharing techniques. One form of data sharing is the stream-merging approach proposed in [5]. In this paper, we formalize a static version of the stream-merging problem, derive an upper bound on the I/O demand of static stream merging, and propose efficient heuristic algorithms for both static and dynamic versions of the stream-merging problem.     

Design and analysis of a video-on-demand server

The availability of high-speed networks, fast computers and improved storage technology is stimulating interest in the development of video on-demand services that provide facilities similar to a video cassette player (VCP). In this paper, we present a design of a video-on-demand (VOD) server, capable of supporting a large number of video requests with complete functionality of a remote control (as used in VCPs), for each request. In the proposed design, we have used an interleaved storage method with constrained allocation of video and audio blocks on the disk to provide continuous retrieval. Our storage scheme interleaves a movie with itself (while satisfying the constraints on video and audio block allocation. This approach minimizes the starting delay and the buffer requirement at the user end, while ensuring a jitter-free display for every request. In order to minimize the starting delay and to support more non-concurrent requests, we have proposed the use of multiple disks for the same movie. Since a disk needs to hold only one movie, an array of inexpensive disks can be used, which reduces the overall cost of the proposed system. A scheme supported by our disk storage method to provide all the functions of a remote control such as “fast-forwarding”, “rewinding” (with play “on” or “off”), “pause” and “play” has also been discussed. This scheme handles a user request independent of others and satisfies it without degrading the quality of service to other users. The server design presented in this paper achieves the multiple goals of high disk utilization, global buffer optimization, cost-effectiveness and high-quality service to the users.     

Issues in the design of a storage server for video-on-demand

We examine issues related to the design of a storage server for video-on-demand (VOD) applications. The storage medium considered is magnetic disks or arrays of disks. We investigate disk scheduling policies, buffer management policies and I/O bus protocol issues. We derive the number of sessions that can be supported from a single disk or an array of disks and determine the amount of buffering required to support a given number of users. Furthermore, we propose a scheduling mechanism for disk accesses that significantly lowers the buffer-size requirements in the case of disk arrays. The buffer size required under the proposed scheme is independent of the number of disks in the array. This property allows for striping video content over a large number of disks to achieve higher concurrency in access to a particular video object. This enables the server to satisfy hundreds of independent requests to the same video object or to hundreds of different objects while storing only one copy of each video object. The reliability implications of striping content over a large number of disks are addressed and two solutions are proposed. Finally, we examine various policies for dealing with disk thermal calibration and the placement of videos on disks and disk arrays.     

大型视频点播系统的一种负载均衡算法

讨论了大型视频点播系统的负载均衡技术,提出了一种新的负载均衡算法,该算法包含了优化的负载分配策略和一种新的影片冷／热播自动识别策略、存储调度管理及突发事件处理策略,实际测试表明该算法可行、有效。     

Efficient video file allocation schemes for video-on-demand services

A video-on-demand (VOD) server needs to store hundreds of movie titles and to support thousands of concurrent accesses. This, technically and economically, imposes a great challenge on the design of the disk storage subsystem of a VOD server. Due to different demands for different movie titles, the numbers of concurrent accesses to each movie can differ a lot. We define access profile as the number of concurrent accesses to each movie title that should be supported by a VOD server. The access profile is derived based on the popularity of each movie title and thus serves as a major design goal for the disk storage subsystem. Since some popular (hot) movie titles may be concurrently accessed by hundreds of users and a current high-end magnetic disk array (disk) can only support tens of concurrent accesses, it is necessary to replicate and/or stripe the hot movie files over multiple disk arrays. The consequence of replication and striping of hot movie titles is the potential increase on the required number of disk arrays. Therefore, how to replicate, stripe, and place the movie files over a minimum number of magnetic disk arrays such that a given access profile can be supported is an important problem. In this paper, we formulate the problem of the video file allocation over disk arrays, demonstrate that it is a NP-hard problem, and present some heuristic algorithms to find the near-optimal solutions. The result of this study can be applied to the design of the storage subsystem of a VOD server to economically minimize the cost or to maximize the utilization of disk arrays.     

Performance analysis of a pull-based parallel video server

In conventional video-on-demand systems, video data are stored in a video server for delivery to multiple receivers over a communications network. The video server's hardware limits the maximum storage capacity as well as the maximum number of video sessions that can concurrently be delivered. Clearly, these limits will eventually be exceeded by the growing need for better video quality and larger user population. This paper studies a parallel video server architecture that exploits server parallelism to achieve incremental scalability. First, unlike data partition and replication, the architecture employs data striping at the server level to achieve fine-grain load balancing across multiple servers. Second, a client-pull service model is employed to eliminate the need for interserver synchronization. Third, an admission-scheduling algorithm is proposed to further control the instantaneous load at each server so that linear scalability can be achieved. This paper analyzes the performance of the architecture by deriving bounds for server service delay, client buffer requirement, prefetch delay, and scheduling delay. These performance metrics and design tradeoffs are further evaluated using numerical examples. Our results show that the proposed parallel video server architecture can be linearly scaled up to more concurrent users simply by adding more servers and redistributing the video data among the servers     

VOD视频服务器上节目的存储与调用

在VOD系统中,为了满足多用户在不同时刻点播同一个视频文件,视频节目的存储需要特殊的方法。采用文件分块存储的方法来解决这一问题,分析了磁盘个数、带宽、用户数、视频流数与文件块的大小之间的关系：同时给出了视频文件分块后的存储方法以及调用视频文件时缓冲区的设计、视频文件的并发读取与重组,从而实现在不同时刻发送同一个视频节目到多个客户端。     

视频点播系统中基于流行度的MBGC策略

视频节目的调度策略是视频点播系统中有效使用存储资源和网络资源的关键技术,优化的节目调度和管理可以极大的提高视频服务器的性能。根据流行度来划分视频节目的优先级,提出了一种视频节目调度算法——可移动边界保护信道法(MBGC),并设计了一个二维的Markov链对算法的性能进行了理论上的分析,最后的数值分析结果表明,提出的算法提高了系统的性能。     

Evaluating video layout strategies for a high-performance storage server

We propose a systems approach to providing video service that integrates the multiresolution data generated by scalable compression algorithms with the high-bandwidth, high-capacity storage provided by disk arrays. We introduce two layout strategies for storing multiresolution video data on magnetic disk arrays, which vary in the degrees of parallelism and concurrency they use to satisfy requests. Our simulation results show that the storage of multiple video resolutions allows a video file server to satisfy considerably more user requests than a server that stores a single resolution of video data.     

任务序列强度感知的大规模集群服务器控制模型

异构云数据中心各类服务器的控制成本和性能上的差异将影响其运维管理成本及QoS博弈平衡关系,针对任务序列强度具有的时效性,提出了任务序列强度感知的大规模任务调度模型。依据当前到达数据中心的任务序列的强度以及集群中服务器的当前状态,在任务调度中强调节约服务器运维管理成本和各服务器负载均衡的基础上实现优化数据中心对任务序列处理的平均响应时间和系统的吞吐量。通过对实验结果的分析,验证了集群服务器控制模型在任务调度中的可信度大于95%,同时通过与当前应用广且具代表性的算法——最短任务优先,公平分发机制进行比较分析,其效果是三者中最好的,也验证了模型的有效性和可行性。     

大型拨号认证计费服务器的设计与实现

大型拨号认证计费服务器必须能够同时认证大量上网拨号用户,以及保存用户上网信息以计费。系统要具备良好的扩展性以应对用户规模的增长。给出了一个大型拨号认证计费服务器的设计与实现,设计了一种分布式服务器结构以完成大量认证计费处理工作。系统设计了一种多线程服务器模型以提高系统响应速度,并给出了如何利用RADIUS协议对拨号上网进行认证和计费。本系统采用标准C 实现,通用于各种接入服务器设备,易于扩展系统规模。     

Odyssey: a high‐performance clustered video server

Video servers are essential in video‐on‐demand and other multimedia applications. In this paper, we present our high‐performance clustered CBR video server, Odyssey. Odyssey is a server connecting PCs with switched Ethernet. It provides efficient support for normal play and interactive browsing functions such as fast‐forward and fast‐backward. We designed a set of algorithms for scheduling, synchronization and admission control, which results in a high utilization of resources. Odyssey is able to deliver a large number of video streams. Copyright © 2003 John Wiley & Sons, Ltd.     

大型SMTP服务器的设计与实现

给出了一种大型SMTP服务器系统的分布式结构、SMTP服务器扩展的方法以及MIME格式的电子邮件解析方式,同时给出了邮件的存储方式、邮件列表的结构和邮件列表的处理方式以及系统核心模块的实现代码。在系统实现中采用了面向对象技术和多线程技术,系统完全由标准C 语言实现。该SMTP系统适于大型电子邮件系统应用,它能够同时处理大量并发邮件操作,运行在各种操作系统与硬件平台上。     

Scheduling reentrant jobs on parallel machines with a remote server

This paper explores a specific combinatorial problem relating to reentrant jobs on parallel primary machines, with a remote server machine. A middle operation is required by each job on the server before it returns to its primary processing machine. The problem, which is new to the literature, is inspired by the logistics of a semi-automated microbiology laboratory. We establish the NP-hard nature of the problem, and demonstrate various structural properties. A heuristic is developed and tested on randomly generated benchmark data. No alternative heuristics are available in the literature for comparison, but results indicate solutions reliably within 1.5% of optimum. Moreover, tests of our heuristic on real-life data from the microbiology laboratory provide a 20% improvement in throughput relative to current practice.     

A real-time video watermarking system with buffer sharing for video-on-demand service

Real-time watermarking for streaming video (such as VOD service) requires significant amounts of computing resources. To address this issue, we present a scalable watermarking scheme integrated in a parallel MPEG-2 engine. A content-based block selection algorithm is proposed to efficiently embed the pseudo-random watermark signatures into DCT blocks. Our watermark scheme also provides a robust way to synchronize the watermarked video to the original source at detectors and is very resilient against cumulative and temporal attack.We optimize the parallel watermark engine to achieve real-time watermarking performance. We found that the system throughput could suffer significant degradation when processing high-level MPEG-2 video (such as HDTV) due to inefficient management of memory space. Therefore, we investigated an efficient buffer management scheme consisting of two methods: First we reduced the transmission buffer in slave nodes by frames sharing between frames in the Group-of-Picture (GOP) level. Then we further reduce the buffer space by a dynamic on-demand allocation on the slave side. By solving the memory-shortage bottleneck, the proposed system can support real-time watermarking for multiple high-resolution (up to 1404 × 960) video.     

基于Darwin的多平台多用户直播、点播系统设计

针对当前多平台多用户播放手机客户端直播、录像视频的需求,提出了一种手机移动客户端主动推送视频流到Darwin服务器直播、转播及录制供多平台多用户播放的解决方案。利用开源的流媒体服务器Darwin结合开源库mp4v2实现将手机客户端上传的视频流实时转发和录制;PC平台客户端由开源RTSP架构live555结合开源编解码库ffmpeg实现视频直播、点播播放。Web平台客户端由Html5技术实现视频点播,由插件实现直播。实验结果表明,该方案在现有的3G网络上具有较高的实时性和稳定性,视频具有较好的质量,能满足多平台多用户视频直播、点播播放的要求。     

基于VxWorks的嵌入式网络视频服务器设计

网络视频服务器是一种嵌入式数字远程监控系统,它集视频采集、实时压缩和网络传输等功能为一体,本文提出了基于VxWorks平台的网络视频服务器的实现方案,在介绍实时操作系统VxWorks和MPC860T的基础上,结合音视频编解码芯片AT2042,给出了网络视频服务器的软、硬件实现方法。     

Using rotational mirrored declustering for replica placement in a disk-array-based video server

In a video-on-demand (VOD) environment, disk arrays are often used to support the disk bandwidth requirement. This can pose serious problems on available disk bandwidth upon disk failure. In this paper, we explore the approach of replicating frequently accessed movies to provide high data bandwidth and fault tolerance required in a disk-array-based video server. An isochronous continuous video stream imposes different requirements from a random access pattern on databases or files. Explicitly, we propose a new replica placement method, called rotational mirrored declustering (RMD), to support high data availability for disk arrays in a VOD environment. In essence, RMD is similar to the conventional mirrored declustering in that replicas are stored in different disk arrays. However, it is different from the latter in that the replica placements in different disk arrays under RMD are properly rotated. Combining the merits of prior chained and mirrored declustering methods, RMD is particularly suitable for storing multiple movie copies to support VOD applications. To assess the performance of RMD, we conduct a series of experiments by emulating the storage and delivery of movies in a VOD system. Our results show that RMD consistently outperforms the conventional methods in terms of load-balancing and fault-tolerance capability after disk failure, and is deemed a viable approach to supporting replica placement in a disk-array-based video server.     

基于VxWorks的嵌入式网络视频服务器设计

网络视频服务器是一种嵌入式数字远程监控系统，它集视频采集、实时压缩和网络传输等功能为一体，本文提出了基于VxWorks平台的网络视频服务器的实现方案，在介绍实时操作系统Vxworks和MPC860T的基础上，结合音视频编解码芯片AT2042，给出了网络视频服务器的软、硬件实现方法。