I/O scheduling for digital continuous media

A growing set of applications require access to digital video and audio. In order to provide playback of such continuous media (CM), scheduling strategies for CM data servers (CMS) are necessary. In some domains, particularly defense and industrial process control, the timing requirements of these applications are strict and essential to their correct operation. In this paper we develop a scheduling strategy for multiple access to a CMS such that the timing guarantees are maintained at all times. First, we develop a scheduling strategy for the steady state, i.e., when there are no changes in playback rate or operation. We derive an optimal Batched SCAN (BSCAN) algorithm that requires minimum buffer space to schedule concurrent accesses. The scheduling strategy incorporates two key constraints: (1) data fetches from the storage system are assumed to be in integral multiples of the block size, and (2) playback guarantees are ensured for frame-oriented streams when each frame can span multiple blocks. We discuss modifications to the scheduling strategy to handle compressed data like motion-JPEG and MPEG. Second, we develop techniques to handle dynamic changes brought about by VCR-like operations executed by applications. We define a suite of primitive VCR-like operations that can be executed. We show that an unregulated change in the BSCAN schedule, in response to VCR-like operations, will affect playback guarantees. We develop two general techniques to ensure playback guarantees while responding to VCR-like operations: passive and active accumulation. Using user response time as a metric we show that active accumulation algorithms outperform passive accumulation algorithms. An optimal response-time algorithm in a class of active accumulation strategies is derived. The results presented here are validated by extensive simulation studies.     

A modular package for efficient I/O operations

We present a package of FORTRAN modules to perform general and efficient I/O operations in external sort, bin sort and related environments. A partition of a user logical record of length IRLU into NREC records of length IRL is carried out when IRLU exceeds the maximum permissible record length for a given computer and file organization. Efficient random direct access to a file requires that IRL be a multiple of the smallest addressable unit on a disk. Overcoming implied DO lists in I/O statements becomes significant in tight memory environments. Corresponding gains in execution times are discussed.     

Energy-aware disk scheduling for soft real-time I/O requests

In this work, we develop energy-aware disk scheduling algorithm for soft real-time I/O. Energy consumption is one of the major factors which bar the adoption of hard disk in mobile environment. Heat dissipation of large scale storage system also calls for an energy-aware scheduling technique to further increase the storage density. The basic idea in this work is to properly determine the I/O burst size so that device can be in standby mode between consecutive I/O bursts and that it can satisfy the soft real-time requirement. We develop an elaborate model which incorporates the energy consumption characteristics, overhead of mode transition in determining the appropriate I/O burst size and the respective disk operating schedule. Efficacy of energy-aware disk scheduling algorithm greatly relies on not only disk scheduling algorithm itself but also various operating system and device firmware related concerns. It is crucial that the various operating system level and device level features need to be properly addressed within disk scheduling framework. Our energy-aware disk scheduling algorithm successfully addresses a number of outstanding issues. First, we examine the effect of OS and hard disk firmware level prefetch policy and incorporate its effect in our disk scheduling framework. Second, our energy aware scheduling framework can allocate a certain fraction of disk bandwidth to handle sporadically arriving non real-time I/O’s. Third, we examine the relationship between lock granularity of the buffer management and energy consumption. We develop a prototype software with energy-aware scheduling algorithm. In our experiment, proposed algorithm can reduce the energy consumption to one fourth if we use energy-aware disk scheduling algorithm. However, energy-aware disk scheduling algorithm increases buffer requirement significantly, e.g., from 4 to 140 KByte. We carefully argue that the buffer overhead is still justifiable given the cost of DRAM chip and importance of energy management in modern mobile devices. The result of our work not only provides the energy efficient scheduling algorithm but also provides an important guideline in capacity planning of future energy efficient mobile devices. This paper is funded by KOSEF through Statistical Research Paper for Complex System at Seoul National University.     

Pre-execution data prefetching with I/O scheduling

Parallel applications suffer from I/O latency. Pre-execution I/O prefetching is effective in hiding I/O latency, in which a pre-execution prefetching thread is created and dedicated to fetch the data for the main thread in advance. However, existing pre-execution prefetching works do not pay attention to the relationship between the main thread and the pre-execution prefetching thread. They just simply pre-execute the I/O accesses using the prefetching thread as soon as possible failing to carefully coordinate them with the operations of the main thread. This drawback induces a series of adverse effects on pre-execution prefetching such as diminishing the degree of the parallelism between computation and I/O, delaying the I/O access of main threads, and aggravating the I/O resource competition in the whole system. In this paper, we propose a new method to overcome this drawback by scheduling the I/O operations among the main threads and the pre-execution prefetching threads. The results of extensive experiments on four popular benchmarks in parallel I/O performance area demonstrate the benefits of the proposed approach.     

应用层并行I/O效率研究

针对划分计算空间到多个计算区域(zone)这类问题,采用基于计算区域组织I/O时,首先需选择计算区域的主进程;其次当某一进程是两个计算区域的主进程时需指定计算区域数据访问先后顺序。设计了多级极大独立集算法实现上述过程。该算法利用图论中的连通图以及极大独立集概念规定了每个计算区域进行数据访问的进程以及优先级。经样例分析,采用此种方法达到I/O并行度最高,并实现在并行度最高情况下通信量最小。     

A DVS-assisted hard real-time I/O device scheduling algorithm

The I/O subsystem has become a major source of energy consumption in a hard real-time monitoring and control system. To reduce its energy consumption without missing deadlines, a dynamic power management (DPM) policy must carefully consider the power parameters of a device, such as its break-even time and wake-up latency, when switching off idle devices. This problem becomes extremely complicated when dynamic voltage scaling (DVS) is applied to change the execution time of a task. In this paper, we present COLORS, a composite low-power scheduling framework that includes DVS in a DPM policy to maximize the energy reduction on the I/O subsystem. COLORS dynamically predicts the earliest-access time of a device and switches off idle devices. It makes use of both static and dynamic slack time to extend the execution time of a task by DVS, in order to create additional switch-off opportunities. Task workloads, processor profiles, and device characteristics all impact the performance of a low-power real-time algorithm. We also identify a key metric that primarily determines its performance. The experimental results show that, compared with previous work, COLORS achieves additional energy reduction up to 20%, due to the efficient utilization of slack time.

自主协助动态并行I/O调度策略研究

随着处理器速度和网络传输速度的快速提升，I／O成为限制计算机性能的一大瓶颈，并行I／O提供了解决这一问题的有效途径。在并行I／O的实现中，数据访问调度策略对系统的性能有着重要的影响。文章通过对集群计算中现有I／O数据访问调度策略的研究，针对调度器瓶颈问题，提出了自主协助动态I／O调度策略，实验结果表明了所提出策略的有效性。     

Heuristics for two-machine no-wait flowshop scheduling with an availability constraint

In this paper we study the two-machine no-wait flowshop problem with an availability constraint. The problem has been shown to be NP-hard, and some heuristics with a worst-case error bound of 2 have been developed for it. We provide two improved heuristics for the problem, and show that each has a worst-case error bound of 5/3.     

Heuristics and metaheuristics for mixed blocking constraints flowshop scheduling problems

Storage or buffer capacities between successive machines in flowshop problems may be unlimited, limited or null. The last two cases can lead to blocking situations. In flowshop scheduling literature, many studies have been performed about classical flowshop problems and also about some problems with only one blocking situation between all machines.     

一种面向重尾分布的SSD磁盘调度算法

存储设备上的大量文件其长度呈重尾态分布,IO请求的响应延迟和请求大小有着密切关系,并且固态硬盘的IO操作不对称。基于以上几点,在内核NOOP调度算法的基础上提出一种针对重尾数据分布下的IO调度算法。该算法通过减少大量小片请求的等待时间,提高固态硬盘的性能。经实验验证,相比内核的NOOP调度算法,平均响应时间减少17%。     

Heuristics for scheduling file-sharing tasks on heterogeneous systems with distributed repositories

We consider the problem of scheduling an application on a computing system consisting of heterogeneous processors and data repositories. The application consists of a large number of file-sharing otherwise independent tasks. The files initially reside on the repositories. The processors and the repositories are connected through a heterogeneous interconnection network. Our aim is to assign the tasks to the processors, to schedule the file transfers from the repositories, and to schedule the executions of tasks on each processor in such a way that the turnaround time is minimized. We propose a heuristic composed of three phases: initial task assignment, task assignment refinement, and execution ordering. We experimentally compare the proposed heuristics with three well-known heuristics on a large number of problem instances. The proposed heuristic runs considerably faster than the existing heuristics and obtains 10–14% better turnaround times than the best of the three existing heuristics.     

Production and vehicle scheduling for ready-mix operations

This paper deals with the problem of selecting and scheduling the orders to be processed by a ready-mix manufacturing plant for immediate delivery to the customer site. Orders have a fixed due date and must be prepared in a single plant with limited capacity. The objective is to maximize the total value of orders served. We first describe the problem as it occurs in practice in some industrial environments, and then present two scenarios to be considered: arbitrary and uniform profit margins for orders. The first scenario corresponds to the fixed job scheduling problem, which is solved by a minimum cost flow problem. A new scheme that reduces the number of nodes in the graph is proposed in this paper. The second scenario on the other hand requires more computationally expensive approaches as a second objective of minimizing the number of vehicles is to be considered. An exact graph-based method and a heuristic branch-and-bound based scheme are described. Computational experiences show that the optimal approach is not feasible for large problems and that the proposed heuristic approach gives high-quality solutions in short running times.     

A general framework for dynamic and automatic I/O scheduling in hard and solid-state drives

The selection of the right I/O scheduler for a given workload can significantly improve the I/O performance. However, this is not an easy task because several factors should be considered, and even the “best” scheduler can change over the time, specially if the workload’s characteristics change too. To address this problem, we present a Dynamic and Automatic Disk Scheduling framework (DADS) that simultaneously compares two different Linux I/O schedulers, and dynamically selects that which achieves the best I/O performance for any workload at any time. The comparison is made by running two instances of a disk simulator inside the Linux kernel. Results show that, by using DADS, the performance achieved is always close to that obtained by the best scheduler. Thus, system administrators are exempted from selecting a suboptimal scheduler which can provide a good performance for some workloads, but may downgrade the system throughput when the workloads change.     

Heuristics for minimizing regular performance measures in unrelated parallel machine scheduling problems

This research compares the performance of various heuristics and one metaheuristic for unrelated parallel machine scheduling problems. The objective functions to be minimized are makespan, total weighted completion time, and total weighted tardiness. We use the least significant difference (LSD) test to identify robust heuristics that perform significantly better than others for a variety of parallel machine environments with these three performance measures. Computational results show that the proposed metaheuristic outperforms other existing heuristics for each of the three objectives when run with a parameter setting appropriate for the objective.     

Heuristics for scheduling problems with an unavailability constraint and position-dependent processing times

We study a single machine scheduling problem, where the machine is unavailable for processing for a pre-specified time period. We assume that job processing times are position-dependent. The objective functions considered are minimum makespan, minimum total completion time and minimum number of tardy jobs. All these problems are known to be NP-hard even without position-dependent processing times. For all three cases we introduce simple heuristics which are based on solving the classical assignment problem. Lower bounds, worst case analysis and asymptotic optimality are discussed. All heuristics are shown numerically to perform extremely well.     

卫星的智能规划与调度

以对地观测卫星为例，分析卫星的结构功能和飞行任务的特点，并在此基础上建立卫星智能规划与调度系统。规划与调度系统由卫星模型和推理机组成，其中模型描述卫星结构功能和各种约束条件，推理机分析并解决这些约束条件，形成一个没有冲突的飞行计划。规划与调度系统还具有修正计划的能力，能满足任务删除、更改和新任务插入等需求。     

Heuristics for the two‐machine scheduling problem with a single server

In this paper, the NP‐hard two‐machine scheduling problem with a single server is addressed. The problem consists of a given set of jobs to be scheduled on two identical parallel machines, where each job must be processed on one of the machines, and prior to processing, the job is set up on its machine using one server; the latter is shared between the two machines. An ant colony optimization (ACO) algorithm is introduced for the problem and its performance was assessed by comparing with an exact solution (branch and bound [B&B]), a genetic algorithm (GA), and simulated annealing (SA). The computational results reflected the superiority of “ACO” in large problems, with a performance similar to SA and GA in smaller problems, while solving the tested problems within a reasonable computational time.     

A schedule of join operations to reduce I/O cost in spatial database systems

In this paper, we propose a graph-based cluster-sequencing method to minimize the I/O cost in spatial join processing. We first define the maximum overlapping (MO) order in a graph, proving that the problem of finding an MO order in a graph is NP-complete. Then, we propose an algorithm to find an approximation to MO order in a graph. We also prove that the approximation to MO order obtained from our method is close to the optimal result. Simulations have been conducted to demonstrate the saving of I/O cost in spatial join by using our method.     

Optimizing I/O server placement for parallel I/O on switch-based irregular networks

In this paper, we study I/O server placement for optimizing parallel I/O performance on switch-based clusters, which typically adopt irregular network topologies to allow construction of scalable systems with incremental expansion capability. Finding optimal solution to this problem is computationally intractable. We quantified the number of messages travelling through each network link by a workload function, and developed three heuristic algorithms to find good solutions based on the values of the workload function. The maximum-workload-based heuristic chooses the locations for I/O nodes in order to minimize the maximum value of the workload function. The distance-based heuristic aims to minimize the average distance between the compute nodes and I/O nodes, which is equivalent to minimizing average workload on the network links. The load-balance-based heuristic balances the workload on the links based on a recursive traversal of the routing tree for the network. Our simulation results demonstrate performance advantage of our algorithms over a number of algorithms commonly used in existing parallel systems. In particular, the load-balance-based algorithm is superior to the other algorithms in most cases, with improvement ratio of 10 to 95% in terms of parallel I/O throughput.