读请求的空间与时间特征建模

为了改善并行应用程序和并行文件系统的I/O性能，对读请求的空间与时间特征进行建模。使用自相关函数分析和Haar小波变换来自动识别自回归、集成的移动平均的模型结构，通过该模型来预测读请求的时间特征；使用马尔可夫模型对读请求的空间特征进行建模、预测。该模型可以将自回归、集成的移动平均时间预测模型与马尔可夫空间预测模型结合，并自适应地预测什么时间、取哪些数据块、取多少数据块。     

Pre-execution data prefetching with I/O scheduling

Parallel applications suffer from I/O latency. Pre-execution I/O prefetching is effective in hiding I/O latency, in which a pre-execution prefetching thread is created and dedicated to fetch the data for the main thread in advance. However, existing pre-execution prefetching works do not pay attention to the relationship between the main thread and the pre-execution prefetching thread. They just simply pre-execute the I/O accesses using the prefetching thread as soon as possible failing to carefully coordinate them with the operations of the main thread. This drawback induces a series of adverse effects on pre-execution prefetching such as diminishing the degree of the parallelism between computation and I/O, delaying the I/O access of main threads, and aggravating the I/O resource competition in the whole system. In this paper, we propose a new method to overcome this drawback by scheduling the I/O operations among the main threads and the pre-execution prefetching threads. The results of extensive experiments on four popular benchmarks in parallel I/O performance area demonstrate the benefits of the proposed approach.     

基于聚类和ARMA时间序列的I/O区域预取

预取是提高存储系统性能的主要手段之一.但现有存储系统的设备层并不知道任何I/O访问的语义信息,因而不能充分利用I/O访问的语义来预取下一时刻要访问的数据,只能利用较简单的方式如I/O访问的局部性、顺序访问和循环访问等特性来实现简单的预测.为此,本文根据存储系统的特点提出了实用且高效的基于连续度的聚类算法来发现密集读请求访问的区域,并采用ARMA时间序列模型来预测密集读请求可能访问的区域及访问时刻,为正确的预取提供了准确的信息.为提高预取的准确性,并采用了动态参数估计的策略.通过大量实验的结果验证了这两种算法的正确性和预测的准确性,能较大的提高存储系统的预取效率.     

Prefetching in file systems for MIMD multiprocessors

The question of whether prefetching blocks on the file into the block cache can effectively reduce overall execution time of a parallel computation, even under favorable assumptions, is considered. Experiments have been conducted with an interleaved file system testbed on the Butterfly Plus multiprocessor. Results of these experiments suggest that (1) the hit ratio, the accepted measure in traditional caching studies, may not be an adequate measure of performance when the workload consists of parallel computations and parallel file access patterns, (2) caching with prefetching can significantly improve the hit ratio and the average time to perform an I/O (input/output) operation, and (3) an improvement in overall execution time has been observed in most cases. In spite of these gains, prefetching sometimes results in increased execution times (a negative result, given the optimistic nature of the study). The authors explore why it is not trivial to translate savings on individual I/O requests into consistently better overall performance and identify the key problems that need to be addressed in order to improve the potential of prefetching techniques in the environment     

多线程环境下基于多预取点的文件预取

为解决当前Linux内核的预取算法在多线程情况下出现预取误判的问题,依据多线程环境下进程对磁盘文件的访问特点,提出一种基于多预取点的预取算法。在Linux内核原有的预取算法的基础上,结合多线程环境下应用程序对数据的访问模式,在Linux内核的页面缓存层进行了实现。实验和分析表明,在IOzone单线程测试中,该算法和Linux内核原预取算法性能相当;在多线程测试中,读取相同大小的文件,耗时比Linux内核原预取算法至少少1/3。新算法对于提高I/O并行度,从而提高整个计算机系统并行化很有帮助。     

Optimal Read-Once Parallel Disk Scheduling

An optimal prefetching and I/O scheduling algorithm L-OPT, for parallel I/O systems, using a read-once model of block references is presented. The algorithm uses knowledge of the next $L$ references, $L$-block lookahead, to create a minimal-length I/O schedule. For a system with $D$ disks and a buffer of capacity $m$ blocks, we show that the competitive ratio of L-OPT is $\Theta(\sqrt{mD/L})$ when $L \geq m$, which matches the lower bound of any prefetching algorithm with $L$-block lookahead. Tight bounds for the remaining ranges of lookahead are also presented. In addition we show that L-OPT is the optimal offline algorithm: when the lookahead consists of the entire reference string, it performs the absolute minimum possible number of I/Os. Finally, we show that L-OPT is comparable with the best online algorithm with the same amount of lookahead; the ratio of the length of its schedule to the length of the optimal schedule is always within a constant factor.     

一种支持并发访问流的文件预取算法

设计并实现了一种按需预取算法,采用更为宽松的顺序性判决条件,并以页面和页面缓存的状态作为可靠的决策依据.它可以发现淹没在随机读中的顺序访问并进行有效的预读,支持对单个文件实例的并发访问而产生的交织访问模式.实验结果表明:相对于原Linux预读算法,该算法在随机干扰下的顺序读性能可提高29%;交织读的性能是传统算法的4~27倍;同时,应用程序可见延迟改善可达35倍.该算法已被Linux 2.6.24内核采用.     

一种自适应的基于预测的I／O性能优化方法

目前计算机的I／O性能已经成为严重制约计算机整体性能提升的瓶颈。在现有技术条件下，I／O性能的优化方法对于提升I／O性能显得尤为重要。本文提出了一种自适应的基于预测的I／O性能的优化方法，该方法采用了基于预测的数据预读策略，改进I／O读操作的响应速度；采用基于预测的缓存分配策略，改进了I／O写操作的响应速度，从而在整体上提高了计算机系统的I／O性能。从实验测试的结果可以看出，这种方法适用于多种不同的负载类型，能够大幅度提升计算机的I／O性能．是一种通用的、效果显著的I／O性能优化方法。     

一种面向视频播放系统的RAID并行预取技术及实现

随着视频数字化技术的广泛应用，视频播放系统中的存在“瓶颈”也日益突出，在分析视频存储系统磁盘阵列的I／O调度算法和在实际应用中视频数据的特点的基础上，提出一种面向视频媒体服务的RAID并行预取实时调度算法，它利用未来数据的可行预测性，预先将其从磁盘取入缓冲区，同时优化任务调度，使主机数据的回送和从磁盘预取数据同步执行，进一步提高了阵列的I／O吞吐率，用I／Ometer测试结果证明，新算法具有很好的性能。     

一种新的自适应并行预取算法

研究了并行存储预取优化算法，根据并行存储的主要访问模式，提出要同时对文件内数据块访问和文件间访问进行建模，并对文件内数据块访问和文件间访问建模分别提出了E_IS_PPM算法和Last_N_Successor算法。最后将两个算法结合起来，提出了文件预取综合算法，算法根据计算和存储的可重叠程度以及文件预取页面的可获得性，自适应地决定预取深度。     

浅析缓存预取技术

针对计算机系统中CPU与I／O性能差距持续扩大的的问题,引出缓存及预取的概念,并分析预取技术需要处理的几个关键问题。根据预取的发展。介绍并比较几类典型的预取算法。针对目前的各种软硬件新技术,探讨预取技术所面临的困难及其发展趋势。     

浅析缓存预取技术

针对计算机系统中CPU与I/O性能差距持续扩大的的问题,引出缓存及预取的概念,并分析预取技术需要处理的几个关键问题。根据预取的发展,介绍并比较几类典型的预取算法。针对目前的各种软硬件新技术,探讨预取技术所面临的困难及其发展趋势。     

核外计算中的几种I/O优化方法

大数据量应用问题引入核外计算模式,由于访问磁盘数据的速度比较慢,I／O成为核外计算性能重要的限制因素．提出了一种使用运行库进行I／O优化的方法,给出了3种有效的优化策略：规则区域筛选、数据预取和边缘重用．编程人员可针对不同的应用问题使用相应的优化API来缩短程序执行时间．实验结果表明,通过减少I／O操作次数和内外存交换的数据量以及隐藏部分I／O操作延迟,有效提高了核外计算的性能．     

Tight bounds for prefetching and buffer management algorithms forparallel I/O systems

The I/O performance of applications in multiple-disk systems can be improved by overlapping disk accesses. This requires the use of appropriate prefetching and buffer management algorithms that ensure the most useful blocks are accessed and retained in the buffer. In this paper, we answer several fundamental questions on prefetching and buffer management for distributed-buffer parallel I/O systems. First, we derive and prove the optimality of an algorithm, P-min, that minimizes the number of parallel I/Os. Second, we analyze P-con, an algorithm that always matches its replacement decisions with those of the well-known demand-paged MIN algorithm. We show that P-con can become fully sequential in the worst case. Third, we investigate the behavior of on-line algorithms for multiple-disk prefetching and buffer management. We define and analyze P-Iru, a parallel version of the traditional LRU buffer management algorithm. Unexpectedly, we find that the competitive ratio of P-Iru is independent of the number of disks. Finally, we present the practical performance of these algorithms on randomly generated reference strings. These results confirm the conclusions derived from the analysis on worst case inputs     

并行文件系统中适度贪婪的Cache预取一体化算法

传统文件系统中的Ｃａｃｈｅ和预取技术是两种降低访问延迟的有效方法。在并行科学计算应用的Ｉ／Ｏ访问模式下,简单的Ｃａｃｈｅ和预取技术已无法提供较高的Ｃａｃｈｅ命中率,该文在分析该Ｉ／Ｏ模式的基础上提出了适度贪婪的Ｃａｃｈｅ和预取一体化算法（ＰＧＩ）,该算法充分利用了并行文件系统环境的特点,采用了适度贪婪的动态滑模技术,可以有铲地消除预取时的抖动,降低系统处理开锁,并同时采用了Ｃａｃｈｅ和预取一体化的     

Real-time integrated prefetching and caching

The high latencies for access to background memory like hard disks or flash memory can be reduced by caching or hidden by prefetching. We consider the problem of scheduling the resulting I/Os when the available fast cache memory is limited and when we have real-time constraints where for each requested data block we are given a time interval during which this block needs to be in main memory. We give a near linear time algorithm for this problem which produces a feasible schedule whenever one exists. Another algorithm additionally minimizes I/Os and still runs in polynomial-time. For the online variant of the problem, we give a competitive algorithm that uses lookahead and augmented disk speed. We show a tight relationship between the amount of lookahead and the speed required to get a competitive algorithm.     

Integrated document caching and prefetching in storage hierarchies based on Markov-chain predictions

Large multimedia document archives may hold a major fraction of their data in tertiary storage libraries for cost reasons. This paper develops an integrated approach to the vertical data migration between the tertiary, secondary, and primary storage in that it reconciles speculative prefetching, to mask the high latency of the tertiary storage, with the replacement policy of the document caches at the secondary and primary storage level, and also considers the interaction of these policies with the tertiary and secondary storage request scheduling. The integrated migration policy is based on a continuous-time Markov chain model for predicting the expected number of accesses to a document within a specified time horizon. Prefetching is initiated only if that expectation is higher than those of the documents that need to be dropped from secondary storage to free up the necessary space. In addition, the possible resource contention at the tertiary and secondary storage is taken into account by dynamically assessing the response-time benefit of prefetching a document versus the penalty that it would incur on the response time of the pending document requests. The parameters of the continuous-time Markov chain model, the probabilities of co-accessing certain documents and the interaction times between successive accesses, are dynamically estimated and adjusted to evolving workload patterns by keeping online statistics. The integrated policy for vertical data migration has been implemented in a prototype system. The system makes profitable use of the Markov chain model also for the scheduling of volume exchanges in the tertiary storage library. Detailed simulation experiments with Web-server-like synthetic workloads indicate significant gains in terms of client response time. The experiments also show that the overhead of the statistical bookkeeping and the computations for the access predictions is affordable. Received January 1, 1998 / Accepted May 27, 1998     

Reference implementation of scalable I/O low-level API on Intel Paragon

The Scalable I/O(SIO)Initiative‘s Low-Level Application Programming Interface(SIO LLAP)provides file system implementers with a simple low-Level interface to support high-level parallel /O interfaces efficiently and effectively.This paper describes a reference implementation and the evaluation of the SIO LLAPI on the Intel Paragon multicomputer.The implementation provides the file system structure and striping algorithm compatible with the Parallel File System(PFS)of Intel Paragon ,and runs either inside the kernel or as a user level library.The scatter-gather addressing read/write,asynchronous I/O,client caching and prefetching mechanism,file access hint mechanism,collective I/O and highly efficient file copy have been implemented.The preliminary experience shows that the SIO LLAPI provides opportunities of significant performance improvement and is easy to implement.Some high level file system interfaces and applications such as PFS,ADIO and Hartree-Fock application,are also implemented on top of SIO.The performance of PFS is at least the same as that of Intel‘s native pfs,and in many cases,such as small sequential file access,huge I/O requests and collective I/O,it is stable and much better,The SIO features help to support high level interfaces easily,quickly and more efficiently,and the cache,prefetching,hints are useful to get better performance based on different access models.The scalability and performance of SIO are limited by the network latency,network scalable bandwidth,memory copy bandwidth,memory size and pattern of I/O requests.The tadeoff between generality and efficienc should be considered in implementation.     

WebCompanion: a friendly client-side Web prefetching agent

Accessing remote sites of the World Wide Web is often a frustrating experience for users because of long Web page retrieval times even over relatively fast Internet connections. Users are more likely to embrace the further expansion of the role of the Web into a major infrastructure for electronic commerce and for information, application, and multimedia delivery if Web accesses can be accelerated. One technique that attempts this is prefetching. We built a client-side Java-implemented prefetching agent, WebCompanion, which employs a novel adaptive, fast, and selective online prefetching strategy based on estimated round-trip times for Web resources. This strategy efficiently hides the access latencies for slow resources while at the same time limiting the network and server overhead and local resource consumption to moderate levels. Our extensive experiments show an average access speedup of greater than 50 percent and an average network byte overhead of less than 150 percent using WebCompanion over a fast Internet connection. We measured a slight acceleration in accessing the Web through WebCompanion even in a pessimistic scenario where the user never requests a prefetched document     

面向流媒体业务的存储设备性能改进策略

本文分析了RAID、I/O调度和预读等技术在提高流媒体存储系统性能方面的局限性。为突破这些传统策略的限制,本文针对存储设备的特点,挖掘磁盘设备的寻道延迟和旋转延迟的提升空间,提出了流的独立存储、潜伏缓存和排练等方法,从多个角度为进一步提升存储系统服务性能提出了新思路。理论分析和实测评估验证了新方法对系统性能的提升效果。