面向新型非易失存储的文件系统研究综述

新型非易失存储(NVM)可字节寻址,具有近似内存的低延迟特性以及外存的非易失性,受限于软硬件技术成熟度,目前首先被用于外存.讨论了NVM用于持久性外存所面临的一系列问题,以及管理上的一些挑战;对现有的典型NVM文件系统及其主要特性进行了梳理.归纳起来,这些特性主要围绕降低一致性开销、降低软件栈开销、内存与外存的融合、分布式文件系统、NVM文件系统安全、容错、空间管理几个方面展开.最后,展望了NVM文件系统仍然有待探讨的几个研究方向,包括扩展性问题、虚拟内存与文件系统的有机融合以及分布式文件系统等.     

Performance aspects of distributed caches using TTL-based consistency

The web is the largest distributed database deploying time-to-live-based weak consistency. Each object has a lifetime-duration assigned to it by its origin server. A copy of the object fetched from its origin server is received with maximum time-to-live (TTL) that equals its lifetime duration. In contrast a copy obtained through a cache have shorter TTL since the age (elapsed time since fetched from the origin) is deducted from its lifetime duration. A request served by a cache constitutes a hit if the cache has a fresh copy of the object. Otherwise, the request is considered a miss and is propagated to another server. It is evident that the number of cache misses depends on the age of the copies the cache receives. Thus, a cache that sends requests to another cache would suffer more misses than a cache that sends requests directly to an authoritative server.In this paper, we model and analyze the effect of age on the performance of various cache configurations. We consider a low-level cache that fetches objects either from their origin servers or from other caches and analyze its miss-rate as function of its fetching policy. We distinguish between three basic fetching policies, namely, fetching always from the origin, fetching always from the same high-level cache, and fetching from a “random” high-level cache. We explore the relationships between these policies in terms of the miss-rate achieved by the low-level cache, both on worst-case sequences, and on sequences generated using particular probability distributions.Guided by web caching practice, we consider two variations of the basic policies. In the first variation the high-level cache uses pre-term refreshes to keep a copy with lower age. In the second variation the low-level cache uses extended lifetime duration. We analyze how these variations affect the miss-rates. Our theoretical results help to understand how age may affect the miss-rate, and imply guidelines for improving performance of web caches.     

Performance-reliability issues in distributed file systems

This paper discusses some policies for performance-reliability improvement in distributed file systems. File replication, file migration, and process migration are considered, and the decision whether to replicate a file, or to move a file or a process, is based on the workload demands on the system resources. The utilization of system components, especially that of the communication network, is important, and the system bottleneck and the hosts' bottlenecks are detected. A model of a distributed system is constructed and consists of a number of hosts connected by a communication network. Concurrency control policy allows transaction type access to the shared resources. A unit of concurrency control (i.e., entity to be locked) is a sector of a file. Concurrency control policy is a single writer or multiple readers. Two-phase locking protocol is used for transaction serialization.     

分布式文件系统中元数据操作的优化研究

分析了PVFS2中的元数据操作流程,以remove操作为例,通过测试发现其瓶颈之处,提出一种将判断过程放置到服务器端来减少通信次数的优化方法,这些方法同样可以应用于其他元数据操作。在PVFS2中实现了该优化方法,并将其同原来的remove操作耗时进行了对比,结果表明,该元数据操作的性能提高了10%左右。     

Multiple query scheduling for distributed semantic caches

In distributed query processing systems, load balancing plays an important role in maximizing system throughput. When queries can leverage cached intermediate results, improving the cache hit ratio becomes as important as load balancing in query scheduling, especially when dealing with computationally expensive queries. The scheduling policies must be designed to take into consideration the dynamic contents of the distributed caching infrastructure. In this paper, we propose and discuss several distributed query scheduling policies that directly consider the available cache contents by employing distributed multidimensional indexing structures and an exponential moving average approach to predicting cache contents. These approaches are shown to produce better query plans and faster query response times than traditional scheduling policies that do not predict dynamic contents in distributed caches. We experimentally demonstrate the utility of the scheduling policies using MQO, which is a distributed, Grid-enabled, multiple query processing middleware system we developed to optimize query processing for data analysis and visualization applications.     

基于分布式文件系统的MPP数据库扫描调度研究

基于分布式文件系统的MPP（大规模并行处理）数据库是目前的研究热点,为改善其执行查询扫描操作前调度执行单元读取数据块的过程,提出一种基于节点负载的调度策略NLS。这种策略同时结合数据本地性和节点负载,通过本地读分配保证调度结果满足良好的数据本地性,基于节点的实时工作负载对中间调度结果进行重分配调整,以达到减少数据扫描操作完成时间的目标。实验结果表明,相比连续性调度策略FCS,NLS在保持90%以上数据本地性的同时,在完成时间上的优化最多达到32%,在测试的9种情况中平均优化25%。     

Secure dynamic fragment and replica allocation in large-scale distributed file systems

We present a distributed algorithm for file allocation that guarantees high assurance, availability, and scalability in a large distributed file system. The algorithm can use replication and fragmentation schemes to allocate the files over multiple servers. The file confidentiality and integrity are preserved, even in the presence of a successful attack that compromises a subset of the file servers. The algorithm is adaptive in the sense that it changes the file allocation as the read-write patterns and the location of the clients in the network change. We formally prove that, assuming read-write patterns are stable, the algorithm converges toward an optimal file allocation, where optimality is defined as maximizing the file assurance.     

Coordinated placement and replacement for large-scale distributed caches

In a large-scale information system such as a digital library or the Web, a set of distributed caches can improve their effectiveness by coordinating their data placement decisions. Using simulation, we examine three practical cooperative placement algorithms, including one that is provably close to optimal, and we compare these algorithms to the optimal placement algorithm and several cooperative and noncooperative replacement algorithms. We draw five conclusions from these experiments: 1) cooperative placement can significantly improve performance compared to local replacement algorithms, particularly when the size of individual caches is limited compared to the universe of objects; 2) although the amortizing placement algorithm is only guaranteed to be within 14 times the optimal, in practice it seems to provide an excellent approximation of the optimal; 3) in a cooperative caching scenario, the recent greedy-dual local replacement algorithm performs much better than the other local replacement algorithms; 4) our hierarchical-greedy-dual replacement algorithm yields further improvements over the greedy-dual algorithm especially when there are idle caches in the system; and 5) a key challenge to coordinated placement algorithms is generating good predictions of access patterns based on past accesses.     

Genetic algorithm based approach for file allocation on distributed systems

One of the important features of distributed computing systems (DCSs) is the potential of high reliability. When the hardware configuration of a DCS is fixed, the system reliability mainly depends on the allocation of various resources. One of the important resources used in a DCS are various files. In this paper, we have developed a reliability oriented file allocation scheme for distributed systems. In this scheme various files are allocated to different nodes of a DCS so that the reliability of executing a program which requires files from remote node(s) is maximized. Several variations of this problem are solved to illustrate the Genetic Algorithm based solution approach. The paper also provides the relation between degree of redundancy of files and the maximum achievable reliability of executing a program. The proposed method is compared with optimal solutions to demonstrate the accuracy of the solution obtained from Genetic Algorithm based methodology.     

Sleepy-LRU: extending the lifetime of non-volatile caches by reducing activity of age bits

The Journal of Supercomputing - Emerging non-volatile memories (NVMs) are known as promising alternatives to SRAMs in on-chip caches. However, their limited write endurance is a major challenge...     

基于新型存储器件的分布式文件系统性能优化

新型存储器件的I/O性能通常比传统固态驱动器（SSD）高一个数量级，然而使用新型存储器件的分布式文件系统相对于使用SSD的分布式文件系统性能并没有显著的提高，这说明目前的分布式文件系统并不能充分发挥新型存储器件的性能。针对这个问题，对Hadoop分布式文件系统（HDFS）的数据写入流程及传输过程进行了量化分析。通过量化分析HDFS数据写入过程各阶段的时间开销，发现在写入数据的各个阶段中，节点间数据传输的时间占比较大。因此提出了对应的优化方案，通过异步写入的方式并行化数据传输与处理过程，使得不同数据包的处理阶段叠加起来，减少了数据包整体的处理时间，从而提升了HDFS的写入性能。实验结果表明，所提方案将HDFS的写入吞吐量提升了15%~24%，总体的写入执行时间降低了28%~36%。     

基于新型存储器件的分布式文件系统性能优化

新型存储器件的I/O性能通常比传统固态驱动器（SSD）高一个数量级,然而使用新型存储器件的分布式文件系统相对于使用SSD的分布式文件系统性能并没有显著的提高,这说明目前的分布式文件系统并不能充分发挥新型存储器件的性能。针对这个问题,对Hadoop分布式文件系统（HDFS）的数据写入流程及传输过程进行了量化分析。通过量化分析HDFS数据写入过程各阶段的时间开销,发现在写入数据的各个阶段中,节点间数据传输的时间占比较大。因此提出了对应的优化方案,通过异步写入的方式并行化数据传输与处理过程,使得不同数据包的处理阶段叠加起来,减少了数据包整体的处理时间,从而提升了HDFS的写入性能。实验结果表明,所提方案将HDFS的写入吞吐量提升了15%~24%,总体的写入执行时间降低了28%~36%。     

Performance gain with variable chunk size in GFS-like file systems

We have designed and implemented Tianwang File System（TFS）, which is a distributed file system much like Google File System（GFS）. The system has its origins in our Tianwang search engine and web mining research work. Our system has the same assumptions and the same architectures with GFS. But the key design choice that the chunk size is variable lets our system to adopt simpler system interactions which significantly improves the performance of the record append operation. In this paper, we discuss many aspects of our design which are different from GFS, and verify their pros and cons by performance experiments. The experiment results shows that the utilization ratio of our record append operation excels GFS by 25%. And the throughput of record append of TFS is also several times better.     

The Trellis security infrastructure for overlay metacomputers and bridged distributed file systems

Researchers often have non-privileged access to a variety of high-performance computer (HPC) systems in different administrative domains, possibly across a wide-area network. Consequently, the security infrastructure becomes an important component of an overlay metacomputer: a user-level aggregation of HPC systems.The Trellis security infrastructure (TSI) is layered on top of the widely-deployed secure shell (SSH) and systems administrators only need to provide unprivileged accounts to the users. The contribution of TSI is in demonstrating that a single sign-on (SSO) system, for a variety of use-case scenarios, can be implemented without requiring a completely new security infrastructure. We describe the use of TSI for a Canada-wide overlay metacomputer, for computational workloads (i.e., CISS-3) that spanned 22 administrative domains, at its peak had over 4000 concurrent jobs, and included a new distributed file system (i.e., Trellis NFS).     

The pulse distributed file system

A network of powerful personal computers, linked by a high-speed local area network, is being seen increasingly as an alternative to a traditional centralized time-sharing operating system. The PULSE project is investigating how such a system may be constructed to give the benefits of a self-sufficient personal computer to each user without losing the facilities for communication and sharing of data inherent in centralized systems. In particular, a distributed file system has been built which provides a single global UNIX

1 UNIX is a trademark of Bell Laboratories.

-like hierarchy, with a consistent appearance when accessed from any machine. Replicated copies of files are maintained to improve reliability, increase performance, and enable each machine to run stand-alone, albeit with reduced facilities.     

QoS-based evaluation of file systems and distributed system services for continuous media provisioning

This paper presents a QoS-based performance analysis of file systems and distributed object services for Continuous Media (CM) provisioning, as well as the details and implementation experiences of a continuous media file system and associated CM servers. For this we have implemented QoS-driven CM servers and the Presto continuous media file system (PFS) in the context of a distributed multimedia application development environment, and validate the performance of PFS against that of the conventional Unix file system through an experimental evaluation. Using our CM server prototype, we next examine the effect of continuous media data delivery on the three different kinds of network protocols such as CORBA, UDP/IP, and TCP/IP, with respect to QoS provisioning and throughput.     

Performance estimation for real-time distributed embedded systems

Many embedded computing systems are distributed systems: communicating processes executing on several CPUs/ASICs. This paper describes a performance analysis algorithm for a set of tasks executing on a heterogeneous distributed system. Tight bounds are essential to the synthesis and verification of application-specific distributed systems, such as embedded computing systems. Our bounding algorithms are valid for a general problem model: The system can contain several tasks with hard real-time deadlines and different periods; each task is partitioned into a set of processes related by data dependencies. The periods of tasks and the computation times of processes are not necessarily constant and can be specified by a lower bound and an upper bound. Such a model requires a more sophisticated algorithm, but leads to more accurate results than previous work. Our algorithm both provides tighter bounds and is faster than previous methods     

Sorting a distributed file in a network

This paper presents a new algorithm based on Quicksort for sorting in place a distributed file in a message switching network. It discusses in detail the algorithmic aspects of the method and compares it to other possible approaches. The sort is analyzed and is shown to require, on the average, O(M·log(M)) messages and a total data traffic of O(N·log(M)) records for a file of size N fragmented over M stations. Arguments for the performance one may expect in practice, where N is much larger than M, are given. They are supported by results from a sequential simulation.