Symmetric active/active metadata service for high availability parallel file systems

High availability data storage systems are critical for many applications as research and business become more data driven. Since metadata management is essential to system availability, multiple metadata services are used to improve the availability of distributed storage systems. Past research has focused on the active/standby model, where each active service has at least one redundant idle backup. However, interruption of service and even some loss of service state may occur during a fail-over depending on the replication technique used. In addition, the replication overhead for multiple metadata services can be very high. The research in this paper targets the symmetric active/active replication model, which uses multiple redundant service nodes running in virtual synchrony. In this model, service node failures do not cause a fail-over to a backup and there is no disruption of service or loss of service state. A fast delivery protocol is further discussed to reduce the latency of the total order broadcast needed. The prototype implementation shows that metadata service high availability can be achieved with an acceptable performance trade-off using the symmetric active/active metadata service solution.     

网络态势分析中服务可用性评估方法

随着网络的迅速发展,网络流量不断增大,各种攻击日趋频繁,在这样的环境下,网络所提供的服务并不是时刻都能保证处于可用状态,需要一个服务评价指标来衡量服务可用性。利用态势评估方法,从节点、系统两个层次进行分析^[4],定义了针对给定特定的系统服务的态势分析函数,从而构建了一个针对网络服务可用性的监测评估方法。该方法能够提供针对某一个特定服务的直观态势分析,并且可以看到系统资源使用情况,使其对系统的安全威胁状况有具体的了解。通过在正常和受DDOS攻击网络中进行实验,验证了该方法的有效性。     

网络服务伪装自动响应模型研究

基于伪装网络服务监测跟踪攻击行为及过程,可以有效研究发现未知攻击方式及其特征。为使伪装网络服务能够自动响应不确定的攻击探测行为,引入了网络服务伪装自动响应模型。该模型对网络会话进行自学习生成状态机,利用经过特征提取的状态机进行模式匹配,进而构造网络会话响应内容。同时利用基于有色事件驱动状态变迁的思想对响应过程进行控制。实验结果验证了该模型的有效性。     

Sorting a distributed file in a network

This paper presents a new algorithm based on Quicksort for sorting in place a distributed file in a message switching network. It discusses in detail the algorithmic aspects of the method and compares it to other possible approaches. The sort is analyzed and is shown to require, on the average, O(M·log(M)) messages and a total data traffic of O(N·log(M)) records for a file of size N fragmented over M stations. Arguments for the performance one may expect in practice, where N is much larger than M, are given. They are supported by results from a sequential simulation.     

Analysis and optimization of service availability in a HA cluster with load-dependent machine availability

Calculations of service availability of a high-availability (HA) cluster are usually based on the assumption of load- independent machine availabilities. In this paper, we study the issues and show how the service availabilities can be calculated under the assumption that machine availabilities are load dependent. We present a Markov chain analysis to derive the steady-state service availabilities of a load-dependent machine availability HA cluster. We show that with a load-dependent machine availability, the attained service availability is now policy dependent. After formulating the problem as a Markov decision process, we proceed to determine the optimal policy to achieve the maximum service availabilities by using the method of policy iteration. Two greedy assignment algorithms are studied: least load and first derivative length (FDL) based, where least load corresponds to some load balancing algorithms. We carry out the analysis and simulations on two cases of load profiles: In the first profile, a single machine has the capacity to host all services in the HA cluster; in the second profile, a single machine does not have enough capacity to host all services. We show that the service availabilities achieved under the first load profile are the same, whereas the service availabilities achieved under the second load profile are different. Since the service availabilities achieved are different in the second load profile, we proceed to investigate how the distribution of service availabilities across the services can be controlled by adjusting the rewards vector.     

Gnutella网络文件共享属性的测量与分析

以Gnutella网络为测量对象,分析了Gnutella网络的基本特点和关键协议,利用Limewire的源代码设计了一个Gnutella网络内容爬虫,测量了Gnutella网络中用户的共享文件类型、共享文件数目、共享空间、文件副本、用户日均在线时长和节点应用层带宽,并对以上测量结果进行统计分析。统计分析表明文件共享数目与共享空间存在显著的相关关系,应用层带宽与日均在线时长则存在弱相关关系。此外,对Gnutella网络中的用户按照共享文件数目、共享空间、应用层带宽、日均在线时长这4个方面进行了聚类分析,分析结果表明,Gnutella网络中的平庸用户占所统计用户数的97%。     

Operating system support for a video-on-demand file service

This paper describes the design and implementation of a continuous media file server intended for use in emerging video-on-demand applications. The main focus and contribution of the paper is in scheduling and admission-control algorithms for accessing the server's processor and storage resources. The scheduling algorithms support multiple classes of tasks with diverse performance requirements and allow for the co-existence of guaranteed real-time requests with sporadic, and unsolicited requests. The scheduler maintains performance guarantees for real-time streams in the presence of unpredictably varying non-real-time traffic while ensuring system stability even during overloads. A prototype video file server was implemented on an Intel 486 platform. Performance results show that a large number of streams can be supported, while maintaining efficient utilization of system resources.     

一个网络文件存储系统TNS关键技术

基于多服务器架构、为多用户服务的网络文件存储系统普遍存在资源分配不均,重复文件多,存储空间浪费严重的问题。设计并实现了TNS网络文件存储系统,该系统基于多服务器存储架构,分别由用户服务器、索引服务器、数据服务器、共享服务器、管理服务器和登录服务器组成,为多用户服务,采用一致性Hash实现负载均衡,支持在客户端进行文件粒度的重复数据删除。经过实际生产环境运行测试,具有良好的负载均衡能力和重复数据删除功能,可以有效节省存储空间,提高存储设备利用率。     

集群文件服务系统中的负载均衡算法的研究

为解决目前集群文件服务系统中负载均衡算法的缺陷,充分考虑文件本身存在的差异,降低大文件分配到负载重的服务器上而造成负载失衡的可能性,避免频繁收集节点的负载信息造成大量额外的网络开销而降低系统吞吐量,提出了改进的负载均衡算法.该算法综合考虑文件负载量、服务器的硬件配置及其实时负载量,引入对上传文件大小和数量的计算,改变服务器负载量定时更新为定量更新.实验结果表明,该算法减少了客户的等待响应时间,提高了系统吞吐量,达到了良好的负载均衡.     

适用于网络流量延时重放的线性文件系统

在分析了传统的网络流量重放和文件系统技术的基础上,提出了一种可应用于入侵检测系统的网络流量延时重放模型,设计并实现了基于Linux的高速线性文件系统.实验结果表明,该线性文件系统磁盘顺序读写性能优于其它主流文件系统,在网络流量延时重放的应用下,系统吞吐量较采用ext2文件系统提高3%,在高速大容量的IP分组存取条件下可有效地减少系统开销,具有一定的实用价值.     

A secure file sharing service for distributed computing environments

Distributed cryptographic file systems enable file sharing among their users and need the adoption of a key management scheme for the distribution of the cryptographic keys to authorized users according to their specific degree of trust. In this paper we describe the architecture of a basic secure file sharing facility relying on a multi-party threshold-based key-sharing scheme that can be overlaid on top of the existing stackable networked file systems, and discuss its application to the implementation of distributed cryptographic file systems. It provides flexible access control policies supporting multiple combination of roles and trust profiles. A proof of concept prototype implementation within the Linux operating system framework demonstrated its effectiveness in terms of performance and security robustness.     

纯用户态的网络文件系统——RUFS

针对在使用高速存储硬件时常规网络文件系统会被软件开销影响整体性能的问题，提出了利用存储性能开发套件（SPDK）搭建文件系统的方法，并在此基础上实现了一个网络文件系统RUFS的原型。该系统通过键值存储模拟文件系统的目录树结构以及对文件系统的元数据进行管理，通过SPDK存储文件的内容。另外，利用远程直接内存访问（RDMA）技术对外提供文件系统服务。RUFS相较于NFS+ext4，在4 KB随机访问上，读写吞吐性能分别提高了202.2%和738.9%，读写平均延迟分别降低了74.4%和97.2%；在4 MB顺序访问上，读写吞吐性能分别提高了153.1%和44.0%。在大部分元数据操作上，RUFS相比NFS+ext4也有显著优势，特别是文件夹创建操作，RUFS的吞吐性能提高了约5 693.8%。该系统能够充分发挥高速网络和高速存储设备的性能优势，为用户提供延时更低、吞吐性能更好的文件系统服务。     

基于CORBA的安全文件共享服务的设计与实现

文件共享服务是企业网络应用环境中一项不可缺少的基本服务。提出了一种基于CORBA的安全文件共享服务模型,介绍了管理服务与共享服务的对象结构、代理机制及它们之间的互操作性,对模型进行了实现,并针对实际使用效果分析了本模型在功能、灵活性及安全性方面的特点。实际结果表明,该模型实现了共享资源的统一管理以及安全灵活的文件访问方法,弥补了传统方法的不足,可以实际应用于大规模的企业网。     

纯用户态的网络文件系统——RUFS

针对在使用高速存储硬件时常规网络文件系统会被软件开销影响整体性能的问题,提出了利用存储性能开发套件（SPDK）搭建文件系统的方法,并在此基础上实现了一个网络文件系统RUFS的原型。该系统通过键值存储模拟文件系统的目录树结构以及对文件系统的元数据进行管理,通过SPDK存储文件的内容。另外,利用远程直接内存访问（RDMA）技术对外提供文件系统服务。RUFS相较于NFS+ext4,在4 KB随机访问上,读写吞吐性能分别提高了202.2%和738.9%,读写平均延迟分别降低了74.4%和97.2%;在4 MB顺序访问上,读写吞吐性能分别提高了153.1%和44.0%。在大部分元数据操作上,RUFS相比NFS+ext4也有显著优势,特别是文件夹创建操作,RUFS的吞吐性能提高了约5 693.8%。该系统能够充分发挥高速网络和高速存储设备的性能优势,为用户提供延时更低、吞吐性能更好的文件系统服务。     

网络编码系统中基于访问频度的数据重建方法

在分布式存储系统中,通常需要在节点失效之后引入新节点并重建数据,以保证系统的可用性。网络编码(Network Coding)存储技术通过数据在存活节点内部作线性组合,可以大幅度降低数据重建时的下载带宽,因此近网络编码技术在节点修复过程中具有非常重要的地位。但同时其大量的线性组合运算也导致了相当可观的时间开销,极大地影响了数据重建的效率和用户的响应请求。基于网络编码文件系统(NCFS),提出了一种结合80-20法则的数据重建方法,并作出了程序实现与仿真验证。实验结果表明,新系统在重建效率、用户平均响应时间及吞吐率方面均有较大提升。     

文件过滤驱动在网络安全终端中的应用

设计并实现了一个运行于Windows系统的网络终端设备信息防泄漏系统。系统通过文件系统过滤驱动的运用,实现了对网络终端机密文件的实时监控和动态加解密,既具有文件加密的安全性,又具有访问控制的方便性,有效地防止了网络终端机密信息的泄漏。     

Wukong: A cloud-oriented file service for mobile Internet devices

Along with the rapid growth of heterogeneous cloud services and network technologies, an increasing number of mobile devices use cloud storage services to enlarge their capacity and share data in our daily lives. We commonly use cloud service client-side software in a straightforward fashion. However, when more devices and users participate in heterogeneous services, the difficulty of managing these services efficiently and conveniently increases. In this paper, we report a novel cloud-oriented file service, Wukong, which provides a user-friendly and highly available facilitative data access method for mobile devices in cloud settings. Wukong supports mobile applications, which may access local files only, transparently accessing cloud services with a relatively high performance. To the best of our knowledge, Wukong is the first file service that supports heterogeneous cloud services for mobile devices by using the innovative storage abstraction layer. We have implemented a prototype with several plugins and evaluated it in a systematic way. We find that this easily operable file service has a high usability and extensibility. It costs about 50 to 150 lines of code to implement a new backend service support plugin. Wukong achieves an acceptable throughput of 179.11 kB/s in an ADSL environment and 80.68 kB/s under a countryside EVDO 3G network with negligible overhead.