网络存储环境下的I／O负载

网络存储改变主机系统与存储设备间的连接关系，利用存储设备的计算能力直接向网络用户提供存储服务。这种结构的改变使系统存储处理模式相应发生变化，导致存储设备 I／O负载特征发生变化。本文提出存储系统负载不仅与应用环境相关，还与系统存储处理模式相关的观点，并据此分析了传统存储系统和网络存储系统的I／O负载特征。     

使用聚集I/O传输的方法提高网络存储系统的性能

网络存储技术为存储系统提供了更高的扩展性、可用性以及更好的灵活性,并且已经广泛应用到各种领域．但小I／O严重影响网络存储系统的性能,现有的一些方法缺少理论模型．通过排队理论建立聚集I／O传输的数学模型,然后在典型的IP网络存储(NAS)系统中,分别在NAS存储设备和客户两端设计了聚集模块,称之为CFS(chunk file system),实现双向聚集I／O．最后对CFS进行了实验,结果和理论分析基本吻合,说明聚集I／O传输能够改善网络存储整体性能．     

基于SAN系统数据密集型程序的性能分析

数据密集型应用如多媒体、数据库等等的大规模发展对存储系统提出更高的要求．与之相适应的是以SAN为代表的存储网络改变了存储系统完全依赖于主机的模式，整体上提高了存储系统性能、可用性、可扩展性．本文在研究整个SAN存储系统结构的基础上，通过分析和估计SAN系统的各个部分性能，结合数据密集型程序的I／O统计特性，估计各种I／O请求的平均响应时间，说明在SAN系统平台上I／O密集型程序的设计要求，为进一步开发大规模存储网络管理和应用软件打下基础．     

网络存储I/O流水机制研究

将I／O请求处理划分为多个阶段，为流水线技术引入网络存储提供了新思路．同时，I／O请求处理的各阶段必定通过缓存（内存）来传递或处理数据．I／O请求处理除了直通方式外，大部分依赖存储转发方式（例如对I／O命令的聚散、排队操作等），存储转发的方式下的网络存储I／O流水线具有一些新的特点厦其自身特有的制约因素．探讨I／O流水机制，对提高网络存储系统整体性能，具有一定的指导及实践意义．     

基于RAM/Disk混合设备模型的FC-SAN存储系统

磁盘存取是基于光纤通道网络的SAN存储系统的目前性能瓶径，在综合和分析目前各种文件系统I／O操作工作负载的研究结果的基础上，提出了一个新的改进FC-SAN存储系统性能的方法：将各种文件系统I／O操作分为大数据量的文件读写操作、小数据量的文件读写操作和文件属性操作，大数据量的文件读写操作还是按照原来的I／O路径进行，存取物理磁盘；但其他各种文件操作包括小数据量的文件读写操作对基于内存的RAMDisk设备进行操作，实验结果显示，基于混合I／O子系统的FC-SAN存储系统的存取速率可以接近线速。     

可扩展的附网存储系统的研究与实现

随着网络数字信息爆炸性的增长和应用高可用性的需求，海量、高可扩用性、高可展性的存储系统正成为网络存储领域的研究热点．针对这种情况，我们借助存储虚拟化和冗余技术，将多个NAS存储服务器器融合成一个可扩展的大NAS(SNAS：Scalable NAS)．在SNAS中：通过SNAS_UIOS_FS文件系统能将多个NAS融合成一个大的NAS，并向用户提供一个统一的存储视图；通过SNAS_LOADBA_FS文件系统能在多个NAS存储服务器之间提供负载均衡；通过iSCSI协议模块使SNAS能同时提供文件10和块10，它融合了NAS和SAN的优点．实验结果显示SNAS有超高速的文件I／O和块I／O响应速度．     

SANS--IP-SAN模拟器的设计与实现

随着存储系统的发展,存储区域网络成为高端存储系统的主流,它具有网络和存储的特点,有更高的性能和更强的灵活性。FC—SAN和IP—SAN是基于不同协议实现的存储区域网络,随着高速IP的发展,IP-SAN具有更好的发展前景。目前尚没有根据IP—SAN的特点设计的模拟器,该文总结了IP-SAN的特点,并在NS2的基础上,增加对应的协议(iSCSI)和存储节点,实现了可以模拟存储区域网络特性的IP—SAN模拟器,为进一步研究存储网络的I／O性能和调度策略等奠定了基础。最后利用该模拟器,对单路径和双路径传输的性能进行了对比实验。     

基于存储熵的存储负载均衡算法

在分布式存储系统中,一般都是以磁盘空间利用率（DU）来判断各存储节点的负载均衡程度,当所有节点的磁盘空间利用率相等时,是整个分布式存储系统的存储负载均衡点。但是在实际的应用场景中,磁盘I/O速率比较低的存储节点和可靠性比较低的存储节点往往成为影响整个存储系统数据读写性能的瓶颈,因此在异构分布式存储系统中,特别是各存储节点磁盘I/O速率和可靠性差异较大的分布式存储系统中,如果仅仅以磁盘空间利用率作为存储负载均衡的判定条件,则其数据的读写效率必然受到限制。从读写效率的角度提出一种度量分布式存储系统中存储负载均衡的新思路。根据负载均衡理论和熵理论给出存储熵（SE）的定义,并提出一种基于存储熵的负载均衡算法,该算法通过系统负载判定、单节点负载判定和负载迁移实现了对分布式存储系统存储负载的量化调整,并通过实验与基于磁盘空间利用率的负载均衡算法进行了对比分析,验证了该算法对分布式存储系统中存储负载具有良好的均衡性,有效地控制了系统负载失衡的问题,提高了分布式存储系统的整体读写效率。     

SCSI子系统中间层多启动互连多路径I/O的存储方式的研究

面对当前应用对现代存储的高性能以及可用性、可靠性的新的要求，在Multi—Initiator—Uni—Port的基础之上提出了SCSI中间层多启动互连多路径I／O的存储方式．它能够在Multi—Initiator—Uni—Port的连接中对同一个Target端口采用不同的I／O路径进行存储，减少了由于单路径I／O传输错误导致系统单点失效的可能性，增强了存储系统的可用性和可靠性；同时，这种存储方式还能够很好地平衡系统的I／O负载，提高系统的吞吐率，提供比传统单路径更好的传输性能．在Linux2．4内核中的SCSI子系统的基础之上实现了这种存储方式，向系统屏蔽了SCSI子系统中的设备具体细节，并且不依赖于SCSI上层的实现，减少了对系统内核的资源占有率，提高了SCSI底层的执行效率．分析了SCSI中间层多启动互连多路径I／O的存储方式的原理，并且将iSCSI的体系结构应用到它的模拟测试中，得到了R—R路径选择分配策略的性能临界点，取得了令人满意的结果．     

一种基于磁盘介质的网络存储系统缓存

随着计算规模越来越大,网络存储系统应用领域越来越广泛,对网络存储系统I/O性能要求也越来越高.在存储系统高负载的情况下,采用低速介质在客户机和网络存储系统的I/O路径上作为数据缓存也变得具有实际的意义.设计并实现了一种基于磁盘介质的存储系统块一级的缓存原型D-Cache.采用两级结构对磁盘缓存进行管理,并提出了相应的基于块一级的两级缓存管理算法.该管理算法有效地解决了因磁盘介质响应速度慢而带来的磁盘缓存管理难题,并通过位图的使用消除了磁盘缓存写Miss时的Copy on Write开销.原型系统的测试结果表明,在存储服务器高负载的情况下,缓存系统能够有效地提高系统的整体性能.     

负载感知的存储子系统调优研究

存储IO的性能远远低于CPU、内存的性能，而且它们之间的差距还在扩大。因此，对于越来越多的数据密集型应用系统来说，存储系统往往是系统瓶颈。存储的性能不仅与存储子系统体系结构以及子系统中各部件的性能相关，还与系统的工作负载和应用环境相关。负载感知的性能调优指存储子系统通过对负载特征的分析实现对应用环境的动态感知，并根据负载特征动态调整系统运行策略。负载感知的性能调优使得存储子系统能够更合理地调度存储系统资源，从而提高IO性能。     

A dynamic resource management in mobile agent by artificial neural network

In this paper, a resource management for dynamic load balancing in mobile agent by artificial neural network scheme (ANN-DLB) is presented to maximize the number of the served tasks in developing high performance cluster. This dynamic load balance with the growth of the service type and user number in the mobile networks of the higher performance is required in service provision and throughput. Most of the conventional policies are used in load indices with the threshold value to decide the load status of the agent hosts by CPU or memory. The main factor influencing the workload is the competitions among the computing resources such as CPU, memory, I/O and network. There are certain I/O data of the intensive applications where load balancing becomes the important issue. This relationship between the computing resources is very complex to define the rules for deciding the workload. This paper proposed a new dynamic load balancing for evaluating the agent hosts’ workload with the artificial neural network (ANN). By applying the automatic learning of the back-propagation network (BPN) model can establish the ANN model and also can measure the agent host loading with five inputs: CPU, memory, I/O, network and run-queue length. The structure of the load balancing system is composed of three design agents: the load index agent (LIA), the resource management agent (RMA) and the load transfer agent (LTA). These experimental results reveal that the proposed ANN-DLB yields better performance than the other methods. These results demonstrate that the proposed method has high throughput, short response time and turnaround time, and less agent host negotiation complexity and migrating tasks than the previous methods.     

Architectural Effects of Symmetric Multiprocessors on TPC-C Commercial Workload

Commercial transaction processing applications are an important workload running on symmetric multiprocessor systems (SMPs). They differ dramatically from scientific, numeric-intensive, and engineering applications because they are I/O bound, and they contain more system software activities. Most SMP servers available in the market have been designed and optimized for scientific and engineering workloads. A major challenge of studying architectural effects on the performance of a commercial workload is the lack of easy access to large-scale and complex database engines running on a multiprocessor system with powerful I/O facilities. Experiments involving case studies have been shown to be highly time-consuming and expensive. In this paper, we investigate the feasibility of using queuing network models with the support of simulation to study the SMP architectural impacts on the performance of commercial workloads. We use the commercial benchmark TPC-C as the workload. A bus-based SMP machine is used as the target platform. Queueing network modeling is employed to characterize the TPC-C workload on the SMP. The system components such as processors, memory, the memory bus, I/O buses, and disks are modeled as service centers, and their effects on performance are analyzed. Simulations are conducted as well to collect the workload-specific parameters (model parameterization) and to verify the accuracy of the model. Our studies find that among disk-related parameters, the disk rotation latency affects the performance of TPC-C most significantly. Among I/O buses and number of disks, the number of I/O buses has the deepest impact on performance. This study also demonstrates that our modeling approach is feasible, cost-effective, and accurate for evaluating the performance of commercial workloads on SMPs, and it is complementary to the measurement-based experimental approaches.     

基于人工免疫安全存储区域网系统的研究

安全存储区域网系统需要保护海量数据,使用现有安全技术存在安全开销过大和存储空间浪费较多等问题,使得现有安全存储区域网系统I/O性能较低,无法使用.人工免疫算法能高效地识别异常数据,为了使安全存储区域网系统能保持较高的I/O性能,本文引入人工免疫算法,研究安全存储区域网系统.给出安全存储区域网中安全结点和基于博弈安全优化模块的结构;再针对安全存储区域网系统的特点,设计结点安全模块中的主要算法,并引入博弈论设计安全存储区域网系统优化算法.实现了结点安全模块的原型系统,测试安全模块的检测效率,验证了结点安全模块在具有较好安全性能的同时,具有安全开销较小且稳定的特点.最后在Lustre上实现了安全存储区域网原型系统,使用通用工具测试的系统I/O性能,结果表明所实现的安全存储区域网原型系统具有能保持较高I/O性能的特性.     

一种NAS系统的设计模型与性能分析研究

与传统的存储服务器相比，附网存储体系具有诸多的优点．它实现了数据在存储设备到客户机之间的直接传输，提高了系统的整体性能，是目前新型的分布式存储解决方案．文中分析了基于附网存储的数据传输特点，并在此基础上提出了系统实现的设计模型．同时，我们对两种不同的存储方案造成的性能差异进行了研究，结果表明，附网存储系统能够有效地减轻服务器的负载，更充分地利用网络资源，如磁盘中缓存、磁盘处理器性能等．     

Optimizing I/O server placement for parallel I/O on switch-based irregular networks

In this paper, we study I/O server placement for optimizing parallel I/O performance on switch-based clusters, which typically adopt irregular network topologies to allow construction of scalable systems with incremental expansion capability. Finding optimal solution to this problem is computationally intractable. We quantified the number of messages travelling through each network link by a workload function, and developed three heuristic algorithms to find good solutions based on the values of the workload function. The maximum-workload-based heuristic chooses the locations for I/O nodes in order to minimize the maximum value of the workload function. The distance-based heuristic aims to minimize the average distance between the compute nodes and I/O nodes, which is equivalent to minimizing average workload on the network links. The load-balance-based heuristic balances the workload on the links based on a recursive traversal of the routing tree for the network. Our simulation results demonstrate performance advantage of our algorithms over a number of algorithms commonly used in existing parallel systems. In particular, the load-balance-based algorithm is superior to the other algorithms in most cases, with improvement ratio of 10 to 95% in terms of parallel I/O throughput.     

影响VoD服务器I/O性能的关键因素

在讨论了基于网络视频点系统模型和通用服务器体系结构的基础上，主要分析了影响视频点播服务器性能I／O能力的几个关键因素，对存储设备的吞吐量，PCI总线速率，SCSI通道的速率和网卡的传输速度等对服务器I／O性能的影响进行了较全面，深入的分析，最后提高一种利用服务器集群的方式提高VoD系统I／O能力的方法。     

Accelerating big data analytics on HPC clusters using two-level storage

Data-intensive applications that are inherently I/O bound have become a major workload on traditional high-performance computing (HPC) clusters. Simply employing data-intensive computing storage such as HDFS or using parallel file systems available on HPC clusters to serve such applications incurs performance and scalability issues. In this paper, we present a novel two-level storage system that integrates an upper-level in-memory file system with a lower-level parallel file system. The former renders memory-speed high I/O performance and the latter renders consistent storage with large capacity. We build a two-level storage system prototype with Tachyon and OrangeFS, and analyze the resulting I/O throughput for typical MapReduce operations. Theoretical modeling and experiments show that the proposed two-level storage delivers higher aggregate I/O throughput than HDFS and OrangeFS and achieves scalable performance for both read and write. We expect this two-level storage approach to provide insights on system design for big data analytics on HPC clusters.