数据包聚合算法提高云计算环境下的网络I/O虚拟化

通过介绍云计算现有环境下网络I/O虚拟化的架构模式,提出一种新的聚合方式来提高云计算环境下网络I/O虚拟化的效率。首先通过分析云计算环境下现有网络I/O虚拟化的过程,进行云环境下的联网性能评估,并在软件和硬件两方面,得出其网络I/O虚拟化的瓶颈。然后,通过提出基于I/O虚拟化的数据包聚合算法,进而大幅度提升云计算环境下网络I/O虚拟化的资源利用率,CPU利用率和内存利用率。最后通过包聚合标注工具,以矩阵的形式,证明该方法的效率。     

面向海量存储的文件系统评测基准

为满足海量存储文件系统级性能评测需求,开发一个通用的符合POSIX.1标准的文件系统基准评测工具LZpack,可为不同文件系统的性能评测比较提供一种有效的方法,也可为使用文件系统操作API集合的应用程序设计者提供性能评价的依据。LZpack采用集群评测的评测架构,可以对文件I/O性能及元数据操作性能进行准确评测。对LZpack的系统结构和关键问题进行描述,对使用LZpack在不同文件系统上的测试结果进行分析,提出LZpack下一步的改进思路。     

SNFS并行文件系统负载均衡技术的研究与实现

在大规模集群系统的并行运算环境中,I/O效率一直是影响系统整体性能的关键因素,并行文件系统技术是目前解决I/O性能瓶颈的有效途径之一。介绍当前并行文件系统的发展现状以及并行文件系统的类型,阐述SNFS并行文件系统的架构以及负载均衡DLC(分布式LAN客户端)技术的实现原理,并给出大规模集群系统环境中SNFS文件系统负载均衡技术的实现方法,最后,通过实际应用分析说明该技术在提升I/O性能上的优势。     

分析文件系统和I/O调度器对SSD性能和寿命的影响

伴随着高性能、高可靠性、低功耗和重量轻等优势,SSD被认为是下一代主流存储设备.文件系统组织和管理文件的实现方式,以及I/O调度器对请求的预处理操作对SSD的性能和寿命有很大的影响.以负载、文件系统、I/O调度器作为测试维度,分析不同负载下的文件系统和调度器的特点以及TRIM优化,并利用Blktrace收集测试过程中发生的每一条I/O操作,用SSDsim评估数据块擦除次数.通过实验发现,在不同的负载下文件系统和调度器对性能和寿命存在显著的差异,Ext4和Deadline表现出更好的综合指标.基于各方面分析,为以后面向SSD文件系统和I/O调度器的设计提供见解.     

IB网上CPU-GPU异构超算平台容器性能评估及优化

为了实现资源和系统环境的隔离,近年来新兴了多种虚拟化工具,容器便是其中之一。在超算资源上运行的问题通常是由软件配置引起的。容器的一个作用就是将依赖打包进轻量级可移植的环境中,这样可以提高超算应用程序的部署效率。为了解基于IB网的CPU-GPU异构超算平台上容器虚拟化技术的性能特征,使用标准基准测试工具对Docker容器进行了全面的性能评估。该方法能够评估容器在虚拟化宿主机过程中产生的性能开销,包括文件系统访问性能、并行通信性能及GPU计算性能。结果表明,容器具备近乎原生宿主机的性能,文件系统I/O开销及GPU计算开销与原生宿主机差别不大。随着网络负载的增大,容器的并行通信开销也相应增大。根据评估结果,提出了一种能够发挥超算平台容器性能的方法,为使用者有针对性地进行系统配置、合理设计应用程序提供依据。     

一种云计算环境下的能效模型和度量方法

提出一种云计算环境下的能效模型和度量方法.首先定义了能效的数学表达及其测量和计算方法,并推导出了能效最大值的发生条件;其次,为方便能效计算,改进了计算机功率和CPU工作状态之间关系的数学表达,通过CPU使用率和频率来计算能效,从而简化了能效测量方法,此外,还设计并实施了大量实验,验证了提出的能效模型的正确性;同时对单机环境,云计算环境中CPU密集型、I/O密集型和交互型运算进行能效评估,总结其能效规律和优化办法,理论和实验证明,所提出的能效模型和计算方法能够准确地评估云系统的能效,并为能效优化奠定基础.     

一种动态优先级排序的虚拟机I/O调度算法

I/O任务调度是影响I/O密集型虚拟机性能的重要因素。现有调度方法主要是针对虚拟机整机I/O带宽的优化,较少兼顾各虚拟域与全局性能,也无法满足域间差异化服务的要求。针对现有方法的不足,提出了一种动态优先级排序的虚拟机I/O调度算法DPS。该算法基于多属性决策理论,以离差最大化方法计算I/O任务的优先级评估属性权重,对I/O任务优先级进行综合评估;通过引入任务所在虚拟域价值,体现云计算环境下虚拟域重要性差异。在Xen系统中通过实验评测DPS调度虚拟化网卡的性能,结果表明,DPS能够有效提高指定域与全局的I/O任务截止期保证率、整机I/O带宽,并能为不同虚拟域的I/O应用提供差异化服务。     

影响文件系统性能的若干因素的实验

影响文件系统的因素有很多,该文从不同访问模式、多通道硬件配置、文件系统老化等方面设计试验,利用多种性能评测工具,对EXT3、XFS两种本地文件系统进行试验,分析这些因素对本地文件系统性能的影响以及作为NFS服务器端的文件系统时,对NFS性能的影响。在试验过程中,编写了测试文件系统元数据性能的Thput benchmark以及使文件系统能够快速老化的工具FastAging。通过分析试验结果总结出针对应用中不同的访问模式配置最优的文件系统的解决方案,使用户能够获得最大的I/O性能。     

I/O负载自相似研究综述

研究I/O负载特征,建立恰当的I/O负载模型,对存储系统的设计、优化和评估有重要的意义.通过研究发现,许多I/O到达模式和访问模式在不同尺度下都具有相似的"突发性".而传统的泊松模型不具有这种特点,因此,研究者提出用自相似模型来描述I/O负载.此外,虽然自相似模型能够较好地描述I/O负载的长相关性,但是,它们却难以描述较小尺度上的局部奇异行为.于是,一些研究者提出用多分形模型来弥补这一缺陷.总结了目前I/O负载自相似研究的成果,分析了I/O负载自相似的研究现状,探讨了I/O负载自相似研究的发展趋势.通过分析得出,目前,自相似的评估问题还未被完全解决,I/O负载的自相似性在存储系统设计中的意义的研究还需要更深入,另外,如何建立更准确高效的I/O负载多分形模型也是一个还需要更深入研究的方面.     

基于对象存储的负载均衡存储策略

对象存储文件系统中将大数据文件分片存储到多个存储节点上,以获取更好的并行I/O性能,提高系统吞吐率.现有对象存储文件系统的存储策略并未充分考虑存储对象本身负载的动态变化,不利于提高系统资源利用率.针对此问题,考虑存储对象的空间及I/O等负载实时变化,提出了一种简单、灵活、高效的负载均衡存储策略,并对该策略进行了实现.实验结果表明,该策略能有效提高对象存储系统资源的利用率和吞吐率,保证对象存储文件系统高效的读写性能.     

Metrics and techniques for quantifying performance isolation in cloud environments

The cloud computing paradigm enables the provision of cost efficient IT-services by leveraging economies of scale and sharing data center resources efficiently among multiple independent applications and customers. However, the sharing of resources leads to possible interference between users and performance problems are one of the major obstacles for potential cloud customers. Consequently, it is one of the primary goals of cloud service providers to have different customers and their hosted applications isolated as much as possible in terms of the performance they observe. To make different offerings, comparable with regards to their performance isolation capabilities, a representative metric is needed to quantify the level of performance isolation in cloud environments. Such a metric should allow to measure externally by running benchmarks from the outside treating the cloud as a black box. In this article, we propose three different types of novel metrics for quantifying the performance isolation of cloud-based systems.We consider four new approaches to achieve performance isolation in Software-as-a-Service (SaaS) offerings and evaluate them based on the proposed metrics as part of a simulation-based case study. To demonstrate the effectiveness and practical applicability of the proposed metrics for quantifying the performance isolation in various scenarios, we present a second case study evaluating performance isolation of the hypervisor Xen.     

Requirements-driven database systems benchmark method

Benchmarks are the vital tools in the performance measurement, evaluation, and comparison of relational database management systems (RDBMS). Standard benchmarks such as the TP1, TPC-A, TPC-B, TPC-C, TPC-D, TPC-H, TPC-R, TPC-W, Wisconsin, and AS³AP benchmarks have been used to assess the performance of relational database management systems. These benchmarks are synthetic and domain-specific. Test results from these benchmarks are estimates of possible system performance for certain pre-determined application types. Database system performance on actual database domain may vary significantly from those in the standard benchmarks. In this paper, we describe a new benchmark method that is computer-assisted and developed from the perspective of the user's requirements.     

几个常见分布式文件系统特征分析和性能对比

近年来随着云计算市场规模不断增长,作为云计算平台基础设施的云存储系统也随之显得越来越重要。数以万计的互联网应用已经运行于云计算环境,同时大量不同的应用也即将从传统运行环境转移到云计算平台。不同的互联网应用的存储需求可能不一样。例如:应用中涉及的单个文件大小,文件数量,IO访问模式,读写比率等,都对底层存储系统提出了不同的要求。这说明在云计算环境中,单个文件系统可能无法满足全部应用的存储需求,本文尝试通过在单一云计算平台中部署多个不同分布式文件系统来优化存储系统的总体性能。为了优化混合式文件系统的性能,首先需要分析不同文件系统的性能特征。本文通过量化方法分析了云计算环境下几个常用的分布式文件系统,这些文件系统分别是ceph,moosefs,glusterfs和hdfs。实验结果显示:即使针对同一文件的相同读写操作,不同分布式文件系统之间的性能也差异显著,当单个文件的大小小于256MB时,moosefs的平均写性能比其它几个文件系统高22.3%;当单个文件大小大于256KB时,glusterfs的平均读性能比其它几个文件系统高21.0%。这些结果为设计和实现一个基于以上几个分布式文件系统的混合式文件系统提供了基础。     

Measuring software dependability by robustness benchmarking

Inability to identify weaknesses or to quantify advancements in software system robustness frequently hinders the development of robust software systems. Efforts have been made to develop benchmarks of software robustness to address this problem, but they all suffer from significant shortcomings. The paper presents the various features that are desirable in a benchmark of system robustness, and evaluates some existing benchmarks according to these features. A new hierarchically structured approach to building robustness benchmarks, which overcomes many deficiencies of past efforts, is also presented. This approach has been applied to building a hierarchically structured benchmark that tests part of the Unix file and virtual memory systems. The resultant benchmark has successfully been used to identify new response class structures that were not detected in a similar situation by other less organized techniques     

Learning to classify parallel input/output access patterns

Input/output performance on current parallel file systems is sensitive to a good match of application access patterns to file system capabilities. Automatic input/output access pattern classification can determine application access patterns at execution time, guiding adaptive file system policies. In this paper, we examine and compare two novel input/output access pattern classification methods based on learning algorithms. The first approach uses a feedforward neural network previously trained on access pattern benchmarks to generate qualitative classifications. The second approach uses hidden Markov models trained on access patterns from previous executions to create a probabilistic model of input/output accesses. In a parallel application, access patterns can be recognized at the level of each local thread or as the global interleaving of all application threads. Classification of patterns at both levels is important for parallel file system performance; we propose a method for forming global classifications from local classifications. We present results from parallel and sequential benchmarks and applications that demonstrate the viability of this approach.     

A generic construct based workload model for business intelligence benchmark

Benchmarks are vital tools in the performance measurement and evaluation of computer hardware and software systems. Standard benchmarks such as the TREC, TPC, SPEC, SAP, Oracle, Microsoft, IBM, Wisconsin, AS³AP, OO1, OO7, XOO7 benchmarks have been used to assess the system performance. These benchmarks are domain-specific in that they model typical applications and tie to a problem domain. Test results from these benchmarks are estimates of possible system performance for certain pre-determined problem types. When the user domain differs from the standard problem domain or when the application workload is divergent from the standard workload, they do not provide an accurate way to measure the system performance of the user problem domain. System performance of the actual problem domain in terms of data and transactions may vary significantly from the standard benchmarks. In this research, we address the issue of domain boundness and workload boundness which results in the ir-representative and ir-reproducible performance readings. We tackle the issue by proposing a domain-independent and workload-independent benchmark method which is developed from the perspective of the user requirements. We present a user-driven workload model to develop a benchmark in a process of workload requirements representation, transformation, and generation. We aim to create a more generalized and precise evaluation method which derives test suites from the actual user domain and application. The benchmark method comprises three main components. They are a high-level workload specification scheme, a translator of the scheme, and a set of generators to generate the test database and the test suite. The specification scheme is used to formalize the workload requirements. The translator is used to transform the specification. The generator is used to produce the test database and the test workload. In web search, the generic constructs are main common carriers we adopt to capture and compose the workload requirements. We determine the requirements via the analysis of literature study. In this study, we have conducted ten baseline experiments to validate the feasibility and validity of the benchmark method. An experimental prototype is built to execute these experiments. Experimental results demonstrate that the method is capable of modeling the standard benchmarks as well as more general benchmark requirements.     

分布式文件系统在线性能测试平台研究与实现

目前由于缺乏针对分布式文件系统的性能测试工具,在很多研究中依然照搬传统的文件系统性能测试工具和方法,很难满足分布式文件系统性能测试的需求(如避免缓存的影响、多客户端并发测试等)。因此,我们研发了一个分布式文件系统在线性能测试平台,此平台通过提供多种大数据量测试负载,有效地避免了系统缓存和系统老化对测试结果的影响,同时支持在线对分布式文件系统执行性能测试,支持多客户端并发测试,并能够对测试结果进行可视化展示和分析。此外,平台也提供了很好的扩展性,满足多种测试需求。     

基于Kubernetes的云原生海量数据存储系统设计与实现

为应对云原生技术的日益发展与普及伴随的云上数据量的激增及该技术在性能与稳定性等方面所出现的瓶颈,提出了一种基于Haystack的存储系统。该存储系统在服务发现、自动容错与缓存方面进行了优化,更适用于云原生业务,以满足数据采集、存储与分析行业不断增长且频次较高的文件存储与读写需求。该存储系统使用对象存储模型来满足高频海量的文件存储,为使用该存储系统的业务提供简单而统一的应用程序接口,应用了文件缓存策略提升资源利用率,同时利用Kubernetes丰富的自动化工具链使该存储系统比其他存储系统更容易部署和扩展且更稳定。实验结果表明,该存储系统在读多于写的大规模碎片数据存储情境下相比目前主流的对象存储与文件系统均有一定的性能与稳定性提升。     

CloudBM:云数据管理系统测试基准

随着云计算技术和云数据管理技术的不断发展,越来越多的云数据管理系统纷纷面世,为人们提供了一种以经济实用的方式管理海量数据的方案。面对各种各样纷繁复杂的云数据管理系统,如何结合不同的应用场景对它们进行全面详细的性能评价是目前亟待解决的挑战性问题。以电信业务中用户访问记录存储系统为应用背景,结合云数据管理系统的应用特点,设计了一个具有广泛代表性的测试基准,并对目前主流的云数据管理系统进行了实际测试。通过对测试结果进行对比分析,为用户提供了一个可供参考的性能评价结果,同时验证了该测试基准的可靠性和广泛适用性。