QoS in grid computing

Grid computing is already a mainstream paradigm for resource-intensive scientific applications, but it also promises to become the future model for enterprise applications. The grid enables resource sharing and dynamic allocation of computational resources, thus increasing access to distributed data, promoting operational flexibility and collaboration, and allowing service providers to scale efficiently to meet variable demands. Large-scale grids are complex systems composed of thousands of components from disjoined domains. Planning the capacity to guarantee quality of service (QoS) in such environments is a challenge because global service-level agreements (SLAs) depend on local SLAs. We provide a motivating example for grid computing in an enterprise environment and then discuss the how resource allocation affects SLAs.     

Feedback-based optimization of a private cloud

The optimization problem addressed by this paper involves the allocation of resources in a private cloud such that cost to the provider is minimized (through the maximization of resource sharing) while attempting to meet all client application requirements (as specified in the SLAs). At the heart of any optimization based resource allocation algorithm, there are two models: one that relates the application level quality of service to the given set of resources and one that maps a given service level and resource consumption to profit metrics. In this paper we investigate the optimization loop in which each application’s performance model is dynamically updated at runtime to adapt to the changes in the system. These changes could be perturbations in the environment that had not been included in the model. Through experimentation we show that using these tracking models in the optimization loop will result in a more accurate optimization and thus result in the generation of greater profit.     

Dynamically adapting to system load and program behavior in multiprogrammed multiprocessor systems

Parallel execution of application programs on a multiprocessor system may lead to performance degradation if the workload of a parallel region is not large enough to amortize the overheads associated with the parallel execution. Furthermore, if too many processes are running on the system in a multiprogrammed environment, the performance of the parallel application may degrade due to resource contention. This work proposes a comprehensive dynamic processor allocation scheme that takes both program behavior and system load into consideration when dynamically allocating processors. This mechanism was implemented on the Solaris operating system to dynamically control the execution of parallel C and Java application programs. Performance results show the effectiveness of this scheme in dynamically adapting to the current execution environment and program behavior, and that it outperforms a conventional time‐shared system. Copyright © 2002 John Wiley & Sons, Ltd.     

SLA-aware energy-efficient scheduling scheme for Hadoop YARN

Apache Hadoop becomes ubiquitous for cloud computing which provides resources as services for multi-tenant applications. YARN (a.k.a. MapReduce 2.0) is one of the key features in the second-generation Hadoop, which provides resource management and scheduling for large-scale MapReduce environments. Two enormous challenges in the YARN scheduler are the abilities to automatically tailor and control resource allocations to different jobs for achieving their Service Level Agreements (SLAs), and minimize energy consumption of the overall cloud computing system. In this work, we propose an SLA-aware energy-efficient scheduling scheme which allocates appropriate amount of resources to MapReduce applications with YARN architecture. In our task scheduling policy, We consider the data locality information to save the MapReduce network traffic. Furthermore, the slack time between the actual execution time of completed tasks and expected completion time of the application is utilized to improve the energy-efficiency of the system. An online userspace governor-based dynamic voltage and frequency scaling (DVFS) scheme is designed in the YARN per-application ApplicationMaster to dynamically change the CPU frequency for upcoming tasks given the slack time from previous completed tasks. Experimental evaluation shows that our proposed scheme outperforms the existing MapReduce scheduling policies in terms of both resource ultization and energy-efficiency.     

混合计算环境中截止期约束下的科学工作流调度策略

整合云和网格基础设施,增强科研机构现有网格系统的计算能力并向应用提供截止时间保障的服务是科学研究领域的热点。在这种"网格-云"混合计算环境中,对何时租借云虚拟资源以及如何租借做出有效决策是一个难题。现有的一些调度策略主要在网格资源静态能力特征的基础上,以作业等待时间作为决策依据,缺乏对资源动态服务能力的有效评估,无法保证科学应用的截止时间需求。本文提出了一种混合环境下的科学工作流执行系统架构并对其核心组件进行了阐述。针对其中的工作流调度问题,利用随机服务模型建模已有网格系统中的资源的动态服务能力,以任务违约风险作为是否租借外部虚拟资源的判断指标,提出了一个科学工作流调度算法HCA_SASWD。实验结果表明,HCA_SASWD相对于其他算法,能有效保证用户的截止时间要求,为需要提供截止时间保障的系统架构提供了参考。     

超算环境科学工作流应用平台的引擎设计和资源调度

高性能计算机体系结构的复杂性对使用者提出了更高要求;而且在工程实际和科学实验中,通常需要使用多种应用软件相互协作才能解决复杂问题。围绕超算资源的易用性和多类软件的集成以及协作需求,开发了超算环境下的科学工作流应用平台,设计了异步并发的流程执行引擎,采取调度算法和调度器、引擎相分离的设计策略,给出了资源调度方案。提出了局部资源池化技术和资源预约算法,并比较分析了五种常用调度算法的性能,给出了算法选择的建议。实际应用表明设计的引擎能够支撑复杂工作流的灵活执行方式,给出的资源调度方案能够满足超算环境下工作流应用的高效执行。     

基于任务-资源分配图优化选取的网格依赖任务调度

任务调度是网格应用系统获得高性能的关键.网格计算中一个大型的应用程序往往被分解为具有依赖关系的多个任务.在资源个体差异较大、广域互连的网格环境下任务间的依赖关系对传统的调度策略提出了新的挑战.任务调度的主要工作是为任务分配资源以及确定任务的执行次序,将依赖任务的可能的资源分配方案表示为任务-资源分配图（T-RAG）,在该图的基础上提出了基于T-RAG优化选取的依赖任务调度模型,将依赖任务调度问题转化为图的优化选取问题,解析最优任务-资源分配图可以同时确定资源分配方案和任务的执行次序即为最优调度方案.最后,实现了基于该模型的任务调度算法,该算法与ILHA算法的对比分析表明,在资源差异较大及任务间存在大量数据传输的情况下所提出的算法更优.     

Semantic-enabled CARE Resource Broker (SeCRB) for managing grid and cloud environment

Grid computing is mainly helpful for executing high-performance computing applications. However, conventional grid resources sometimes fail to offer a dynamic application execution environment and this increases the rate at which the job requests of users are rejected. Integrating emerging virtualization technologies in grid and cloud computing facilitates the provision of dynamic virtual resources in the required execution environment. Resource brokers play a significant role in managing grid and cloud resources as well as identifying potential resources that satisfy users’ application requests. This research paper proposes a semantic-enabled CARE Resource Broker (SeCRB) that provides a common framework to describe grid and cloud resources, and to discover them in an intelligent manner by considering software, hardware and quality of service (QoS) requirements. The proposed semantic resource discovery mechanism classifies the resources into three categories viz., exact, high-similarity subsume and high-similarity plug-in regions. To achieve the necessary user QoS requirements, we have included a service level agreement (SLA) negotiation mechanism that pairs users’ QoS requirements with matching resources to guarantee the execution of applications, and to achieve the desired QoS of users. Finally, we have implemented the QoS-based resource scheduling mechanism that selects the resources from the SLA negotiation accepted list in an optimal manner. The proposed work is simulated and evaluated by submitting real-world bio-informatics and image processing application for various test cases. The result of the experiment shows that for jobs submitted to the resource broker, job rejection rate is reduced while job success and scheduling rates are increased, thus making the resource management system more efficient.     

An End-to-end Workflow Pipeline for Large-scale Grid Computing

In this paper we describe a service-based, software architecture that enables end-to-end, high-level workflow processing in a Grid environment consisting of many heterogeneous resources. Our architecture is essentially a pipeline that extends from the abstract application specification phase to the deployment and execution stages through to returning the results to the user. We envision a large-scale Grid environment that contains heterogeneous resources. Our architecture caters for flexible deployment, performance, reliability and charging for resource usage. These are addressed at the specification level as well as at the realisation (brokering) and execution levels. The proposed architecture is derived from previous work in LeSC that has produced the ICENI pipeline, and our experience with e-Science projects, such as GENIE, e-Protein and RealityGrid from which we derive a set of key requirements.     

面向服务网格的虚拟环境部署运行管理系统

虚拟技术的最新进展为网格计算提供了封装资源的新方式,其封装性、隔离性和安全性能够有效屏蔽底层资源的异构性,根据用户应用需求定制执行环境,更好地适应于网格环境的复杂性和应用的多样性。为了满足当前服务网格的需求发展,基于新的虚拟机技术,研究适合于服务网格的虚拟环境部署运行管理系统,该系统为用户提供可视化、易操作的远程虚拟环境部署和运行管理功能;并实现一个标准的网格服务,结合服务网格平台CROWN,该服务可根据用户应用的特定需求动态透明地部署虚拟执行环境,并根据资源状态自适配调度执行用户任务。并对系统进行了实验分析,实验结果验证了系统的良好可用性和运行性能。     

Policy-Based Management for Federation of Virtualized Infrastructures

This paper presents Policy-based Federation (PBF) architecture for interworked Future Internet Virtualized Infrastructures (VIs). Each VI is an individually managed autonomous domain. Users may request slices of virtual resources across the federation, managed and controlled via inter-domain policies that abide by agreed upon federated SLAs. The key component of our PBF architecture is a Policy Service, which provides support for intra-domain policies (Obligation, Authorization, Role-Based Access Control) and for inter-domain Delegation policies. Delegation policies reserve resources in remote domains, update the number of resources exchanged, set alien domain obligations for cross-domain resource provisioning and define the exchange of internal domain information through the execution of remote semantic queries. Key to the architecture is the PBF Policy Ontology that specifies common federation concepts within the context of a user slice and the PBF services that trigger management actions. A prototype of the proposed architecture was developed and deployed in a European Future Internet federated testbed.     

一种支持Java应用中计算按需远程执行的方法

按需远程执行是软件应用实现对资源按需占有,从而保障性能并提高资源利用率的重要手段。给出了一种通过自动程序转换来支持Java应用中计算按需远程执行的方法,其核心是支持计算按需远程执行的设计模式。介绍了将Java应用转换成该模式所面临的技术挑战、处理机制以及DPartner转换系统。与已有工作相比,DPartner有两大特色：一是程序转换自动执行;二是转换后应用可实现真正按需的远程执行,使性能和资源利用率得以提升。此外,DPartner被设计为可对只有Java字节码的遗产应用进行转换,更具实用性。     

Dependable Grid Workflow Scheduling Based on Resource Availability

Due to the highly dynamic feature, dependable workflow scheduling is critical in the Grid environment. Various scheduling algorithms have been proposed, but seldom consider the resource reliability. Current Grid systems mainly exploit fault tolerance mechanism to guarantee the dependable workflow execution, which, however, wastes system resources. The paper proposes a dependable Grid workflow scheduling system (called DGWS). It introduces a Markov Chain-based resource availability prediction model. Based on the model, a reliability cost driven workflow scheduling algorithm is presented. The performance evaluation results, including the simulation on both parametric randomly generated DAGs and two real scientific workflow applications, demonstrate that compared to present workflow scheduling algorithms, DGWS improves the success ratio of tasks and diminishes the makespan of workflow, so improves the dependability of workflow execution in the dynamic Grid environments.     

Resource allocation robustness in multi-core embedded systems with inaccurate information

Multi-core technologies are widely used in embedded systems and the resource allocation is vita to guarantee Quality of Service (QoS) requirements for applications on multi-core platforms. For heterogeneous multi-core systems, the statistical characteristics of execution times on different cores play a critical role in the resource allocation, and the differences between the actual execution time and the estimated execution time may significantly affect the performance of resource allocation and cause system to be less robust. In this paper, we present an evaluation method to study the impacts of inaccurate execution time information to the performance of resource allocation. We propose a systematic way to measure the robustness degradation of the system and evaluate how inaccurate probability parameters may affect the performance of resource allocations. Furthermore, we compare the performance of three widely used greedy heuristics when using the inaccurate information with simulations.     

QoS based resource provisioning and scheduling in grids

As Grid computing has emerged as a technology for providing the computational resources to industries and scientific projects, new requirements arise. Nowadays, resource management has become an important research area in the Grid computing environment. To provision the appropriate resource to a corresponding application is a tedious task. So, it is important to check and verify the provisioning of the resource before the application’s execution. In this paper, a resource provisioning framework has been presented that offers a resource provisioning policy, which caters to provisioned resource allocation and resource scheduling. The framework has been formally specified and verified. Formal specification and verification of the framework helps in predicting possible errors before the scheduling process itself, and thus results in efficient resource provisioning and scheduling of Grid resources.     

BULLET: Particle Swarm Optimization Based Scheduling Technique for Provisioned Cloud Resources

Cloud resource scheduling requires mapping of cloud resources to cloud workloads. Scheduling results can be optimized by considering Quality of Service (QoS) parameters as inherent requirements of scheduling. In existing literature, only a few resource scheduling algorithms have considered cost and execution time constraints but efficient scheduling requires better optimization of QoS parameters. The main aim of this research paper is to present an efficient strategy for execution of workloads on cloud resources. A particle swarm optimization based resource scheduling technique has been designed named as BULLET which is used to execute workloads effectively on available resources. Performance of the proposed technique has been evaluated in cloud environment. The experimental results show that the proposed technique efficiently reduces execution cost, time and energy consumption along with other QoS parameters.     

SLA-Driven Clustering of QoS-Aware Application Servers

In this paper, we discuss the design, implementation, and experimental evaluation of a middleware architecture for enabling service level agreement (SLA)-driven clustering of QoS-aware application servers. Our middleware architecture supports application server technologies with dynamic resource management: application servers can dynamically change the amount of clustered resources assigned to hosted applications on-demand so as to meet application-level quality of service (QoS) requirements. These requirements can include timeliness, availability, and high throughput and are specified in SLAs. A prototype of our architecture has been implemented using the open-source J2EE application server JBoss. The evaluation of this prototype shows that our approach makes possible JBoss' resource usage optimization and allows JBoss to effectively meet the QoS requirements of the applications it hosts, i.e., to honor the SLAs of those applications     

Ex-post efficient resource allocation for Self-organizing Cloud

Self-organizing Cloud is a scalable model to provide powerful computability with distributed computers. The resource allocation on it is very challenging since it not only involves various types of divisible resources but needs to cope with social competitions. We propose a novel resource allocation scheme with three features on ex-post efficiency. (1) Ex-post win–win effect: each participant (consumers and suppliers) should always feel satisfied with their ex-post payoffs. (2) Incentive compatibility: we can prove each rational participant gets its optimal payoff iff their resource demands and expected prices are truthfully declared. (3) Ex-post maximized efficiency: more powerful resources should be consumed with higher likelihood, such that the whole system runs quite efficiently with maximized resource utilization. Our simulation shows the approach significantly improves resource suppliers’ incomes by 20% compared to their expectations. Meanwhile, we can guarantee consumers’ payments below their budgets, with no any degradation of task execution performance.     

Adaptability mechanisms for autonomic system implementation with AAOP

This paper presents a new approach to implementing an adaptability loop in Autonomic Computing (AC) systems, which is based on adaptable aspects. The approach utilizes the concept of adaptable aspect‐oriented programming (AAOP) in which a set of AOP aspects is used to run an application in the manner specified by its adaptability strategy. We present a model execution environment based on this concept, enabling the execution of applications with applied adaptability strategies. In the AAOP‐based AC system, the application is instrumented with aspects selected by the system from a set of all available aspects (sensors, effectors, and goal aspects) in such a way that the system can monitor and manage the application. This model can be used to implement systems that are able to monitor an application and its execution environment, and perform actions such as changing the current set of non‐functional constraints in response to changes in the application or its environment. The model can be used for various types of non‐functional goals, in various programming languages, both in centralized and distributed environments. This paper describes its Java‐based implementation and non‐functional goals referring to resource management. As a consequence, the application uses resources in a way specified in its adaptability strategy. Resource consumption management logic is transparent for the application, meaning that no modifications in the application source code are needed. Copyright © 2010 John Wiley & Sons, Ltd.     

Agent based sensors resource allocation in sensor grid

Sensor enabled grid may combine real time data about physical environment with vast computational resources derived from the grid architecture. One of the major challenges of designing a sensor enabled grid is how to efficiently schedule sensor resource to user jobs across the collection of sensor resources. The paper presents an agent based scheme for assigning sensor resources to appropriate sensor grid users on the basis of negotiation results among agents. The proposed model consists of two types of agents: the sensor resource agents that represent the economic interests of the underlying sensor resource providers of the sensor grid and the sensor user agents that represent the interests of grid user application using the grid to achieve goals. Interactions between the two agent types are mediated by means of market mechanisms. We model sensor allocation problems by introducing the sensor utility function. The goal is to find a sensor resource allocation that maximizes the total profit. This paper proposes a distributed optimal sensor resource allocation algorithm. The performance evaluation of proposed algorithm is evaluated and compared with other resource allocation algorithms for sensor grid. The paper also gives the application example of proposed approach.