Modeling and analysis of the effects of QoS and reliability on pricing, profitability, and risk management in multiperiod grid-computing networks

In this paper we develop a network equilibrium model for optimal pricing and resource allocation in Computational Grid Network. We consider a general network economy model with Grid Resource Providers, Grid Resource Brokers and Grid Users. The proposed framework allows for the modeling and theoretical analysis of Computational Grid Markets that considers a non-cooperative behavior of decision-makers in the same tier of the grid computing network (such as, for example, Grid Resource Providers) as well as cooperative behavior between tiers (between Resource Providers and Grid Brokers). We introduce risk management into the decision making process by analyzing the decision-marker's reliability and quality of service (QoS) requirement. We analyze resource allocation patterns as well as equilibrium price based on demand, supply, and cost structure of the grid computing market network. We specifically answer the following questions with several numerical examples: How do system reliability levels affect the QoS levels of the service providers and brokers under competition? How do system reliability levels affect the profits of resource providers and brokers in a competitive market? How do system reliability levels influence the pricing of the services in a competitive environment? How do users' service request types, QoS requirements, and timing concerns affect users' behaviors, costs and risks in equilibrium? How does the market mechanism allocate resources to satisfy the demands of users? We find that for users who request same services certain timing flexibility can not only reduce the costs but also lower the risks. The results indicated that the value of QoS can be efficiently priced based on the heterogeneous service demands.     

From volunteer to trustable computing: Providing QoS-aware scheduling mechanisms for multi-grid computing environments

The exploitation of service oriented technologies, such as Grid computing, is being boosted by the current service oriented economy trend, leading to a growing need of Quality of Service (QoS) mechanisms. However, Grid computing was created to provide vast amounts of computational power but in a best effort way. Providing QoS guarantees is therefore a very difficult and complex task due to the distributed and heterogeneous nature of their resources, specially the volunteer computing resources (e.g., desktop resources).The scope of this paper is to empower an integrated multi QoS support suitable for Grid Computing environments made of either dedicated and volunteer resources, even taking advantage of that fact. The QoS is provided through SLAs by exploiting different available scheduling mechanisms in a coordinated way, and applying appropriate resource usage optimization techniques. It is based on the differentiated use of reservations and scheduling in advance techniques, enhanced with the integration of rescheduling techniques that improve the allocation decisions already made, achieving a higher resource utilization and still ensuring the agreed QoS. As a result, our proposal enhances best-effort Grid environments by providing QoS aware scheduling capabilities.This proposal has been validated by means of a set of experiments performed in a real Grid testbed. Results show how the proposed framework effectively harnesses the specific capabilities of the underlying resources to provide every user with the desired QoS level, while, at the same time, optimizing the resources’ usage.     

QoS in grid computing

Grid computing is already a mainstream paradigm for resource-intensive scientific applications, but it also promises to become the future model for enterprise applications. The grid enables resource sharing and dynamic allocation of computational resources, thus increasing access to distributed data, promoting operational flexibility and collaboration, and allowing service providers to scale efficiently to meet variable demands. Large-scale grids are complex systems composed of thousands of components from disjoined domains. Planning the capacity to guarantee quality of service (QoS) in such environments is a challenge because global service-level agreements (SLAs) depend on local SLAs. We provide a motivating example for grid computing in an enterprise environment and then discuss the how resource allocation affects SLAs.     

A QoS-driven approach for service-oriented device anycasting in ubiquitous environments

The vision of pervasive computing is to let users enjoy ICT-enabled services in an “any time, anywhere, on any device” manner. It aims to supply ubiquitous services through communications among a set of devices deployed in a ubiquitous environment. Due to the diverse QoS needs of different kinds of ubiquitous services and users, it is a critical challenge to select an optimal set of devices with the objective of achieving service-specific QoS goals like low packet loss, short packet delay, and high energy efficiency. In this paper, the problem is first formulated as the service-oriented device anycasting problem (SDAP) and then proved as an NP-complete problem. By adopting a tree-based service representation model, Basu et al. proposed the dynamic task-embedding anycasting (DTA) approach. This approach effectively solves the SDAP in a distributed way. However, the service quality is likely sacrificed because the tree scheme does not sufficiently describe a ubiquitous service. In this paper, we propose a novel approach called the service-oriented device anycasting (SDA) approach that adopts a graph-based service representation model called the service profile (SP). By introducing a layered structure into the SP, the SDA approach can reach a compromise between service quality and computational complexity. In addition, the QoS-driven utility function is proposed to quantify service quality by matching the capabilities of heterogeneous devices to various QoS needs. Finally, the simulation results show that the SDA approach outperforms the DTA approach by saving roughly 20% of device energy and prolonging the network lifetime. Packet loss and packet delay are also improved by roughly 25% and 8%, respectively. The advantage of the SDA approach is more obvious in environments with highly mobile devices and multiple users.     

Online algorithms for advance resource reservations

We consider the problem of providing QoS guarantees to Grid users through advance reservation of resources. Advance reservation mechanisms provide the ability to allocate resources to users based on agreed-upon QoS requirements and increase the predictability of a Grid system, yet incorporating such mechanisms into current Grid environments has proven to be a challenging task due to the resulting resource fragmentation. We use concepts from computational geometry to present a framework for tackling the resource fragmentation, and for formulating a suite of scheduling strategies. We also develop efficient implementations of the scheduling algorithms that scale to large Grids. We conduct a comprehensive performance evaluation study using simulation, and we present numerical results to demonstrate that our strategies perform well across several metrics that reflect both user- and system-specific goals. Our main contribution is a timely, practical, and efficient solution to the problem of scheduling resources in emerging on-demand computing environments.     

QoS-Aware Admission Control and Dynamic Resource Provisioning Framework in Ubiquitous Multimedia Computing Environments

A ubiquitous service deployment is emerging in the multimedia, networking, and wireless mobile computing area. Therefore, there has been an increasing demand for ubiquitous computing environments to support a certain degree of quality of service (QoS) to meet various service requirements from different computing and networking applications, and to better utilize the computing resources. However, supporting QoS in the ubiquitous computing environments has also raised great concerns regarding the applicability of any QoS solution. Management of such ubiquitous multimedia applications requires new mechanisms, i.e., Soft-QoS framework, to be developed for admission control, negotiation, allocation, and scheduling. In this paper, we present a novel negotiated admission control algorithm that exploits the degradability property of applications to improve the performance of the system. The algorithm is based on setting aside a portion of the resources as reserves and managing it intelligently, so that the total utility of the system can be maximized. The mixed greedy and predictive strategy leads to an efficient protocol that also improves the system performance. We use the constructs of application benefit functions and resource demand functions in the integrated admission control and negotiation protocol. We applied our Soft-QoS framework to the admission controlling and resource scheduling for ubiquitous multimedia devices such as Continuous Media (CM) or Video-On-Demand (VOD) servers, where multimedia applications can generally tolerate certain variations on QoS parameters by providing multiple classes with consistently proportional rather than absolute QoS. Extensive simulation experiments are presented in the paper to evaluate the performance of the novel mechanisms and compare it against some other methods used in the past.This work was in part funded by DARPA through the SPAWARSYSCEN under Contract Number N66001-97-C-8525 and SK Telecom, Korea, under Contract Number (KU-R0405721).     

网格环境下费用约束的科学工作流可靠调度算法

网格基础设施是目前科学工作流应用规划、部署和执行的主要支撑环境.然而由于网格资源的自治、动态及异构性,如何在保障用户QoS约束下有效调度科学工作流是一个研究热点.针对费用约束下的科学工作流调度问题,为了提高其执行的可靠性,本文使用随机服务模型描述资源节点的动态服务能力并考虑本地任务负载对资源执行性能的影响,给出一种资源可靠性的评估方法,在此基础上提出一种费用约束下的科学工作流可靠调度算法RSASW.仿真实验结果表明RSASW算法相对于GAIN3,GreedyTime-CD及PFAS算法,对工作流的执行具有很好的可靠性保障.     

网格工作流中基于商品市场的服务选择

服务选择是网格工作流的重要环节。针对现有Web服务组合方法对服务质量（QoS）贪婪索取而不利于网格资源有效利用的问题,提出基于商品市场的"按需"网格服务选择方法：根据面向工作流全局的评价模型,以服务使用者需求为约束条件,将服务选择建模为0-1多维背包问题;并引入经济学中的"价格-供需关系"改进了服务选择模型。仿真实验表明：基于商品市场的服务选择均衡了系统负载,有利于网格资源的高效利用。     

QoS guided Min-Min heuristic for grid task scheduling

Task scheduling is an integrated component of computing.With the emergence of Grid and ubiquitous computing,new challenges appear in task scheduling based on properties such as security,quality of service,and lack of central control within distributed administrative domains.A Grid task scheduling framework must be able to deal with these issues.One of the goals of Grid task scheduling is to achivev high system throughput while matching applications with the available computing resources.This matching of resources in a non-deterministically shared heterogeneous environment leads to concerns over Quality of Service (QoS).In this paper a novel QoS guided task scheduling algorithm for Grid computing is introduced.The proposed novel algorithm is based on a general adaptive scheduling heuristics that includes QoS guidance.The algorithm is evaluated within a simulated Grid environment.The experimental results show that the nwe QoS guided Min-Min heuristic can lead to significant performance gain for a variety of applications.The approach is compared with others based on the quality of the prediction formulated by inaccurate information.     

An efficient job management of computing service using integrated idle VM resources for high-performance computing based on OpenStack

In recent years, various studies on OpenStack-based high-performance computing have been conducted. OpenStack combines off-the-shelf physical computing devices and creates a resource pool of logical computing. The configuration of the logical computing resource pool provides computing infrastructure according to the user’s request and can be applied to the infrastructure as a service (laaS), which is a cloud computing service model. The OpenStack-based cloud computing can provide various computing services for users using a virtual machine (VM). However, intensive computing service requests from a large number of users during large-scale computing jobs may delay the job execution. Moreover, idle VM resources may occur and computing resources are wasted if users do not employ the cloud computing resources. To resolve the computing job delay and waste of computing resources, a variety of studies are required including computing task allocation, job scheduling, utilization of idle VM resource, and improvements in overall job’s execution speed according to the increase in computing service requests. Thus, this paper proposes an efficient job management of computing service (EJM-CS) by which idle VM resources are utilized in OpenStack and user’s computing services are processed in a distributed manner. EJM-CS logically integrates idle VM resources, which have different performances, for computing services. EJM-CS improves resource wastes by utilizing idle VM resources. EJM-CS takes multiple computing services rather than single computing service into consideration. EJM-CS determines the job execution order considering workloads and waiting time according to job priority of computing service requester and computing service type, thereby providing improved performance of overall job execution when computing service requests increase.

     

Service selection and workflow mapping for Grids: an approach exploiting quality‐of‐service information

The advent of heterogeneous and distributed environments, such as Grid environments, made feasible the solution to computational‐intensive problems in a reliable and cost‐effective manner. In parallel, workflows with increased complexity that require specialized systems to deal with them are emerging, so as to carry out more composite and mission‐critical applications. In that rationale, quality‐of‐service (QoS) issues need to be tackled in order to ensure that each application satisfies the corresponding user requirements. Therefore, considering the quality provision aspect as fundamental for enabling Grid applications to become QoS compliant, we present an approach for service selection using QoS criteria. The latter is achieved with a suite of components that allow the different mappings of application workflow processes to Grid services that not only meet the user goals and requirements but also maximize his/her benefit in terms of the offered QoS level. We also demonstrate the operation of the aforementioned suite of components and evaluate its performance and effectiveness using a Grid scenario, based on a 3D image rendering application. Copyright © 2008 John Wiley & Sons, Ltd.     

Service level agreement based adaptive Grid superscheduling

Grid computing brings heterogeneity and decentralization to the world of science and technology. It leverages every bit of idle computing resources and provides a straightforward middleware for integrating cross-domain scientific devices and legacy systems. In a super big Grid, job scheduling is challenging specifically when it needs to have access to vast amount of resources. The process of mapping jobs onto Grid resources requires significant consideration in terms of Grid architecture design, consumer demands and provider revenues. In this paper, we simultaneously utilize the legacy architecture of superscheduling, forwarding strategy, service level, success rate, and service pricing strategies and finally propose a service level agreement based on adaptive superscheduling (SAS) algorithm. SAS algorithm presents unified connectivity via efficient diffusion of jobs through the Grid infrastructure that is fueled from the previous scheduling events across the Grid. Moreover, by enforcing the service level agreement terms from a rich set of ask and bid prices, system performance, and load statistics, SAS successfully boosts revenue and utilization statistics. We perform an extensive experimental analysis for different Grid scales. Based on our experimental result, the SAS algorithm maximizes revenue while guarantees quality of service. More specifically, the quality of service is achieved through a high ratio of completed jobs and remarkable utilization of resources.     

Autonomic QoS control in enterprise Grid environments using online simulation

As Grid Computing increasingly enters the commercial domain, performance and quality of service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. This paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.     

Analysis and Provision of QoS for Distributed Grid Applications

Grid computing provides the infrastructure necessary to access and use distributed resources as part of virtual organizations. When used in this way, Grid computing makes it possible for users to participate in collaborative and distributed applications such as tele-immersion, visualization, and computational simulation. Some of these applications operate in a collaborative mode, requiring data to be stored and delivered in a timely manner. This class of applications must adhere to stringent real-time constraints and Quality-of-Service (QoS) requirements. A QoS management approach is therefore required to orchestrate and guarantee the timely interaction between such applications and services. We discuss the design and a prototype implementation of a QoS system, and demonstrate how we enable Grid applications to become QoS compliant. We validate this approach through a case study of an image processing task derived from a nanoscale structures application.     

A resource management and fault tolerance services in grid computing

In grid computing, resource management and fault tolerance services are important issues. The availability of the selected resources for job execution is a primary factor that determines the computing performance. In this paper, we propose a resource manager for optimal resource selection. Our resource manager automatically selects the set of optimal resources among candidate resources that achieves optimal performance using a genetic algorithm. Typically, the probability of a failure is higher in the grid computing than in a traditional parallel computing and the failure of resources affects job execution fatally. Therefore, a fault tolerance service is essential in computational grids. And grid services are often expected to meet some minimum levels of Quality of Service (QoS) for a desirable operation. To address this issue, we also propose a fault tolerance service that satisfies QoS requirements. We extend the definition of failures from the conventional notion of failures in distribute systems in order to provide a fault tolerance service that deals with various types of resource failures, which include process failures, processor failures, and network failures. We also design and implement a fault detector and a fault manager. The implementation and simulation results indicate that our approaches are promising in that (1) the resource manager finds the optimal set of resources that guarantees efficient job execution, (2) the fault detector detects the occurrence of resource failures and (3) the fault manager guarantees that the submitted jobs complete and the performance of job execution is improved due to job migration even if some failures occur.     

Achieving efficiency, quality of service and robustness in multi-organizational Grids

Scalability, flexibility, quality of service provisioning, efficiency and robustness are the desired characteristics of most computing systems. Although the emerging Grid computing paradigm is scalable and flexible, achieving both efficiency and quality of service provisioning in Grids is a challenging task but is necessary for the wide adoption of Grids. Grid middleware should also be robust to uncertainties such as those in user-estimated runtimes of Grid applications. In this paper, we present a complete middleware framework for Grids that achieves user satisfaction by providing QoS guarantees for Grid applications, cost effectiveness by efficiently utilizing resources and robustness by intelligently handling uncertain runtimes of applications.     

Flexible Grid service management through resource partitioning

In this paper, a distributed and scalable Grid service management architecture is presented. The proposed architecture is capable of monitoring task submission behaviour and deriving Grid service class characteristics, for use in performing automated computational, storage and network resource-to-service partitioning. This partitioning of Grid resources amongst service classes (each service class is assigned exclusive usage of a distinct subset of the available Grid resources), along with the dynamic deployment of Grid management components dedicated and tuned to the requirements of a particular service class introduces the concept of Virtual Private Grids. We present two distinct algorithmic approaches for the resource partitioning problem, the first based on Divisible Load Theory (DLT) and the second built on Genetic Algorithms (GA). The advantages and drawbacks of each approach are discussed and their performance is evaluated on a sample Grid topology using NSGrid, an ns-2 based Grid simulator. Results show that the use of this Service Management Architecture in combination with the proposed algorithms improves computational and network resource efficiency, simplifies schedule making decisions, reduces the overall complexity of managing the Grid system, and at the same time improves Grid QoS support (with regard to job response times) by automatically assigning Grid resources to the different service classes prior to scheduling.     

A computational economy for grid computing and its implementation in the Nimrod-G resource broker

Computational grids that couple geographically distributed resources such as PCs, workstations, clusters, and scientific instruments, have emerged as a next generation computing platform for solving large-scale problems in science, engineering, and commerce. However, application development, resource management, and scheduling in these environments continue to be a complex undertaking. In this article, we discuss our efforts in developing a resource management system for scheduling computations on resources distributed across the world with varying quality of service (QoS). Our service-oriented grid computing system called Nimrod-G manages all operations associated with remote execution including resource discovery, trading, scheduling based on economic principles and a user-defined QoS requirement. The Nimrod-G resource broker is implemented by leveraging existing technologies such as Globus, and provides new services that are essential for constructing industrial-strength grids. We present the results of experiments using the Nimrod-G resource broker for scheduling parametric computations on the World Wide Grid (WWG) resources that span five continents.     

Trustworthy remote compiling services for grid-based scientific applications

Grid computing, which is characterized by large-scale sharing and collaboration of dynamic resources, is becoming an emerging computing platform on a global scale for data-intensive and computation-intensive scientific application. However, the complications of large-scale scientific computations and simulations harnessing massive computing resources are compounded by extensive heterogeneity in environments arising from “the Grid.” Scientists and engineers lack an intuitive grid-based compilation tool, which has contributed to the difficulty of exploiting these diverse resources and developing their applications on the grid. While manual configuration of various toolkits simplifying the end-to-end completion of a job is adequate for a computational grid with a limited number of nodes, the compilation procedure becomes inefficient for a computational grid with an increasing number of heterogeneous computational service providers. On the other hand, a global-scale computational grid is a potentially untrustworthy computing environment. How to take advantage of the potentially untrustworthy grid resources to provide trustworthy computational services for large-scale scientific applications is another critical issue. In this article, a remote compiling service for a heterogeneous computational grid is developed. In addition to running compilation tasks, the remote compiling service provides security enforcement and validation facilities, including intermediate value checking, secure source program submission, restricted compilation, and binary inspection, to support trustworthy compilation and execution of grid-based scientific applications. Overall, it is expected that our remote compiling services on the grid can tackle the heterogeneity problem of the grid and provide a secure, trustworthy, reliable, and state-of-the-art mechanism to develop grid-aware scientific applications.

On the Improvement of Grid Resource Utilization: Preventive and Reactive Rescheduling Approaches

One of the key motivations of computational and data Grids is the ability to make coordinated use of heterogeneous computing resources which are geographically dispersed. However, the provision of Quality of Service (QoS) to Grid users is still a challenge that needs the attention of the research community. Reservation of resources in advance has been proposed as a way of providing QoS guarantees but they may not always be possible. For this reason, this work focuses on meta-scheduling of jobs in advance as a way of enhancing the provision of QoS. Thereby, jobs are scheduled some time before they are actually executed, but no resource is physically reserved. One of the drawbacks of this scenario is that fragmentation may appear in resources (free time slots but not large enough to execute a job) which leads to poor resource utilization. For that reason, two techniques have been developed to tackle poor resource utilization, whose main idea consists of rescheduling already scheduled jobs so that a new incoming job can be allocated. These rescheduling techniques have been implemented within a middleware that supports meta-scheduling in advance, which relies on the GridWay meta-scheduler. Finally, these proposals have been tested using a real testbed involving heterogeneous computing resources distributed across different national organizations, with different experiments showing their efficiency.