Incentive-Based Scheduling for Market-Like Computational Grids

A sustainable, market-like computational grid has two characteristics: it must allow resource providers and resource consumers to make autonomous scheduling decisions; and both parties of providers and consumers must have sufficient incentives to stay and play in the market. In this paper, we formulate this intuition of optimizing incentives for both parties as a dual-objective scheduling problem. The two objectives identified are to maximize the success rate of job execution, and to minimize fairness deviation among resources. The challenge is to develop a grid scheduling scheme that enables individual participants to make autonomous decisions while produces a desirable emergent property in the grid system, namely, the two objectives are achieved simultaneously. We present an incentive-based scheduling scheme which utilizes a peer-to-peer decentralized scheduling framework, a set of local heuristic algorithms, and three market instruments of job announcement, price, competition degree. The performance of this scheme is evaluated via extensive simulation using synthetic and real workloads. The results show that our approach outperforms other scheduling schemes in optimizing incentives for both consumers and providers, leading to highly successful job execution and fair profit allocation.     

Multi-QoS constrained and Profit-aware scheduling approach for concurrent workflows on heterogeneous systems

The execution of a workflow application can result in an imbalanced workload among allocated processors, ultimately resulting in a waste of resources and a higher cost to the user. Here, we consider a dynamic resource management system in which processors are reserved not for a job but only to run a task, thus allowing a higher resource usage rate. This paper presents a scheduling algorithm that manages concurrent workflows in a dynamic environment in which jobs are submitted by users at any moment in time, on shared heterogeneous resources, and constrained to a specified budget and deadline for each job. Recent research attempted to propose dynamic strategies for concurrent workflows but only addressed fairness in resource sharing among applications while minimizing the execution time. The Multi-QoS Profit-Aware scheduling algorithm (MQ-PAS) proposed here is able to increase the profit achieved by the provider by considering the budget available for each job to define tasks priorities. We study the scalability of the algorithm with different types of workflows and infrastructures. The experimental results show that our strategy improves provider revenue significantly and obtains comparable successful rates of completed jobs.     

云环境中基于分解的多目标工作流调度算法

云服务提供商在给用户提供海量虚拟资源的同时,也面临着一个现实的问题,即怎样调度这些资源,以最小的代价(完工时间、执行费用、资源利用率等)完成工作流的执行。针对IaaS环境下的工作流调度问题,以完工时间和执行费用作为目标,提出了一种基于分解的多目标工作流调度算法。该算法结合了基于列表的启发式算法和多目标进化算法的选择过程,采用一种分解方法,将多目标优化问题分解为一组单目标优化子问题,然后同时求解这些单目标子问题,使得调度过程更为简单有效。算法利用天马项目发布的现实世界中的工作流进行实验,结果表明,和MOHEFT算法以及NSGA-II*算法相比较,所提出的算法能得到更优的Pareto解集,同时具有更低的时间复杂度。     

Resource scheduling with conflicting objectives in grid environments: Model and evaluation

The paper is to consider resource scheduling with conflicting objectives in the grid environment. The objectives of the grid users, the grid resources and the grid system clash with each other. Grid users want to access enough system resources to achieve the desired level of quality of service (QoS). Resource providers pay more attention to the performance of their resources. Our resource scheduling employs market strategies to determine which jobs are executed at what time on which resources and at what prices. A grid resource provider uses its utility function to maximize its profit and a grid user uses its utility function to complete tasks while minimizing its spending. The paper proposes grid system objective optimization scheduling that provides a joint optimization of objectives for both the resource provider and grid user, which combines the benefits of both resource provider objective optimization and user objective optimization. Experiments are designed to study the performances of three resource-scheduling optimization algorithms. Performance metrics are classified into efficiency metrics, utility metrics and time metrics.     

A distributed multiple dimensional QoS constrained resource scheduling optimization policy in computational grid

This paper is to solve efficient QoS based resource scheduling in computational grid. It defines a set of QoS dimensions with utility function for each dimensions, uses a market model for distributed optimization to maximize the global utility. The user specifies its requirement by a utility function. A utility function can be specified for each QoS dimension. In the grid, grid task agent acted as consumer pay for the grid resource and resource providers get profits from task agents. The task agent' utility can then be defined as a weighted sum of single-dimensional QoS utility function. QoS based grid resource scheduling optimization is decomposed to two subproblems: joint optimization of resource user and resource provider in grid market. An iterative multiple QoS scheduling algorithm that is used to perform optimal multiple QoS based resource scheduling. The grid users propose payment for the resource providers, while the resource providers set a price for each resource. The experiments show that optimal QoS based resource scheduling involves less overhead and leads to more efficient resource allocation than no optimal resource allocation.     

自适应邻域的多目标网格任务调度算法研究

针对网格计算中的多目标网格任务调度问题,提出了一种基于自适应邻域的多目标网格任务调度算法。该算法通过求解多个网格任务调度目标函数的非劣解集,采用自适应邻域的方法来保持网格任务调度多目标解集的分布性,尝试解决网格任务调度中多目标协同优化问题。实验结果证明,该算法能够有效地平衡时间维度和费用维度目标,提高了资源的利用率和任务的执行效率,与Min-min和Max-min算法相比具有较好的性能。     

Job scheduling and processor allocation for grid computing on metacomputers

Scheduling is a fundamental issue in achieving high performance on metacomputers and computational grids. For the first time, the job scheduling problem for grid computing on metacomputers is studied as a combinatorial optimization problem. A cost model is proposed for modeling communication heterogeneity on computational grids. A processor allocation algorithm is developed which always finds an optimal processor allocation that minimizes the effective execution time of a job when the job is being scheduled. It is proven that the list scheduling (LS) algorithm can achieve reasonable worst-case performance bound in grid environments supporting distributed supercomputing with large applications. We compare the performance of various job scheduling and processor allocation algorithms for grid computing on metacomputers. We evaluate the performance of 128 combinations of two job scheduling algorithms, four initial job ordering strategies, four processor allocation algorithms, and four metacomputers by extensive simulation. It is found that the combination of largest job first (LJF) initial job ordering and minimum effective execution time (MEET) or largest machine first (LMF) processor allocation algorithm yields the best average-case performance, and the choice of FCFS and LS depends on the range of job sizes. It is also observed that communication heterogeneity does have significant impact on schedule lengths.     

An efficient resource allocation for maximizing benefit of users and resource providers in ad hoc grid environment

Ad hoc grids allow a group of individuals to accomplish a mission that involves computation and communication among the grid components, often without fixed structure. In an ad hoc grid, every node in the network can spontaneously arise as a resource consumer or a resource producer at any time when it needs a resource or it possesses an idle resource. At the same time, the node in ad hoc grid is often energy constrained. The paper proposes an efficient resource allocation scheme for grid computing marketplace where ad hoc grid users can buy usage of memory and CPU from grid resource providers. The ad hoc grid user agents purpose to obtain the optimized quality of service to accomplish their tasks on time with a given budget, and the goal of grid resource providers as profit-maximization. Combining perspectives of both ad hoc grid users and resource providers, the paper present ad hoc grid resource allocation algorithm to maximize the global utility of the ad hoc grid system which are beneficial for both grid users and grid resource providers. Simulations are conducted to compare the performance of the algorithms with related work.     

A system-centric scheduling policy for optimizing objectives of application and resource in grid computing

In grid computing, grid users who submit applications and resources providers who provide resources have different motivations when they join the grid. Application-centric scheduling aims to optimize the performance of individual application. Resource-centric scheduling aims to optimize the resource utilization of resources provider. Due to autonomy both in grid users and resource providers, the objectives of application-centric and resource-centric scheduling often conflict. The paper proposes a system-centric scheduling that provides a solution of joint optimization of the objectives for both the grid resource and grid application. Utility functions are used to express the objectives of grid resource and application. The system-centric scheduling policy can be formulated as joint optimization of utilities of grid applications and grid resources, which combine both application centric and resource-centric scheduling benefits. Simulations are conducted to study the performance of the system-centric scheduling algorithm. The experiment results show that the system-centric scheduling algorithm yields significantly better performance than application-centric scheduling algorithm and resource-centric scheduling algorithm.     

Market-oriented Grids and Utility Computing: The State-of-the-art and Future Directions

Traditional resource management techniques (resource allocation, admission control and scheduling) have been found to be inadequate for many shared Grid and distributed systems, that consist of autonomous and dynamic distributed resources contributed by multiple organisations. They provide no incentive for users to request resources judiciously and appropriately, and do not accurately capture the true value, importance and deadline (the utility) of a user’s job. Furthermore, they provide no compensation for resource providers to contribute their computing resources to shared Grids, as traditional approaches have a user-centric focus on maximising throughput and minimising waiting time rather than maximising a providers own benefit. Consequently, researchers and practitioners have been examining the appropriateness of ‘market-inspired’ resource management techniques to address these limitations. Such techniques aim to smooth out access patterns and reduce the chance of transient overload, by providing a framework for users to be truthful about their resource requirements and job deadlines, and offering incentives for service providers to prioritise urgent, high utility jobs over low utility jobs. We examine the recent innovations in these systems (from 2000–2007), looking at the state-of-the-art in price setting and negotiation, Grid economy management and utility-driven scheduling and resource allocation, and identify the advantages and limitations of these systems. We then look to the future of these systems, examining the emerging ‘Catallaxy’ market paradigm. Finally we consider the future directions that need to be pursued to address the limitations of the current generation of market oriented Grids and Utility Computing systems.     

一种快速网格任务调度策略

网格任务调度目标有很多,如用户要求任务轮转时间短、花费代价小,而资源提供者希望资源利用率高等,这些目标相互冲突,因此网格任务调度不仅是一个NP难问题,而且是一个多目标优化问题.本文根据网格环境下任务的时间相关性特点,对传统蚁群算法进行了改进,提出了一种快速网格任务调度算法.该算法不仅解决了网格调度中多目标优化问题,而且依据任务调度历史信息生成蚁群算法的初始信息素分布,提高了蚁群算法的求解速度.     

Task scheduling using NSGA II with fuzzy adaptive operators for computational grids

Scheduling algorithms have an essential role in computational grids for managing jobs, and assigning them to appropriate resources. An efficient task scheduling algorithm can achieve minimum execution time and maximum resource utilization by providing the load balance between resources in the grid. The superiority of genetic algorithm in the scheduling of tasks has been proven in the literature. In this paper, we improve the famous multi-objective genetic algorithm known as NSGA-II using fuzzy operators to improve quality and performance of task scheduling in the market-based grid environment. Load balancing, Makespan and Price are three important objectives for multi-objective optimization in the task scheduling problem in the grid. Grid users do not attend load balancing in making decision, so it is desirable that all solutions have good load balancing. Thus to decrease computation and ease decision making through the users, we should consider and improve the load balancing problem in the task scheduling indirectly using the fuzzy system without implementing the third objective function. We have used fuzzy operators for this purpose and more quality and variety in Pareto-optimal solutions. Three functions are defined to generate inputs for fuzzy systems. Variance of costs, variance of frequency of involved resources in scheduling and variance of genes values are used to determine probabilities of crossover and mutation intelligently. Variance of frequency of involved resources with cooperation of Makespan objective satisfies load balancing objective indirectly. Variance of genes values and variance of costs are used in the mutation fuzzy system to improve diversity and quality of Pareto optimal front. Our method conducts the algorithm towards best and most appropriate solutions with load balancing in less iteration. The obtained results have proved that our innovative algorithm converges to Pareto-optimal solutions faster and with more quality.     

An agent-based approach for dynamic adjustment of scheduled jobs in computational grids

Grid computing is a newly developed technology for complex systems with large-scale resource sharing, wide-area communication, and multi-institutional collaboration. Grid scheduling is an important infrastructure in the grid computing environment. Most of the existing grids scheduling methods focus on maximizing processor utilization without taking grid load into consideration. This may lead to significant inefficiencies in performance such as large job queues and processing delays. In this paper, we propose a multiagent-based scheduling system for computational grids with a new approach. Agent technology is suitable for a computational grid because of the dynamic, heterogeneous, and autonomous nature of the grid. The main idea of the proposed system is a combination of a static scheduling using a fixed scheduling algorithm and a dynamic adjustment through the autonomous behavior of agents. The superiority of the proposed system, in reducing the load of the grid and minimizing the response time for executing user applications, is demonstrated by simulation experiments.     

Cooperative power-aware scheduling in grid computing environments

Energy usage and its associated costs have taken on a new level of significance in recent years. Globally, energy costs that include the cooling of server rooms are now comparable to hardware costs, and these costs are on the increase with the rising cost of energy. As a result, there are efforts worldwide to design more efficient scheduling algorithms. Such scheduling algorithm for grids is further complicated by the fact that the different sites in a grid system are likely to have different ownerships. As such, it is not enough to simply minimize the total energy usage in the grid; instead one needs to simultaneously minimize energy usage between all the different providers in the grid. Apart from the multitude of ownerships of the different sites, a grid differs from traditional high performance computing systems in the heterogeneity of the computing nodes as well as the communication links that connect the different nodes together. In this paper, we propose a cooperative, power-aware game theoretic solution to the job scheduling problem in grids. We discuss our cooperative game model and present the structure of the Nash Bargaining Solution. Our proposed scheduling scheme maintains a specified Quality of Service (QoS) level and minimizes energy usage between all the providers simultaneously; energy usage is kept at a level that is sufficient to maintain the desired QoS level. Further, the proposed algorithm is fair to all users, and has robust performance against inaccuracies in performance prediction information.     

基于收益率门槛限制考虑的网格资源拍卖问题

基于收益率门槛限制的视角,通过建立效用函数模型并结合动态博弈理论,对网格资源的拍卖问题进行了探讨.在对网格资源提供者与竞标网格资源使用者的动态博弈过程进行分析时发现,网格资源提供者的最优策略选择决定于其对货币收益与非货币收益的偏好程度,以及网格资源使用者的最高报价.在收益率门槛给定的条件下,当参与竞标的网格资源使用者具有较低的生产利润或付出较高的努力成本时,网格资源使用者将会选择价格较低的投标策略.研究结果表明收益率门槛机制的引入,在一定程度上可以使得参与双方的效用达到最大化.     

异构云环境下AHP定权的多目标强化学习作业调度方法

随着新型基础设施建设(新基建)的加速,云计算将获得新的发展契机.数据中心作为云计算的基础设施,其内部服务器不断升级换代,这造成计算资源的异构化.如何在异构云环境下,对作业进行高效调度是当前的研究热点之一.针对异构云环境多目标优化调度问题,设计一种AHP定权的多目标强化学习作业调度方法.首先定义执行时间、平台运行能耗、成...     

Fairness in dataflow scheduling in the cloud

Expensive dataflow queries which may involve large-scale computations operating on significant volumes of data are typically executed on distributed platforms to improve application performance. Among these, cloud computing has emerged as an attractive option for users to execute dataflows allowing them to select proper configurations (e.g., number of machines) to achieve desired trade-offs between execution time and monetary cost. Discovering dataflow schedules that exhibit the best trade-offs within a plethora of potential solutions can be challenging, especially in a heterogeneous environment where resource characteristics like performance and price can be varied. To increase resource utilization, users may also submit multiple dataflows for execution concurrently. Traditionally, building fair schedules (schedules where the slowdown of all dataflows due to resource sharing is similar) while achieving good performance is a major concern. However, considering fairness in the cloud computing setting where monetary cost is part of the optimization objectives significantly increases the difficulty of the scheduling problem. This paper proposes an algorithm for the scheduling of multiple dataflows on heterogeneous clouds that identifies Pareto-optimal solutions (schedules) in the three-dimensional space formed from the different trade-offs between overall execution time, monetary cost and fairness. The results show that in most cases the proposed approach can provide solutions with fairer schedules without significantly impacting the quality of the execution time to monetary cost skyline compared to the state of the art where the fairness of a solution is not taken into account.     

Real-time workflows oriented online scheduling in uncertain cloud environment

Workflow scheduling has become one of the hottest topics in cloud environments, and efficient scheduling approaches show promising ways to maximize the profit of cloud providers via minimizing their cost, while guaranteeing the QoS for users’ applications. However, existing scheduling approaches are inadequate for dynamic workflows with uncertain task execution times running in cloud environments, because those approaches assume that cloud computing environments are deterministic and pre-computed schedule decisions will be statically followed during schedule execution. To cover the above issue, we introduce an uncertainty-aware scheduling architecture to mitigate the impact of uncertain factors on the workflow scheduling quality. Based on this architecture, we present a scheduling algorithm, incorporating both event-driven and periodic rolling strategies (EDPRS), for scheduling dynamic workflows. Lastly, we conduct extensive experiments to compare EDPRS with two typical baseline algorithms using real-world workflow traces. The experimental results show that EDPRS performs better than those algorithms.     

基于模糊多目标决策的网格资源调度算法研究

针对如何提高网格资源的使用效率和用户满意度及系统效率等问题,提出了一个基于层次调度模型的、将资源的表示与需求用XML方式描述、以模糊多目标决策理论为资源调度策略,以用户满意度和系统资源利用率为主要目标的综合网格资源调度算法.该算法不仅最大程度提高用户的满意度,而且较好地解决了网格资源的均衡使用,极大地提高了系统效率,对网格系统综合性能有明显地提高.     

基于改进多目标布谷鸟搜索的资源调度算法

针对IaaS(Infrastructure as a Service)云计算中资源调度的多目标优化问题,提出一种基于改进多目标布谷鸟搜索的资源调度算法。在多目标布谷鸟搜索算法的基础上,通过改进随机游走策略和丢弃概率策略提高了算法的局部搜索能力和收敛速度。以最大限度地减少完成时间和成本为主要目标,将任务分配特定的VM(Virtual Manufacturing)满足云用户对云提供商的资源利用的需求,从而减少延迟,提高资源利用率和服务质量。实验结果表明,该算法可以有效地解决IaaS云计算环境中资源调度的多目标问题,与其他算法相比,具有一定的优势。