混合云中基于截止时间和费用约束的调度方法研究*

针对混合云调度中私有云利用率不高和公有云费用偏高的问题,基于性能和费用目标提出了两个调度策略—截止时间优先和费用优先策略,建立了混合云中的任务和资源模型,能够根据用户提交的任务需求自适应选择合适的调度资源,对截止时间要求比较高的任务可以优先调度至公有云,对费用要求高的任务可以优先调度至私有云,而且两种策略均满足截止时间和一定的费用约束,因此相对于其它类似的基准调度方法,本文的两种调度策略在调度完成时间、费用、截止时间超出率和私有云利用率等方面均有很好的表现,尤其是当任务量比较大的时候,两种调度策略表现出更好的自适应性和优势。     

混合云环境中数据敏感工作流调度

混合云环境下调度包含敏感数据的工作流主要考虑在满足数据安全性以及工作流截止时间的前提下,对工作流任务在混合云上进行分配,实现计算资源与任务的映射,并优化调度费用。采用了整数规划来建模求解包含数据敏感性、截止时间和调度费用3种约束条件的混合云工作流调度问题,同时为优化模型求解速度,基于“帕雷托最优”原理对工作流任务在混合云上的分配方案进行筛选以减小模型求解规模。实验表明,优先排除不合理的任务分配方案可有效减小整数规划模型的求解规模,缩短模型计算时间,在产生较小误差的情况下获得较优的调度结果。     

Online cost-rejection rate scheduling for resource requests in hybrid clouds

The hybrid cloud computing model has attracted considerable attention in recent years. Due to security and controllability of private cloud, some resource requests are required to be scheduled in private parts of hybrid cloud. However, these requests could be rejected because of the resource limitation of private parts of hybrid cloud. In this paper, all resource requests are classified into two categories, i.e., the special requests required to process in private cloud, and the normal requests insensitive in private or public clouds. By considering the normal requests undivided to dispatch a cloud and divided to dispatch different clouds, we propose the online cost-rejection rate scheduling strategy (OCS) for the normal requests undivided, and the online scheduling strategy for some normal requests divided (OCS-divided), which could make suitable requests placement decisions in real-time and minimize the cost of renting public cloud resources with a low rate of rejected requests. Then, we transform both online models into two one-shot optimization problems by taking advantage of Lyapunov optimization techniques, and employ the optimal decay algorithm to solve the one-shot problems. The simulation results demonstrate that our scheduling strategies can achieve the trade-off between cost and rejection rate, and process the real-time resource requests in hybrid clouds. OCS-divided can achieve an average cost saving of about 25% compared with OCS and maximize the resource utilization.     

Optimal scheduling across public and private clouds in complex hybrid cloud environment

The hybrid cloud extends the private cloud model by using both local and remote resources. The private cloud will rely on the resources leased from public cloud providers for the execution of private cloud applications. The paper presents optimal scheduling across public and private clouds in complex hybrid cloud environment. The contributions of this paper have three aspects. 1) The proposed hybrid cloud scheduling policy considers the benefits of private cloud applications and public cloud provider, it can adapt to the changes in the system to find the scheduling optimization. The scheduling optimization is decomposed and conducted across the private cloud and public cloud. 2) Secondly, The paper describes negotiations in hybrid cloud marketplace and gives an example to explain how these rules are resolved by the cloud marketplace. 3) Thirdly, the paper proposes an optimal scheduling algorithm across public and private clouds. The paper also describes negotiations in hybrid cloud marketplace and gives an example to explain how these rules are resolved by the cloud marketplace. In the simulations, the profit of public cloud provider and resource utilization of the proposed algorithm are better than other related works.     

Online optimization for scheduling preemptable tasks on IaaS cloud systems

In Infrastructure-as-a-Service (IaaS) cloud computing, computational resources are provided to remote users in the form of leases. For a cloud user, he/she can request multiple cloud services simultaneously. In this case, parallel processing in the cloud system can improve the performance. When applying parallel processing in cloud computing, it is necessary to implement a mechanism to allocate resource and schedule the execution order of tasks. Furthermore, a resource optimization mechanism with preemptable task execution can increase the utilization of clouds. In this paper, we propose two online dynamic resource allocation algorithms for the IaaS cloud system with preemptable tasks. Our algorithms adjust the resource allocation dynamically based on the updated information of the actual task executions. And the experimental results show that our algorithms can significantly improve the performance in the situation where resource contention is fierce.     

云平台下满足任务截止时间的资源分配策略

提出一种云平台下满足任务截止时间的资源分配策略。根据云平台的实际情况构造一个2层的资源分配模型,采用改进的银行家算法进行资源分配,在满足任务截止期限的前提下使任务的花费最小。在CloudSim环境下进行仿真实验,结果表明,使用该策略能满足任务截止时间、减少任务费用并提高系统性能。     

基于效益博弈的云计算资源动态可协调分配策略研究

在对用户的任务进行计算资源分配时,为了有效提高计算资源的利用效率,减少任务执行所需要的成本,提出了一种基于效益博弈的云计算资源动态可协调分配机制。该机制采用时间矩阵和费用矩阵作为任务效益的衡量指标,提出效益博弈模型,通过该模型的效益计算方程来得到最好的资源分配策略。为了使得计算资源能够合理地按需进行分配,提出了动态可协调分配机制,在合理地分配资源,满足所有任务正常执行时所需资源的同时,最大化任务的执行效益。实验仿真及对比结果表明,在任务完成时间、任务执行的平均成本、任务完成成功率上,本文算法都取得了较好的效果。     

An Effective Cloud Workflow Scheduling Approach Combining PSO and Idle Time Slot-Aware Rules

Workflow scheduling is a key issue and remains a challenging problem in cloud computing.Faced with the large number of virtual machine(VM)types offered by cloud providers,cloud users need to choose the most appropriate VM type for each task.Multiple task scheduling sequences exist in a workflow application.Different task scheduling sequences have a significant impact on the scheduling performance.It is not easy to determine the most appropriate set of VM types for tasks and the best task scheduling sequence.Besides,the idle time slots on VM instances should be used fully to increase resources'utilization and save the execution cost of a workflow.This paper considers these three aspects simultaneously and proposes a cloud workflow scheduling approach which combines particle swarm optimization(PSO)and idle time slot-aware rules,to minimize the execution cost of a workflow application under a deadline constraint.A new particle encoding is devised to represent the VM type required by each task and the scheduling sequence of tasks.An idle time slot-aware decoding procedure is proposed to decode a particle into a scheduling solution.To handle tasks'invalid priorities caused by the randomness of PSO,a repair method is used to repair those priorities to produce valid task scheduling sequences.The proposed approach is compared with state-of-the-art cloud workflow scheduling algorithms.Experiments show that the proposed approach outperforms the comparative algorithms in terms of both of the execution cost and the success rate in meeting the deadline.     

混合云中的一个高效协调器

云计算提供了4种部署模型:公有云、私有云、社区云和混合云.通常,一个私有云中可用的资源是有限的,因此云用户不得不从公有云租用资源.这意味着云用户将会产生额外的费用.越来越多的企业选择混合云来部署它们的应用.在混合云中,为了实现用户的利益最大化,必须满足使用资源的费用最小化和用户的QoS,为此为混合云用户提供了一个既能最小化资源费用又能保证满足QoS的资源分配方法.实验结果表明,该算法在保持低操作成本的同时还满足了用户的QoS.     

A delay-based dynamic scheduling algorithm for bag-of-task workflows with stochastic task execution times in clouds

Bag-of-Tasks (BoT) workflows are widespread in many big data analysis fields. However, there are very few cloud resource provisioning and scheduling algorithms tailored for BoT workflows. Furthermore, existing algorithms fail to consider the stochastic task execution times of BoT workflows which leads to deadline violations and increased resource renting costs. In this paper, we propose a dynamic cloud resource provisioning and scheduling algorithm which aims to fulfill the workflow deadline by using the sum of task execution time expectation and standard deviation to estimate real task execution times. A bag-based delay scheduling strategy and a single-type based virtual machine interval renting method are presented to decrease the resource renting cost. The proposed algorithm is evaluated using a cloud simulator ElasticSim which is extended from CloudSim. The results show that the dynamic algorithm decreases the resource renting cost while guaranteeing the workflow deadline compared to the existing algorithms.     

Profit maximization based task scheduling in hybrid clouds using whale optimization technique

ABSTRACT

In cloud computing system, task scheduling plays an important key role. The tasks provided by the user to allocate in cloud have to pay for the share of resources that are used by them. The requirement of task scheduling in the cloud environment has become more and more complex, and the amount of resources and tasks is growing rapidly. Therefore, an efficient task-scheduling algorithm is necessary for allocating the task efficiently in the cloud, which can achieve minimum resource utilization, minimum processing time, high efficiency, and maximum profit. In hybrid clouds to maximize the profit of a private cloud while guaranteeing the service delay bound of delay-tolerant tasks is studied in this article. Here, a new metaheuristic technique inspired from the bubble-net hunting technique of humpback whales, namely whale optimization algorithm (WOA), has been applied to solve the task-scheduling problem. Then WOA algorithm is compared with existing algorithms such as artificial bee colony algorithm (ABC) and Genetic algorithm (GA). The experimental result shows that the proposed WOA algorithm greatly increases the efficiency and achieves maximum profit for the private cloud.     

Science in the Cloud: Allocation and Execution of Data-Intensive Scientific Workflows

An important challenge for the adoption of cloud computing in the scientific community remains the efficient allocation and execution of data-intensive scientific workflows to reduce execution time and the size of transferred data. The transferred data overhead is becoming significant with emerging scientific workflows that have input/output files and intermediate data products ranging in the hundreds of gigabytes. The allocation of scientific workflows on public clouds can be described through a variety of perspectives and parameters, and has been proved to be NP-complete. This paper proposes an evolutionary approach for task allocation on public clouds considering data transfer and execution time. In our framework, a solution is represented using an allocation chromosome that encodes the allocation of tasks to nodes, and an ordering chromosome that defines the execution order according to the scientific workflow representation. We propose a multi-objective optimization that relies on a cloud cost model and employs tailored evolution operators. Starting from a population of possible solutions, we employ crossover and mutation operators on both chromosomes aiming at optimizing the data transferred between nodes as well as the total workflow runtime. The crossover operators combine parts of solutions to reduce data overhead, whereas the mutation operators swamp between parts of the same chromosome according to pre-defined rules. Our experimental study compares between the proposed approach and current state-of-the art approaches using synthetic and real-life workflows. Our algorithm performs similarly to existing heuristics for small workflows and shows up to 80 % improvements for larger synthetic workflows. To further validate our approach we compare between the allocation and scheduling obtained by our approach with that obtained by popular scientific workflow managers, when real workflows with hundreds of tasks are executed on a public cloud. The results show a 10 % improvement in runtime over existing schedulers, caused by a 80 % reduction in transferred data and optimized allocation and ordering of tasks. This improved data locality has greater impact as it can be employed to improve and study data provenance and facilitate data persistence for scientific workflows.     

异构云环境下基于分簇的云资源感知任务调度方案

针对提高异构云平台中资源调度的效率,提出了一种基于任务和资源分簇的异构云计算平台任务调度方案。利用K-means算法,根据任务的CPU和I/O处理时间对任务分簇,根据资源的计算能力对资源分簇;然后,将任务簇对应到合适的资源簇,并利用最早截止时间优先（EDF）算法对任务簇中的独立任务进行调度,利用提出的改进型最小关键路径（MCP）算法对依赖性任务进行调度。实验结果表明,在资源异构的云计算环境中,该方案执行任务时间短、能耗低。     

一种多重约束下确保成功率的云工作流调度方法

为提高多重约束下的调度成功率,提出一种满足期限和预算双重约束的云工作流调度算法.将可行工作流调度方案求解分解为工作流结构分层、预算分配、期限分配、任务选择和实例选择.工作流结构分层将所有工作流任务划分层次形成包任务,以提高并行执行程度;预算分配对整体预算在层次间进行分割;期限分配将全局期限在不同层次间分割;任务选择基于...     

适用于实例密集型云工作流的调度算法

工作流调度算法仅适用于单个复杂工作流实例,而不适用于实例密集型云工作流实例,为此,提出了基于实例密集型的云工作流调度算法(MCUD)。MCUD算法先对待处理的一组工作流实例进行分类,再对分类后的同类工作流实例采用一种新的分配方法将用户指定的总最后期限分配到各任务;同时,在调度的过程中动态地调整后续任务的子最后期限。MCUD算法对同类工作流实例中的任务分配不同子最后期限,减小了资源竞争,提高了资源的利用率。仿真实验表明,MCUD相比于其他算法,在满足总的最后期限的前提下更进一步地降低了执行成本和执行时间。     

一种云环境下科学工作流执行计划的优化方法

为降低云环境下科学工作流的执行代价,提出了一种执行计划的优化方法。引入猴群算法,依靠对当前执行计划的层内和层间优化,在保证工作流全局截止时间约束的前提下,通过同层任务的逻辑聚合和任务的层间调整,尽可能减少各层任务数的差异,以避免资源的闲置浪费,缩短任务的等待时间。实验表明,该方法与类似研究相比,可降低资源消耗量,减小总的延迟时间。     

一种云工作流任务调度能效优化算法

工作流任务执行时带来的高能耗不仅会增加云资源提供方的经济成本,而且会降低云系统的可靠性。为了满足截止时间的同时,降低工作流执行能耗,提出一种工作流能效调度算法CWEES。算法将能效优化调度划分为三个阶段：初始任务映射、处理器资源合并和任务松驰。初始任务映射旨在通过任务自底向上分级排序得到任务调度初始序列,处理器资源合并旨在通过重用松驰时间合并相对低效率的处理器,降低资源使用数量,任务松驰旨在为每个任务重新选择带有合适电压/频率等级的最优目标资源,在不违背任务顺序和截止时间约束前提下降低工作流执行总能耗。通过随机工作任务模型对算法的性能进行了仿真实验分析。结果表明,CWEES算法不仅资源利用率更高,而且可以在满足截止时间约束下降低工作流执行能耗,实现执行效率与能耗的均衡。     

Cost-Time Efficient Scheduling Plan for Executing Workflows in the Cloud

The emergence of Cloud Computing as a model of service provisioning in distributed systems instigated researchers to explore its pros and cons on executing different large scale scientific applications, i.e., Workflows. One of the most challenging problems in clouds is to execute workflows while minimizing the execution time as well as cost incurred by using a set of heterogeneous resources over the cloud simultaneously. In this paper, we present, Budget and Deadline Constrained Heuristic based upon Heterogeneous Earliest Finish Time (HEFT) to schedule workflow tasks over the available cloud resources. The proposed heuristic presents a beneficial trade-off between execution time and execution cost under given constraints. The proposed heuristic is evaluated for different synthetic workflow applications by a simulation process and comparison is done with state-of-art algorithm i.e. BHEFT. The simulation results show that our proposed scheduling heuristic can significantly decrease the execution cost while producing makespan as good as the best known scheduling heuristic under the same deadline and budget constraints.     

Cost minimization for bag-of-tasks workflows in a federation of clouds

We address the problem of resource allocation for bag-of-tasks (BoT) workflows in a federation of clouds and formulate it as an integer linear programming problem. The proposed model minimizes financial cost including fees for running VMs and fees for data transfer, and fulfills deadline and resource constraints in the clouds. We also formulate the problem of BoT scheduling in the hybrid clouds, and compare the financial cost in the federation of clouds with that in the hybrid clouds. Moreover, this paper discusses sensitivity analysis to investigate stability in the related allocation problem. Numerical results show that the resource allocation in the federation is considerably preferred to that in the hybrid clouds in terms of stability and cost-saving. In this paper, we also propose an approach named GRASP-FC for obtaining an approximate optimal solution of BoT scheduling in the federation. GRASP-FC is an extension of greedy randomized adaptive search procedure (GRASP), and it can be of great interest from the computational points of view.

     

基于多种云环境的任务调度算法综述

任务调度算法是云计算资源分配部署的核心方法。针对当前云计算发展面临的任务需求和数据量指数级增长的问题,重点对任务调度算法进行了系统的梳理和归纳,以云环境为分类依据,研究分析了单云、联盟云、混合云、多云四类调度算法。在单云环境中,从传统启发式、元启发式以及混合式任务调度算法角度进行阐述。在联盟云、混合云、多云环境中,从工作流和独立任务调度算法角度进行阐述。通过比较,总结了现有算法的优点、缺点以及优化性能,并形成结论性意见和开放性问题,为未来对容器云、数据云以及兼顾资源分配与任务调度算法的研究奠定基础。