云工作流中基于分时虚拟机的任务层调度算法

云计算是新的一种面向市场的商业计算模式,向用户按需提供服务,云计算的商业特性使其关注向用户提供服务的服务质量。任务调度和资源分配是云计算中两个关键的技术,所使用的虚拟化技术使得其资源分配和任务调度有别于以往的并行分布式计算。目前主要的调度算法是借鉴网格环境下的调度策略,研究基于QoS的调度算法,存在执行效率较低的问题。我们对云工作流任务层调度进行深入研究,分析由底层资源虚拟化形成的虚拟机的特性,结合工作流任务的各类QoS约束,提出了基于虚拟机分时特性的任务层ACS调度算法。经过试验,我们提出的算法相比于文献[1]中的算法在对于较多并行任务的执行上存在较大的优势,能够很好的利用虚拟的分时特性,优化任务到虚拟机的调度。     

云工作流中基于分时虚拟机的任务层调度算法

云计算是新的一种面向市场的商业计算模式,向用户按需提供服务,云计算的商业特性使其关注向用户提供服务的服务质量。任务调度和资源分配是云计算中两个关键的技术,所使用的虚拟化技术使得其资源分配和任务调度有别于以往的并行分布式计算。目前主要的调度算法是借鉴网格环境下的调度策略,研究基于QoS的调度算法,存在执行效率较低的问题。我们对云工作流任务层调度进行深入研究,分析由底层资源虚拟化形成的虚拟机的特性,结合工作流任务的各类QoS约束,提出了基于虚拟机分时特性的任务层ACS调度算法。经过试验,我们提出的算法相比于文献[1]中的算法在对于较多并行任务的执行上存在较大的优势,能够很好的利用虚拟的分时特性,优化任务到虚拟机的调度。     

基于负载均衡的云计算任务调度算法的研究

本文针对当前云计算系统负载不均衡和任务完成效率有待提高的问题,提出了一种基于系统整体负载均衡与最小完成时间LB—ECT算法。根据云计算环境下资源需求动态变化,利用任务在虚拟机上执行时间的预测进行任务到虚拟机上的分配、调度。优化系统的整体效率。采用云计算仿真平台CloudSim对本算法进行仿真实验与分析,实验仿真结果表明,LB—ECT算法能够有效提高系统的整体负载均衡能力．明显缩短任务的总完成时间．     

基于云科学工作流调度的代价与能效优化算法

为了降低云环境中科学工作流调度的执行代价与数据中心能耗,提出了一种基于能效感知的工作流调度代价最优化算法CWCO-EA。算法在满足截止时间约束下,以最小化工作流执行代价与降低能耗为目标,将工作流的任务调度划分为四步执行。首先,通过代价效用的概念设计虚拟机选择策略,实现了子makespan约束下的任务与最优虚拟机间的映射;其次,通过串行与并行任务合并策略,同步降低了工作流的执行代价与能耗;然后,通过空闲虚拟机重用机制,改善了租用虚拟机的利用率,进一步提高了能效;最后,通过任务松驰策略实现了租用虚拟机的能力回收,节省了能耗。通过四种科学工作流的仿真实验,结果表明,CWCO-EA算法比较同类型算法,在满足截止时间的同时,可以同步降低工作流的执行代价与执行能耗。     

改进遗传算法的并行任务调度

并行任务调度是一个NP完全问题,它关注资源的分配和并行任务调度,要求具有高性能的调度算法,且能求解出高质量的解。提出了一种基于改进遗传算法的并行任务调度算法,在算法初始化种群产生时引入任务向量矩阵来表示任务、资源以及调度的关系,并采用启发式方法得到初始化种群,提高种群质量;采用规则约束的交叉和变异操作,提高个体的质量;提出了加速进化策略,有效地避免了早熟。仿真实验结果表明,该改进算法能更有效地求解并行任务调度问题。     

多租户高可用并行任务调度框架

描述了一种多租户高可用并行任务调度框架MTHPT的设计思想、体系结构和实现技术,MTHPT包括3部分：任务定义与配置、异步并行任务调度模式、消息告警与监视.任务调度引擎和任务执行组件采用分开部署、异步并行调度和快速回调的模式,快速释放调度引擎占用的线程资源,解决了部分任务执行周期长、定时任务无法按时执行等影响业务系统性能的问题.任务调度配置提供了多租户应用模式.实验分析及评估表明,MTHPT提高了应用系统的任务调度并行调度效率和稳定性.     

云环境下基于改进NSGA II的虚拟机调度算法

在云环境中,如何将大量的虚拟机调度到物理节点上是一个基本且复杂的问题。文中首先对虚拟机的调度建立装箱问题模型,将该模型的求解转化一个多目标优化问题,目标分别为负载均衡、提高任务执行效率和降低能耗;接着对基于非支配排序的遗传算法（ Non-dominated Sorting Genetic Algorithm,NSGA II）进行改进,利用回溯法中的剪枝函数确定最优初始种群,引入正态分布密度函数限制优秀精英。仿真结果表明,基于改进NSGA II的虚拟机调度算法在任务执行时间、负载均衡和能量消耗三个方面优于其他一些常用算法。     

异构多核全局限制性可抢占并行任务可调度分析

异构多核平台可以利用不同类别体系结构的处理器来执行特定任务,从而达到提高性能和降低功耗的目的.然而,向大规模异构平台迁移极其困难,且大规模的、必要的程序并行会导致软件调度的复杂度.虽然,基于有向无环图（directed acyclic graph, DAG）并行任务模型已有相关的研究工作,但是基于DAG任务模型的限制性可抢占的调度策略研究仍存在不足.鉴于此,主要讨论了DAG任务在异构平台上进行全局固定优先级限制性可抢占调度时的最差响应时间(worst case response time,WCRT)分析,对并行任务的每个结点可用的处理器资源进行了一定的限制,即只能执行在规定类型处理器上的任务.基于最新的单分类并行任务的可调度性分析,提出了多个并行任务的可调度性分析.进一步,提出了高优先级任务的干涉量与低优先级任务的阻塞量的计算方法;结合最新的分类并行任务的任务内干涉计算方法,最终提出了一种伪多项式的分析方法.实验结果表明,提出的算法能够在合理的时间范围内得到任务集可调度性的分析结果,且任务集的接受率随各个参数的变化符合预期.     

融合遗传和粒子群算法的云工作流调度算法

针对云工作流调度问题,提出一种融合遗传算法和粒子群优化算法的工作流调度负载均衡算法.充分利用多元启发式方法融合的优势,避免遗传算法的收敛过慢和粒子群算法易于陷入局部最优的缺陷,有效将工作流任务映射至虚拟机资源,实现全局工作流执行跨度最小化和虚拟机分配的负载均衡.以算例详细说明算法实现思路,在现实科学工作流条件下进行仿真测试,验证算法性能.与几种单一元启发式调度方法相比,验证该算法拥有更高执行效率和负载均衡度.     

Map/Reduce下快速剪枝算法在复杂任务调度中的应用

云环境下传统任务分配与调度算法对于复杂任务调度的整体效率较低,为了提高Map/Reduce对复杂任务分配调度的整体效率,提出了一种基于任务处理时间的快速剪枝算法。该算法首先将复杂任务按照任务依赖关系大小进行最佳拓扑排序,使任务按顺序执行,从而提高调度准确率。然后使用节点处理任务的预测时间与节点处理能力的比值作为子任务在每个节点的处理时间进行量化建模,建立任务和处理时间的度量矩阵,通过采用按阶剪枝方法逐渐缩小任务分配规模,对N个节点处理N个任务的分配问题,进行N-1次操作可获得任务分配的最优解。运用Hadoop平台进行实验验证,从任务调度效率与资源使用率角度将剪枝算法与公平调度算法、遗传算法和GRAPHENE算法进行对比验证。实验结果表明剪枝算法能明显提高任务调度的整体效率,充分利用各节点的计算能力提高Map/Reduce调度效率。     

Dynamic scheduling of a batch of parallel task jobs on heterogeneous clusters

This paper addresses the problem of minimizing the scheduling length (make-span) of a batch of jobs with different arrival times. A job is described by a direct acyclic graph (DAG) of parallel tasks. The paper proposes a dynamic scheduling method that adapts the schedule when new jobs are submitted and that may change the processors assigned to a job during its execution. The scheduling method is divided into a scheduling strategy and a scheduling algorithm. We also propose an adaptation of the Heterogeneous Earliest-Finish-Time (HEFT) algorithm, called here P-HEFT, to handle parallel tasks in heterogeneous clusters with good efficiency without compromising the makespan. The results of a comparison of this algorithm with another DAG scheduler using a simulation of several machine configurations and job types shows that P-HEFT gives a shorter makespan for a single DAG but scores worse for multiple DAGs. Finally, the results of the dynamic scheduling of a batch of jobs using the proposed scheduler method showed significant improvements for more heavily loaded machines when compared to the alternative resource reservation approach.     

云中多媒体应用中基于混合DAG的最优任务调度研究

云计算的平台优势使得它在多媒体应用中得到广泛使用。由于多媒体服务的多样性和异构性,如何将多媒体任务有效地调度至虚拟机进行处理成为当前多媒体应用的研究重点。对此,研究了云中多媒体最优任务调度问题,首先引入有向无环图来模拟任务中的优先级及任务之间的依赖性,分别对串行、并行、混合结构任务调度模型进行任务调度研究,根据有限资源成本将关键路径中任务节点融合,提出一种实用的启发式近似最优调度方法。实验结果表明,所提调度方法能够以最短的执行时间在有限的资源成本下完成最优的任务分配。     

Xen虚拟机资源调度优化算法

针对Xen虚拟化平台中虚拟机资源分配不合理的问题,提出了两种资源调度优化算法,即细粒度优化算法和粗粒度优化算法.细粒度优化算法主要解决单个物理节点上虚拟机资源分配不合理问题,能够根据物理节点上运行的各虚拟机的资源利用情况来调整资源分配量,适当增加利用率较高的虚拟机的资源,减少资源利用率低的虚拟机的资源,从而优化资源分配,提高资源利用效率,避免不必要的虚拟机迁移.粗粒度优化算法是针对集群中多个物理节点之间虚拟机负载不均衡问题而提出的.该算法结合粒子群优化技术,选择将集群系统中热点物理机上的部分虚拟机迁移到最适合的冷点物理机上,从而避免高载物理机宕机.实验结果表明,这两种资源调度优化算法能够有效解决虚拟机资源分配不合理的问题,具有较好的适用性和应用前景.     

Task mapper and application-aware virtual machine scheduler oriented for parallel computing

We design a task mapper TPCM for assigning tasks to virtual machines, and an application-aware virtual machine scheduler TPCS oriented for parallel computing to achieve a high performance in virtual computing systems. To solve the problem of mapping tasks to virtual machines, a virtual machine mapping algorithm (VMMA) in TPCM is presented to achieve load balance in a cluster. Based on such mapping results, TPCS is constructed including three components: a middleware supporting an application-driven scheduling, a device driver in the guest OS kernel, and a virtual machine scheduling algorithm. These components are implemented in the user space, guest OS, and the CPU virtualization subsystem of the Xen hypervisor, respectively. In TPCS, the progress statuses of tasks are transmitted to the underlying kernel from the user space, thus enabling virtual machine scheduling policy to schedule based on the progress of tasks. This policy aims to exchange completion time of tasks for resource utilization. Experimental results show that TPCM can mine the parallelism among tasks to implement the mapping from tasks to virtual machines based on the relations among subtasks. The TPCS scheduler can complete the tasks in a shorter time than can Credit and other schedulers, because it uses task progress to ensure that the tasks in virtual machines complete simultaneously, thereby reducing the time spent in pending, synchronization, communication, and switching. Therefore, parallel tasks can collaborate with each other to achieve higher resource utilization and lower overheads. We conclude that the TPCS scheduler can overcome the shortcomings of present algorithms in perceiving the progress of tasks, making it better than schedulers currently used in parallel computing.     

On the efficiency of several VM provisioning strategies for workflows with multi-threaded tasks on clouds

Cloud computing promises the delivery of on-demand pay-per-use access to unlimited resources. Using these resources requires more than a simple access to them as most clients have certain constraints in terms of cost and time that need to be fulfilled. Therefore certain scheduling heuristics have been devised to optimize the placement of client tasks on allocated virtual machines. The applications can be roughly divided in two categories: independent bag-of-tasks and workflows. In this paper we focus on the latter and investigate a less studied problem, i.e., the effect the virtual machine allocation policy has on the scheduling outcome. For this we look at how workflow structure, execution time, virtual machine instance type affect the efficiency of the provisioning method when cost and makespan are considered. To aid our study we devised a mathematical model for cost and makespan in case single or multiple instance types are used. While the model allows us to determine the boundaries for two of our extreme methods, the complexity of workflow applications calls for a more experimental approach to determine the general relation. For this purpose we considered synthetically generated workflows that cover a wide range of possible cases. Results have shown the need for probabilistic selection methods in case small and heterogeneous execution times are used, while for large homogeneous ones the best algorithm is clearly noticed. Several other conclusions regarding the efficiency of powerful instance types as compared to weaker ones, and of dynamic methods against static ones are also made.     

一种基于资源预分配的虚拟机软实时调度方法

虚拟机技术作为云计算的重要技术之一,近年来得到广泛关注,但是由于虚拟机管理层的存在,导致语义鸿沟,使得实时应用程序、并发程序等在虚拟机上的运行性能受到影响。分析和研究了Xen虚拟机管理器的Credit调度算法,针对其在并发调度和软实时调度方面存在的不足,提出了改进调度算法,实现了算法的调度器原型。新的调度算法对软实时虚拟机进行Credit比例预分配,采用动态调度时间片机制,以non-work-conserving方式实现软实时任务周期调度,保障调度周期满足运行周期要求。通过区分并发和非并发软实时虚拟机,采取不同的调度策略,在满足资源利用率的基础上,确保实时任务的顺利运行。测试结果表明,该调度算法在对并发和非并发软实时任务调度上,具有良好的表现,较好满足了软实时应用调度需求。     

Honey bee behavior inspired load balancing of tasks in cloud computing environments

Scheduling of tasks in cloud computing is an NP-hard optimization problem. Load balancing of non-preemptive independent tasks on virtual machines (VMs) is an important aspect of task scheduling in clouds. Whenever certain VMs are overloaded and remaining VMs are under loaded with tasks for processing, the load has to be balanced to achieve optimal machine utilization. In this paper, we propose an algorithm named honey bee behavior inspired load balancing (HBB-LB), which aims to achieve well balanced load across virtual machines for maximizing the throughput. The proposed algorithm also balances the priorities of tasks on the machines in such a way that the amount of waiting time of the tasks in the queue is minimal. We have compared the proposed algorithm with existing load balancing and scheduling algorithms. The experimental results show that the algorithm is effective when compared with existing algorithms. Our approach illustrates that there is a significant improvement in average execution time and reduction in waiting time of tasks on queue.     

云计算环境下基于蚁群优化算法的资源调度策略

针对当前云计算环境中节点规模巨大,单个节点资源配置较低,寻找有效计算资源效率不高的缺点,文中在Google公司的Map/Reduce框架上提出了两个基于蚁群优化的资源调度策略ACO1和ACO2,并在这两个资源调度策略中引入双向蚂蚁机制。在该双向蚂蚁机制中蚂蚁通过相互交流,能够快速地发现合适的虚拟机资源,从而使得Master节点能够快速地为用户任务分配虚拟机。实验结果表明这两个利用了双向蚂蚁机制的资源调度策略显著减少了为用户任务寻找虚拟机的时间,从而使得用户任务能够更快地获得虚拟机,保证用户作业能够按时完成。     

面向Flink的负载均衡任务调度算法的研究与实现

Apache Flink是现在主流的大数据分布式计算引擎之一,其中任务调度问题是分布式计算系统中的关键问题。由于集群的异构性以及不同算子复杂度不同,大数据计算系统Flink中不可避免地会出现负载不均的情况,针对这种问题,提出了基于资源反馈的负载均衡任务调度算法RFTS。通过实时资源监控、区域划分和基于人工萤火虫优化的任务调度算法3个模块,把负载过重的机器中处于等待状态的任务分配给负载较轻的机器,来实现集群的负载均衡,提高系统集群利用率和执行效率。最后通过基于TPC-C和TPC-H数据集的实验结果表明,RFTS算法从执行时间和吞吐量2个方面有效提升了Apache Flink计算系统的性能。     

Dynamic forecast scheduling algorithm for virtual machine placement in cloud computing environment

In most cloud computing platforms, the virtual machine quotas are seldom changed once initialized, although the current allocated resources are not efficiently utilized. The average utilization of cloud servers in most datacenters can be improved through virtual machine placement optimization. How to dynamically forecast the resource usage becomes a key problem. This paper proposes a scheduling algorithm called virtual machine dynamic forecast scheduling (VM-DFS) to deploy virtual machines in a cloud computing environment. In this algorithm, through analysis of historical memory consumption, the most suitable physical machine can be selected to place a virtual machine according to future consumption forecast. This paper formalizes the virtual machine placement problem as a bin-packing problem, which can be solved by the first-fit decreasing scheme. Through this method, for specific virtual machine requirements of applications, we can minimize the number of physical machines. The VM-DFS algorithm is verified through the CloudSim simulator. Our experiments are carried out on different numbers of virtual machine requests. Through analysis of the experimental results, we find that VM-DFS can save 17.08 % physical machines on the average, which outperforms most of the state-of-the-art systems.