Kubernetes异构资源细粒度调度策略的设计与实现

在异构资源环境中高效利用计算资源是提升任务效率和集群利用率的关键。Kuberentes作为容器编排领域的首选方案,在异构资源调度场景下调度器缺少GPU细粒度信息无法满足用户自定义需求,并且CPU/GPU节点混合部署下调度器无法感知异构资源从而导致资源竞争。综合考虑异构资源在节点上的分布及其硬件状态,提出一种基于Kubernetes的CPU/GPU异构资源细粒度调度策略。利用设备插件机制收集每个节点上GPU的详细信息,并将GPU资源指标提交给调度算法。在原有CPU和内存过滤算法的基础上,增加自定义GPU信息的过滤,从而筛选出符合用户细粒度需求的节点。针对CPU/GPU节点混合部署的情况,改进调度器的打分算法,动态感知应用类型,对CPU和GPU应用分别采用负载均衡算法和最小最合适算法,保证异构资源调度策略对不同类型应用的正确调度,并且在CPU资源不足的情况下充分利用GPU节点的碎片资源。通过对GPU细粒度调度和CPU/GPU节点混合部署情况下的调度效果进行实验验证,结果表明该策略能够有效进行GPU调度并且避免资源竞争。     

Experiences from integrating algorithmic and systemic load balancing strategies

Load balancing increases the efficient use of existing resources for parallel and distributed applications. At a coarse level of granularity, advances in runtime systems for parallel programs have been proposed in order to control available resources as efficiently as possible by utilizing idle resources and using task migration. Simultaneously, at a finer granularity level, advances in algorithmic strategies for dynamically balancing computational loads by data redistribution have been proposed in order to respond to variations in processor performance during the execution of a given parallel application. Combining strategies from each level of granularity can result in a system which delivers advantages of both. The resulting integration is systemic in nature and transfers the responsibility of efficient resource utilization from the application programmer to the runtime system. This paper presents the design and implementation of a system that combines an algorithmic fine-grained data parallel load balancing strategy with a systemic coarse-grained task-parallel load balancing strategy, and reports on recent experimental results of running a computationally intensive scientific application under this integrated system. The experimental results indicate that a distributed runtime environment which combines both task and data migration can provide performance advantages with little overhead. It also presents proposals for performance enhancements of the implementation, as well as future explorations for effective resource management. Copyright © 2001 John Wiley & Sons, Ltd.     

计算网格中基于时间均衡的并行粗粒度任务调度算法

考虑网格资源异构、自治、动态等特性,讨论本地用户具有强占优先权情况下的任务调度问题,提出了TBBS(Time-Balancing Based Scheduling Algorithm)算法.建立调度优化模型,以期望完成时间最小为目标选择执行任务的最佳资源组合.以时间均衡策略将任务分解并调度到资源上执行,减少了子任务同步时因等待而产生的延时,获得较好的并行计算性能.采用重复调度策略,适应计算网格中资源的特性.     

数据驱动的多星任务网络预测调度算法

多星任务调度是具有NP-hard特性的优化问题,随着卫星资源规模和任务需求规模的双重增长,传统调度方法求解效率不高.在轨卫星在常年运行过程中积累了丰富的调度数据.针对大规模多星任务调度场景,建立多星多波束任务调度模型,并提出数据驱动的多星任务网络预测调度算法对其求解.以分割的思想,实现多星场景下任务可调度性预测.从历史调度数据中,提取设定的3个静态特征和5个动态特征,构建并训练预测网络,预测任务被不同卫星完成的概率,并以冲突避免、负载均衡等为原则,得到初始任务和资源卫星的分配方案.进一步设计双链结构的进化算法,以双链编码形式表征上述关系,配合设计的交叉、修复等进化算子,优化初始方案中的任务序列与资源分配关系,输出最终任务调度方案.仿真结果表明,与改进蚁群算法、混合遗传算法和数据驱动并行调度算法相比,所提出算法在运行时间、方案收益和卫星负载均衡3方面均有较好的表现.     

网络负载均衡优化算法的建模与仿真

研究网络资源管理中的负载均衡与优化问题,网络资源有限且负载具有突发性,造成资源浪费。采用传统单一蚁群算法或遗传算法均存在各自不足,难以适应用网络负载变化特点,使网络资源利用率低,网络拥塞严重。为了提高网络资源利用率,使网络负载更加均衡,提出一种蚁群-遗传算法的网络负载均衡方法。首先利用遗传算法对网络负载均衡问题进行全局搜索,使网络负载均衡的解迅速处于全局最优区域解附近,然后将遗传算法的解作为蚁群算法初始信息素,进行进一步搜索,最后找到网络负载均衡的最优解。仿真结果表明,用蚁群-遗传算法提高了网络资源利用率,降低网络丢包率,提高网络整体性能。     

A hybrid heuristic queue based algorithm for task assignment in mobile cloud

This paper presents a novel algorithm for task assignment in mobile cloud computing environments in order to reduce offload duration time while balancing the cloudlets’ loads. The algorithm is proposed for a two-level mobile cloud architecture, including public cloud and cloudlets. The algorithm models each cloud and cloudlet as a queue to consider cloudlets’ limited resources and study response time more accurately. Performance factors and resource limitations of cloudlets such as waiting time for clients in cloudlets can be determined using queue models. We propose a hybrid genetic algorithm (GA) - Ant Colony Optimization (ACO) algorithm to minimize mean completion time of offloaded tasks for the whole system. Simulation results confirm that the proposed hybrid heuristic algorithm has significant improvements in terms of decreasing mean completion time, total energy consumption of the mobile devices, number of dropped tasks over Queue based Random, Queue based Round Robin and Queue based weighted Round Robin assignment algorithms. Also, to prove the superiority of our queue based algorithm, it is compared with a dynamic application scheduling algorithm, HACAS, which has not considered queue in cloudlets.     

面向建材装备集团制造的分布式多项目资源调度

针对建材装备集团项目执行过程中存在的项目内和项目间多类别资源协同共用现象,提出并行调度机制下考虑多类别资源转移时间和转移成本的分布式多项目资源调度问题,以最小化资源转移成本和项目执行工期为目标建立问题的数学模型.为改善进化算法在局部搜索能力方面的不足,提出将禁忌搜索与进化算法相结合,构造一种内嵌禁忌搜索寻优搜索的多目标混合进化算法,在保证算法全局搜索能力的前提下提升局部精确搜索能力.同时,考虑资源转移成本和时间对任务选取的影响,改进任务选择的优先权值,提出并行调度机制下资源转移冲突消解策略.数据实验表明,所提算法能够有效避免不合理的资源转移,在求解质量方面具有良好的性能.     

云计算环境下资源调度系统设计与实现

在云计算环境下,对开放的网络大数据库信息系统中的数据进行优化调度,提高数据资源的利用效率和配置优化能力;传统的资源调度算法采用资源信息的自相关匹配方法进行资源调度,当数据传输信道中的干扰较大及资源信息流的先验数据缺乏时,资源调度的均衡性不好,准确配准度不高;提出一种基于云计算资源负载均衡控制和信道自适应均衡的资源调度算法,并进行调度系统的软件开发和设计;首先构建了云计算环境下开放网络大数据库信息资源流的时间序列分析模型,采用自适应级联滤波算法对拟合的资源信息流进行滤波降噪预处理,提取滤波输出的资源信息流的关联维特征,通过资源负载均衡控制和信道自适应均衡算法实现资源调度改进;仿真结果表明,采用资源调度算法进行资源调度系统的软件设计,提高了资源调度的信息配准能力和抗干扰能力,计算开销较小,技术指标具有优越性。     

分布式计算中基于资源分级的自适应Min-Min算法*

Min-Min任务调度算法的思路总是优先调度执行时间较短的小任务,无法得到理想的最优跨度及资源负载平衡．针对该问题,提出基于资源分级的自适应Min-Min算法．分配任务前,先参考现有资源的属性进行分级处理,再与任务在资源中的最小完成时间作乘积得到的最小任务资源组合进行调度;在任务调度过程中,引入自适应阈值,调节长任务的调度等级,从而达到优化效果．通过模拟仿真实验,表明该算法在时间跨度和负载平衡上均有较好性能．     

Multi-objective partial parallel disassembly line balancing problem using hybrid group neighbourhood search algorithm

To reduce waste during disassembly production and improve disassembly efficiency, this study investigates a type of partial parallel disassembly line applicable for the simultaneous disassembly of different products. A multi-objective mathematical model for a partial parallel disassembly line balancing problem is built considering four optimisation goals, namely, the minimisation of the cycle time, number of workstations, idle index, and quantity of disassembly resources. In addition, a novel multi-objective hybrid group neighbourhood search algorithm is proposed. First, a certain set of neighbourhood individuals (from the current population of individuals) is generated via neighbourhood search mechanisms based on optimal embedding and exchange operations. Then, a Pareto filtering process is performed on a mixed population composed of the individuals of the current population and all neighbourhoods. Subsequently, the current population individuals are renewed based on the mixed population. To prevent the algorithm from falling into a local optimum and to enhance the algorithm’s global search performance, we conduct a local search strategy based on a simulated annealing operation on the newly generated population individuals. The effectiveness and superiority of the proposed algorithm are proven by solving two complete disassembly line balancing problems at different scales and a partial disassembly line balancing problem, and also by comparison with several algorithms investigated in existing literature. Finally, the proposed model and algorithm are applied to a partial parallel disassembly line designed for the simultaneous disassembly of two types of waste products in a household appliance disassembly enterprise. The results of the partial parallel disassembly line are compared with those of an initial single-product straight disassembly line, and the comparison results show that the solution results of the optimisation goals for the partial parallel disassembly line are more superior than those of the initial single-product straight disassembly line.     

A hybrid policy for fault tolerant load balancing in grid computing environments

Due to the emergence of grid computing over the Internet, there is a need for a hybrid load balancing algorithm which takes into account the various characteristics of the grid computing environment. Hence, this research proposes a fault tolerant hybrid load balancing strategy namely AlgHybrid_LB, which takes into account grid architecture, computer heterogeneity, communication delay, network bandwidth, resource availability, resource unpredictability and job characteristics. AlgHybrid_LB juxtaposes the strong points of neighbor-based and cluster based load balancing algorithms. Our main objective is to arrive at job assignments that could achieve minimum response time and optimal computing node utilization. Major achievements include low complexity of proposed approach and drastic reduction of number of additional communications induced due to load balancing. A simulation of the proposed approach using Grid Simulation Toolkit (GridSim) is conducted. Experimental results show that the proposed algorithm performs very well in a large grid environment.     

用于图像处理的FPGA存储器优化分配

现场可编程门阵列(FPGA)在计算机视觉应用领域有着广阔的前景,然而FPGA有限的片上存储器资源难以满足应用场景下性能、尺寸和功率的需求。针对这个问题,研究片上存储器的资源分配,在最小化片上资源使用和整体功耗的前提下提出一种易于实现的分区平衡算法。实验结果表明,与商用FPGA高级综合工具相比,本文算法的利用率提高达60%,且动态功耗降低了约70%。在高级算法MeanShift跟踪的实验中,实验结果显示,分区算法可以在不影响关键性能的前提下降低总功耗高达30%。     

Parallel resource co-allocation for the computational grid

Resource co-allocation is one of the crucial problems affecting the utility of the grid. Because the numbers of the application tasks and amounts of required resources are enormous and quick responses to the requirements of users are necessary in the real grid environment, real-time resource co-allocation may be large-scale. A parallel resource co-allocation algorithm based on the framework for mapping with resource co-allocation is proposed in this paper. Through the result of experiments, it is concluded that the parallel method reduces the execution time of the resource co-allocation algorithm significantly, and makes the overall response time to the end-users small.     

A proximity-aware load balancing in peer-to-peer-based volunteer computing systems

One of the main challenges in peer-to-peer-based volunteer computing systems is an efficient resource discovery algorithm. Load balancing is a part of resource discovery algorithm and aims to minimize the overall response time of the system. This paper introduces an analytical model based on distributed parallel queues to optimize the average response time of the system in a distributed manner. The proposed resource discovery algorithm consists of two phases. In the first phase, it selects peers in a load-balanced manner based on QoS constraints of request. In the second phase, a proximity-aware feature is applied to select the peer with minimum communication overhead among selected peers in the first phase. Two dispatching strategies are proposed for the load balancing based on stochastic analysis of routing in the distributed parallel queues. These policies adopt probabilistic and deterministic sequences to redirect requests to the capable peers in the system. Simulation results show that the proposed resource discovery algorithm improves the response time of user’s requests by a factor of 1.8 under a moderate load.     

基于PI3000平台的分布式任务调度研究与实现

在PI3000平台中任务调度是应用系统中很重要的部分,应用环境的复杂程度导致各式各样调度的框架和算法.文中提出一个新的分布式负载均衡的任务调度框架,以解决在并行任务中特定的应用环境下的任务调度问题.这个框架是动态的、可重用的,通过提供给各种不同的资源环境对应的接口,来进行跨服务的调度.整个框架关注于实际应用环境下任务、资源的动态不稳定性和任务计算的快速响应.     

An efficient grid scheduling strategy for data parallel applications

Scheduling large-scale application in heterogeneous grid systems is a fundamental NP-complete problem that is critical to obtain good performance and execution cost. To achieve high performance in a grid system it requires effective task partitioning, resource management and load balancing. The heterogeneous and dynamic nature of a grid, as well as the diverse demands of applications running on the grid, makes grid scheduling a major task. Existing schedulers in wide-area heterogeneous systems require a large amount of information about the application and the grid environment to produce reasonable schedules. However, this required information may not be available, may be too expensive to collect, or may increase the runtime overhead of the scheduler such that the scheduler is rendered ineffective. We believe that no one scheduler is appropriate for all grid systems and applications. This is because while data parallel applications in which further data partitioning is possible can be further improved by efficient management of resources, smart selection of resources and load balancing can be possible, in functional/not-dividable-task parallel applications such partitioning is either not possible or difficult or expensive in term of performance. In this paper, we propose a scheduler for data parallel applications (SDPA) which offers an efficient task partitioning and load balancing strategy for data parallel applications in grid environment. The proposed SDPA offers two major features: maintaining job priority even if insufficient number of free resources is available and pre-task assignment to cut the idle time of nodes. The SDPA selects nodes smartly according to the nature of task and the nodes’ resources availability. Simulation results conducted reveal that SDPA achieves performance improvement over reported strategies in the reviewed literature in terms of execution time, throughput and waiting time.     

支持随机服务请求的云虚拟机按需物理资源分配方法

本文针对云平台按负载峰值需求配置处理机资源、提供单一的服务应用和资源需求动态变化导致资源利用率低下的问题,采用云虚拟机中心来同时提供多种服务应用.利用灰色波形预测算法对未来时间段内到达虚拟机的服务请求量进行预测,给出兼顾资源需求和服务优先等级的虚拟机服务效用函数,以最大化物理机的服务效用值为目标,为物理机内的各虚拟机动态配置物理资源.通过同类虚拟机间的全局负载均衡和多次物理机内各虚拟机的物理资源再分配,进一步增加服务请求量较大的相应类型的虚拟机的物理资源分配量.最后,给出了虚拟机中心基于灰色波形预测的按需资源分配算法ODRGWF.模拟实验表明所提算法能够有效提高云平台中处理机的资源利用率,对提高用户请求完成率以及服务质量都具有实际意义.     

Multimedia resource allocation strategy of wireless sensor networks using distributed heuristic algorithm in cloud computing environment

The virtualized resource allocation (mapping) algorithm is the core issue of network virtualization technology. Universal and excellent resource allocation algorithms not only provide efficient and reliable network resources sharing for systems and users, but also simplify the complexity of resource scheduling and management, improve the utilization of basic resources, balance network load and optimize network performance. Based on the application of wireless sensor network, this paper proposes a wireless sensor network architecture based on cloud computing. The WSN hardware resources are mapped into resources in cloud computing through virtualization technology, and the resource allocation strategy of the network architecture is proposed. The experiment evaluates the performance of the resource allocation strategy. The proposed heuristic algorithm is a distributed algorithm. The complexity of centralized algorithms is high, distributed algorithms can handle problems in parallel, and reduce the time required to get a good solution with limited traffic.

     

新的BPOVC算法在图像三维重建中的运用

OVC算法是基于图像的三维体重建技术的重要算法,但其算法复杂度较高,POVC算法是并行化的OVC算法。在POVC算法的基础上,使用最近邻居负载平衡方法,得到新的BPOVC算法,该算法具有明显优于POVC算法的并行效率和并行加速比,能够使OVC算法具有更高的实用价值,同时也体现了负载平衡算法在并行算法设计中的一个十分有意义的运用。     

一种基于多种资源的负载平衡算法的设计与模拟

系统资源的有效利用是集群系统的关键问题，负载平衡是实现资源有效利用的重要手段。本文中，我们提出了一种基于资源使用率和向量负载指数的、采用进程迁移机制的负载平衡算法，并通过踪迹驱动的方法进行了大量的模拟和分析。