Integration of task abortion and security requirements in GA-based meta-heuristics for independent batch grid scheduling

Scheduling and resource allocation in large scale distributed environments, such as Computational Grids (CGs), arise new requirements and challenges not considered in traditional distributed computing environments. Among these new requirements, task abortion and security become needful criteria for Grid schedulers. The former arises due to the dynamics of the Grid systems, in which resources are expected to enter and leave the system in an unpredictable way. The latter requirement appears crucial in Grid systems mainly due to a multi-domain nature of CGs. The main aim of this paper is to develop a scheduling model that enables the aggregation of task abortion and security requirements as additional, together with makespan and flowtime, scheduling criteria into a cumulative objective function. We demonstrate the high effectiveness of genetic-based schedulers in finding near-optimal solutions for multi-objective scheduling problem, where all criteria (objectives) are simultaneously optimized. The proposed meta-heuristics are experimentally evaluated in static and dynamic Grid scenarios by using a Grid simulator. The obtained results show the fast reduction of the values of basic scheduler performance metrics, especially in the dynamic case, that confirms the usefulness of the proposed approach in real-life scenarios.     

用爬山法实现无中心式网格调度

为方便网格资源的扩展,网格调度应当是无中心的.为在尽可能多的计算资源中为单地点作业优化资源选择,这里采用了爬山算法.当一个网格调度器收到一个单地点作业,爬山法被激活,根据网格调度器之间的相邻关系为作业找出最适合的计算系统,这里每个计算系统的适合度用预测的作业响应时间表示.实验模拟了无中心式网格调度与计算系统之间的性能差别,每个计算系统的本地调度采用保守式装填法,网格工作负荷由模型得到,并用一段工作负荷的平均响应时间衡量调度性能.实验结果表明,即使在作业提交点分布不均匀且运行时间估计不准确情况下,爬山法仍可有效改善单地点作业的调度.     

Meeting security and user behavior requirements in Grid scheduling

Most of current research in Grid computing is still focused on the improvement of the performance of Grid schedulers. However, unlike traditional scheduling, in Grid systems there are other important requirements to be taken into account. One such a requirement is the secure scheduling, namely achieving an efficient allocation of tasks to reasonable trustful resources. In this paper we formalize the Grid scheduling problem as a non-cooperative non-zero sum game of the Grid users in order to address the security requirements. The premise of this model is that in a large-scale Grid, the cooperation among all users in the system is unlikely to happen. The users’ cost of playing the game is interpreted as a total cost of the secure job execution in Grid. The game cost function is minimized, at global (Grid) and local (users) levels, by using four genetic-based hybrid meta-heuristics. We have evaluated the proposed model under the heterogeneity, the large-scale and dynamics conditions using a Grid simulator. The relative performance of four hybrid schedulers is measured by the makespan and flowtime metrics. The obtained results suggested that it is more resilient for the Grid users to pay some additional scheduling cost, due to verification of the security conditions, instead of taking the risk of assigning their tasks to unreliable resources.     

Self-adjustment of resource allocation for grid applications

Grids involve coordinated resource sharing and problem solving in heterogeneous dynamic environments to meet the needs of a generation of researchers requiring large amounts of bandwidth and more powerful computational resources. The lack of resource ownership by grid schedulers and fluctuations in resource availability require mechanisms which will enable grids to adjust themselves to cope with fluctuations. The lack of a central controller implies a need for self-adaptation. Grids must thus be enabled with the ability to discover, monitor and manage the use of resources so they can operate autonomously. Two different approaches have been conceived to match the resource demands of grid applications to resource availability: Dynamic scheduling and adaptive scheduling. However, these two approaches fail to address at least one of three important issues: (i) the production of feasible schedules in a reasonable amount of time in relation to that required for the execution of an application; (ii) the impact of network link availability on the execution time of an application; and (iii) the necessity of migrating codes to decrease the execution time of an application. To overcome these challenges, this paper proposes a procedure for enabling grid applications, composed of various dependent tasks, to deal with the availability of hosts and links bandwidth. This procedure involves task scheduling, resource monitoring and task migration, with the goal of decreasing the execution time of grid applications. The procedure differs from other approaches in the literature because it constantly considers changes in resource availability, especially network bandwidth availability, to trigger task migration. The proposed procedure is illustrated via simulation using various scenarios involving fluctuation of resource availability. An additional contribution of this paper is the introduction of a set of schedulers offering solutions which differ in terms of both schedule length and computational complexity. The distinguishing aspect of this set of schedulers is the consideration of time requirements in the production of feasible schedules. Performance is then evaluated considering various network topologies and task dependencies.     

基于层次化的网格资源三层调度模型

在分析现有的资源调度方案及模型的基础上,提出了基于层次化的网格资源三层调度模型．它由主调度器、次级调度器和计算节点组成。主调度器根据任务的性质和需求,并参考下层次级调度器的执行情况,将部分任务分发到各次级调度器上,实现了主调度器与次级调度器之间的并行工作。基于该模型提出轮循任务分发策略。通过分析和模拟．该资源调度模型及任务分发策略在调度性能上明显优于集中式调度方案。     

网格环境下基于信任机制的资源调度研究

信任是网格资源调度中一个很重要的因素,也是影响网格计算有效性和性能的关键技术之一。将信任机制引入到网格资源调度中,提出了网格环境下的信任模型和基于信任机制的资源调度模型,在调度策略上对传统的Min-Min算法进行了改进,提出了基于信任机制的Trust-Min-Min算法。仿真结果表明,算法不仅可以缩短任务的总执行时间,而且可以有效地平衡负载,是网格环境下一种有效的资源调度方法。     

GAMESH: A grid architecture for scalable monitoring and enhanced dependable job scheduling

Grid computing is a largely adopted paradigm to federate geographically distributed data centers. Due to their size and complexity, grid systems are often affected by failures that may hinder the correct and timely execution of jobs, thus causing a non-negligible waste of computing resources. Despite the relevance of the problem, state-of-the-art management solutions for grid systems usually neglect the identification and handling of failures at runtime. Among the primary goals to be considered, we claim the need for novel approaches capable to achieve the objectives of scalable integration with efficient monitoring solutions and of fitting large and geographically distributed systems, where dynamic and configurable tradeoffs between overhead and targeted granularity are necessary. This paper proposes GAMESH, a Grid Architecture for scalable Monitoring and Enhanced dependable job ScHeduling. GAMESH is conceived as a completely distributed and highly efficient management infrastructure, concentrating on two crucial aspects for large-scale and multi-domain grid environments: (i) the scalable dissemination of monitoring data and (ii) the troubleshooting of job execution failures. GAMESH has been implemented and tested in a real deployment encompassing geographically distributed data centers across Europe. Experimental results show that GAMESH (i) enables the collection of measurements of both computing resources and conditions of task scheduling at geographically sparse sites, while imposing a limited overhead on the entire infrastructure, and (ii) provides a failure-aware scheduler able to improve the overall system performance, even in the presence of failures, by coordinating local job schedulers at multiple domains.     

一种基于人工免疫的新的网格资源调度算法

由于广域网性能的巨大提高和功能强大且价格低廉的计算机不断增多,网格计算以一种极具有前途和吸引力的新范式出现。网格计算是集成地理位置分布,异构,多领域资源的一种平台,它提供透明、安全、同等、高性能资源共享。要获取计算网格中潜在的能量,设计一种有效和高效的网格资源调度算法很重要。网格独特的特点使得网格环境下的资源调度是相当复杂的。本文将重点设计一种新的基于免疫算法的网格资源调度算法。     

Meld Scheduling: A Technique for Relaxing Scheduling Constraints

Meld scheduling melds the schedules of neighboring scheduling regions to respect latencies of operations issued in one region but completing after control transfers to the other. In contrast, conventional schedulers ignore latency constraints from other regions leading to potentially avoidable stalls in an interlocked (superscalar) machine or incorrect schedules for noninterlocked (VLIW) machines. Alternatively, schedulers that conservatively require all operations to complete before the branch takes effect produce inefficient schedules. In this paper, we present general data structures for maintaining latency constraint information at region boundaries. We present a meld scheduling algorithm for noninterlocked processors that generates latency constraints at the boundaries of scheduled regions and utilizes this information during the scheduling of other regions. We present a range of design options and describe the reasons behind our particular choices. We evaluate the performance of meld scheduling on a range of machine models on a set of SPEC92 and UNIX benchmarks.     

基于资源监控统计的云计算主动调度方法

现有的云计算资源调度策略没有考虑资源池的总体资源需求,造成了资源的损耗,影响虚拟机的正常运行．提出了一种基于资源监控统计的云计算主动调度方法,充分考虑资源池的总体资源需求情况,引入监控数据采集模块和监控数据分析模块,同时加入监控数据库作为系统的基础,定制一套适用于资源调度的虚拟机监控数据统计和分析的方法,使得基于监控统计数据的虚拟机调度方法可以轻松实现．     

Grid Resource Availability Prediction-Based Scheduling and Task Replication

The frequent and volatile unavailability of volunteer-based Grid computing resources challenges Grid schedulers to make effective job placements. The manner in which host resources become unavailable will have different effects on different jobs, depending on their runtime and their ability to be checkpointed or replicated. A multi-state availability model can help improve scheduling performance by capturing the various ways a resource may be available or unavailable to the Grid. This paper uses a multi-state model and analyzes a machine availability trace in terms of that model. Several prediction techniques then forecast resource transitions into the model’s states. We analyze the accuracy of our predictors, which outperform existing approaches. We also propose and study several classes of schedulers that utilize the predictions, and a method for combining scheduling factors. We characterize the inherent tradeoff between job makespan and the number of evictions due to failure, and demonstrate how our schedulers can navigate this tradeoff under various scenarios. Lastly, we propose job replication techniques, which our schedulers utilize to replicate those jobs that are most likely to fail. Our replication strategies outperform others, as measured by improved makespan and fewer redundant operations. In particular, we define a new metric for replication efficiency, and demonstrate that our multi-state availability predictor can provide information that allows our schedulers to be more efficient than others that blindly replicate all jobs or some static percentage of jobs.     

Probe scheduling for efficient detection of silent failures

Most discovery systems for silent failures work in two phases: a continuous monitoring phase that detects presence of failures through probe packets and a localization phase that pinpoints the faulty element(s). We focus on the monitoring phase, where the goal is to balance the probing overhead with the cost associated with longer failure detection times.We formulate a general model for the underlying fundamental subset-test scheduling problem. We unify the treatment of schedulers and cost objectives and make several contributions: We propose Memoryless schedules—a natural subclass of stochastic schedules which is simple and suitable for distributed deployment. We show that the optimal memoryless schedulers can be efficiently computed by convex programs (for SUM objectives, which minimize average detection time) or linear programs (for MAX objectives, which minimize worst-case detection time), and surprisingly perhaps, are guaranteed to have expected detection times that are not too far off the (NP hard) stochastic optima. We study Deterministic schedules, which provide a guaranteed bound on the maximum (rather than expected) cost of undetected faults, but like general stochastic schedules, are NP hard to optimize. We develop novel efficient deterministic schedulers with provable approximation ratios.Finally, we conduct an experimental study, simulating our schedulers on real networks topologies, demonstrates a significant performance gains of the new memoryless and deterministic schedulers over previous approaches.     

Controllable fair queuing for meeting performance goals

Computing and storage utilities must control resource usage to meet contractual performance targets for hosted customers under dynamic conditions, including flash crowds and unexpected resource failures. This paper explores properties of proportional share resource schedulers that are necessary for stability and responsiveness under feedback control. It shows that the fairness properties commonly defined for proportional share schedulers using Weighted Fair Queuing (WFQ) are not preserved across changes to the relative weights of competing request flows. As a result, conventional WFQ schedulers are not controllable by a resource controller that adapts by adjusting the weights. The paper defines controllable fairness properties, presents an algorithm to adjust any WFQ scheduler when the weights change, and proves that the algorithm results in controllable-fair schedulers.

The analytic results are confirmed by experimental evaluation using a three-tier Web service and a prototype controllable-fair scheduler called C-SFQ(D). C-SFQ(D) extends concurrency-controlled Start-time Fair Queuing (SFQ(D), which supports proportional sharing in multi-tasking computing resources. The prototype includes an adaptive control system that adjusts the flow weights in C-SFQ(D) to meet latency and throughput targets under a variety of conditions. The experimental results demonstrate the importance of controllable-fair scheduling for feedback control of computing utilities.     

网格计算中任务调度算法的研究和改进

任务调度一直是网格计算中的热点问题,任务调度的目的是最优地分配任务,实现最佳的调度策略,以高效地完成计算任务。在网格环境中,资源的合理有效利用是实现任务调度的关键问题之一。本文首先论述静态任务调度算法和动态任务算法的原理和优缺点等,然后结合Min-min、Max-min算法的优点设计一种新的调度算法SA-MM,根据资源的使用情况自适应调度相应算法进行任务到资源的映射。最后,用GridSim模拟工具对网格计算中Min-min、Max-min和SA-MM任务调度算法进行仿真实验,分析和比较它们的调度长度(MakeSpan)和资源负载情况等影响任务调度效率的指标。     

网格计算经济中的资源绑定

针对网格计算经济,提出资源绑定的模型,并结合任务对资源的需求和QoS要求,就绑定资源的描述、价格、资源绑定的优化,绑定流程等进行了深入研究.在此基础上,实现了基于绑定的调度算法.实验结果证明,资源绑定在任务完成时间和成本支出上取得了良好的效果,促进了资源调度,有利于提高整个网格资源管理系统的效率.     

计算网格中动态负载平衡的分布调度模式

网格计算下对资源进行有效的管理和调度可以提高系统的利用率.在对现有若干调度方法的研究和分析基础上,针对计算网格中的负载平衡问题,提出了一种分布式网格作业调度模型,并给出相关算法.算法通过建立主从模式的负载信息收集机制,提供给节点全局负载信息,加速重负载节点的负载转移速度.通过有效的负载平衡模式,解决资源调度中负载平衡及其可靠性问题.     

网格计算经济中资源的可靠性控制

针对网格资源的可靠性问题,提出了一个包括注册真实性,信任评价和调度契约构成的分层控制模型.针对该模型,提出了资源注册信息的验证方法和契约的模式.仿真实验表明,该模型在资源注册信息验证的基础上,优先调度了信息真实、运行可靠的资源,通过可靠性资源的选择和控制促进了计算经济的可靠性.     

融合遗传算法和蚁群算法动态网格任务调度算法研究

网格计算是当今计算机科学领域最新兴起的一项有很高学术价值和应用价值的研究课题。未来互联网的发展方向是将网络中众多闲置的计算资源、存储资源以及科学仪器等可用资源充分合理的加以利用。如何高效地使用网格资源,即网格调度问题也随之成为研究的重点,虽然在传统的分布式并行计算中有很多成熟的任务调度算法,但由于网格的新特性,使得必须研究新的算法来解决一些新出现的问题,如调度问题的NP安全性,调度算法的高效性,资源的异构性以及资源分配决策的并行性和分布性等。     

Task mapper and application-aware virtual machine scheduler oriented for parallel computing

We design a task mapper TPCM for assigning tasks to virtual machines, and an application-aware virtual machine scheduler TPCS oriented for parallel computing to achieve a high performance in virtual computing systems. To solve the problem of mapping tasks to virtual machines, a virtual machine mapping algorithm (VMMA) in TPCM is presented to achieve load balance in a cluster. Based on such mapping results, TPCS is constructed including three components: a middleware supporting an application-driven scheduling, a device driver in the guest OS kernel, and a virtual machine scheduling algorithm. These components are implemented in the user space, guest OS, and the CPU virtualization subsystem of the Xen hypervisor, respectively. In TPCS, the progress statuses of tasks are transmitted to the underlying kernel from the user space, thus enabling virtual machine scheduling policy to schedule based on the progress of tasks. This policy aims to exchange completion time of tasks for resource utilization. Experimental results show that TPCM can mine the parallelism among tasks to implement the mapping from tasks to virtual machines based on the relations among subtasks. The TPCS scheduler can complete the tasks in a shorter time than can Credit and other schedulers, because it uses task progress to ensure that the tasks in virtual machines complete simultaneously, thereby reducing the time spent in pending, synchronization, communication, and switching. Therefore, parallel tasks can collaborate with each other to achieve higher resource utilization and lower overheads. We conclude that the TPCS scheduler can overcome the shortcomings of present algorithms in perceiving the progress of tasks, making it better than schedulers currently used in parallel computing.     

网格计算资源调度策略的三级模式

网格计算下对资源进行有效的管理和调度是一个很复杂的问题。该文结合已有的调度策略,提出了一种新的改进调度策略的三级模式,并给出算法。