Multi-objective list scheduling of workflow applications in distributed computing infrastructures

Executing large-scale applications in distributed computing infrastructures (DCI), for example modern Cloud environments, involves optimization of several conflicting objectives such as makespan, reliability, energy, or economic cost. Despite this trend, scheduling in heterogeneous DCIs has been traditionally approached as a single or bi-criteria optimization problem. In this paper, we propose a generic multi-objective optimization framework supported by a list scheduling heuristic for scientific workflows in heterogeneous DCIs. The algorithm approximates the optimal solution by considering user-specified constraints on objectives in a dual strategy: maximizing the distance to the user’s constraints for dominant solutions and minimizing it otherwise. We instantiate the framework and algorithm for a four-objective case study comprising makespan, economic cost, energy consumption, and reliability as optimization goals. We implemented our method as part of the ASKALON environment (Fahringer et al., 2007) for Grid and Cloud computing and demonstrate through extensive real and synthetic simulation experiments that our algorithm outperforms related bi-criteria heuristics while meeting the user constraints most of the time.     

Resource-aware hybrid scheduling algorithm in heterogeneous distributed computing

Today, almost everyone is connected to the Internet and uses different Cloud solutions to store, deliver and process data. Cloud computing assembles large networks of virtualized services such as hardware and software resources. The new era in which ICT penetrated almost all domains (healthcare, aged-care, social assistance, surveillance, education, etc.) creates the need of new multimedia content-driven applications. These applications generate huge amount of data, require gathering, processing and then aggregation in a fault-tolerant, reliable and secure heterogeneous distributed system created by a mixture of Cloud systems (public/private), mobile devices networks, desktop-based clusters, etc. In this context dynamic resource provisioning for Big Data application scheduling became a challenge in modern systems. We proposed a resource-aware hybrid scheduling algorithm for different types of application: batch jobs and workflows. The proposed algorithm considers hierarchical clustering of the available resources into groups in the allocation phase. Task execution is performed in two phases: in the first, tasks are assigned to groups of resources and in the second phase, a classical scheduling algorithm is used for each group of resources. The proposed algorithm is suitable for Heterogeneous Distributed Computing, especially for modern High-Performance Computing (HPC) systems in which applications are modeled with various requirements (both IO and computational intensive), with accent on data from multimedia applications. We evaluate their performance in a realistic setting of CloudSim tool with respect to load-balancing, cost savings, dependency assurance for workflows and computational efficiency, and investigate the computing methods of these performance metrics at runtime.     

分布式并行计算环境下混合遗传算法的研究

为提高混合遗传算法的计算效率和求解质量,提出一个并行混合遗传算法框架。该框架主要由遗传算法、小生境操作和单纯形3部分组成,遗传算法和小生境操作采用串行执行方式,单纯形采用分布式并行执行方式。分布式并行计算环境由4台计算机通过交换机连接构成,并设计了一个动态任务调度方案。一个典型工程算例验证了新算法的有效性,并且在分布式并行环境下取得了较好的加速比和并行效率。     

Large-scale climate simulations harnessing clusters,grid and cloud infrastructures

The current availability of a variety of computing infrastructures including HPC, Grid and Cloud resources provides great computer power for many fields of science, but their common profit to accomplish large scientific experiments is still a challenge. In this work, we use the paradigm of climate modeling to present the key problems found by standard applications to be run in hybrid distributed computing infrastructures and propose a framework to allow a climate model to take advantage of these resources in a transparent and user-friendly way. Furthermore, an implementation of this framework, using the Weather Research and Forecasting system, is presented as a working example. In order to illustrate the usefulness of this framework, a realistic climate experiment leveraging Cluster, Grid and Cloud resources simultaneously has been performed. This test experiment saved more than 75% of the execution time, compared to local resources. The framework and tools introduced in this work can be easily ported to other models and are probably useful in other scientific areas employing data- and CPU-intensive applications.     

An efficient list-based task scheduling algorithm for heterogeneous distributed computing environment

The task scheduling in heterogeneous distributed computing systems plays a crucial role in reducing the makespan and maximizing resource utilization. The diverse nature of the devices in heterogeneous distributed computing systems intensifies the complexity of scheduling the tasks. To overcome this problem, a new list-based static task scheduling algorithm namely Deadline-Aware-Longest-Path-of-all-Predecessors (DA-LPP) is being proposed in this article. In the prioritization phase of the DA-LPP algorithm, the path length of the current task from all its predecessors at each level is computed and among them, the longest path length value is assigned as the rank of the task. This strategy emphasizes the tasks in the critical path. This well-optimized prioritization phase leads to an observable minimization in the makespan of the applications. In the processor selection phase, the DA-LPP algorithm implements the improved insertion-based policy which effectively utilizes the unoccupied leftover free time slots of the processors which improve resource utilization, further least computation cost allocation approach is followed to minimize the overall computation cost of the processors and parental prioritization policy is incorporated to further reduce the scheduling length. To demonstrate the robustness of the proposed algorithm, a synthetic graph generator is used in this experiment to generate a huge variety of graphs. Apart from the synthetic graphs, real-world application graphs like Montage, LIGO, Cybershake, and Epigenomic are also considered to grade the performance of the DA-LPP algorithm. Experimental results of the DA-LPP algorithm show improvement in performance in terms of scheduling length ratio, makespan reduction rate , and resource reduction rate when compared with other algorithms like DQWS, DUCO, DCO and EPRD. The results reveal that for 1000 task set with deadline equals to two times of the critical path, the scheduling length ratio of the DA-LPP algorithm is better than DQWS by 35%, DUCO by 23%, DCO by 26 %, and EPRD by 17%.     

基于多班教学优化的多目标分布式混合流水车间调度

单工厂环境下的混合流水车间调度问题已受到广泛关注,而多工厂环境下的分布式混合流水车间调度问题(distributed hybrid flow shop scheduling problem,DHFSP)研究进展则较小.针对考虑顺序相关准备时间的DHFSP,提出一种多班教学优化(multi-class teaching-...     

分布式空间信息的对等协同混合发现模型*

海量空间信息的处理需要分布式协同工作的GIS平台的支持,为了解决经典的分散式结构化的分布式哈希表逻辑网络结构增加的延时和在构建哈希表的过程中逻辑覆盖网络往往和物理网络不一致的问题,提出一种分布式空间信息的对等协同混合发现模型。基于空间资源发现代理节点和普通邻居节点,该模型实现了集中式的全局空间资源发现模型与分散式结构化的分布式哈希表模型之间的自动切换,能够自适应地调整空间资源的逻辑网络结构以提供更好的性能。基于节点交换机制,设计了构建路由表和降低延时的算法,通过发现有利于覆盖网络和物理网络匹配的节点交换来     

一种云环境下基于混合型BBO的任务调度算法

任务调度在云计算中占有重要地位,是影响云计算性能的关键因素,被证明是NP问题。启发式算法是解决该问题的最有效方法之一,针对近年来出现的一种新型启发式算法--BBO算法展开研究,由于BBO算法在求解过程中收敛速度较慢,因此结合粒子群算法提出了一种新型算法的任务调度算法--HMBBO,并结合Cloudsim云仿真平台,进行了以Makespan为目标函数的比对实验。实验结果表明,与几种经典的启发式算法相比,HMBBO算法具有寻优能力强、收敛速度快、求解质量高的特点,为解决云计算环境中任务调度问题提供了一种新思路。     

Grids and Grid technologies for wide‐area distributed computing

The last decade has seen a substantial increase in commodity computer and network performance, mainly as a result of faster hardware and more sophisticated software. Nevertheless, there are still problems, in the fields of science, engineering, and business, which cannot be effectively dealt with using the current generation of supercomputers. In fact, due to their size and complexity, these problems are often very numerically and/or data intensive and consequently require a variety ofheterogeneous resources that are not available on a single machine. A number of teams have conducted experimental studies on the cooperative use of geographically distributed resources unified to act as a single powerful computer. This new approach is known by several names, such as metacomputing, scalable computing, global computing, Internet computing, and more recently peer‐to‐peer or Grid computing. The early efforts in Grid computing started as a project to link supercomputing sites, but have now grown far beyond their original intent. In fact, many applications can benefit from the Grid infrastructure, including collaborative engineering, data exploration, high‐throughput computing, and of course distributed supercomputing. Moreover, due to the rapid growth of the Internet and Web, there has been a rising interest in Web‐based distributed computing, and many projects have been started and aim to exploit the Web as an infrastructure for running coarse‐grained distributed and parallel applications. In this context, the Web has the capability to be a platform for parallel and collaborative work as well as a key technology to create a pervasive and ubiquitous Grid‐based infrastructure. This paper aims to present the state‐of‐the‐art of Grid computing and attempts to survey the major international efforts in developing this emerging technology. Copyright © 2002 John Wiley & Sons, Ltd.     

A taxonomy and survey of grid resource management systems for distributed computing

The resource management system is the central component of distributed network computing systems. There have been many projects focused on network computing that have designed and implemented resource management systems with a variety of architectures and services. In this paper, an abstract model and a comprehensive taxonomy for describing resource management architectures is developed. The taxonomy is used to identify approaches followed in the implementation of existing resource management systems for very large‐scale network computing systems known as Grids. The taxonomy and the survey results are used to identify architectural approaches and issues that have not been fully explored in the research. Copyright © 2001 John Wiley & Sons, Ltd.     

基于结构化对等网络的高效分布式任务调度

提出与描述了一个基于结构化对等网络的高效分布式任务调度策略HDTS(High-Efficient Distributed Task Scheduling)。HDTS建立在结构化对等网络的应用层覆盖网络上,保证了系统的非集中性、可扩展性、自组织性,以及规模大的优点。HDTS把基于Chord风格的对等网络协议和容错及高效的多播调度策略结合起来,允许分布式计算的各种大量的子任务在对等网络的节点上高效的调度、分配、执行。除了支持主-从风格的并行计算外,系统允许具有数据依赖的分布式旅行商算法正确的执行,使系统具有通用性和开放性。测试结果表明:HDTS具有正确性与高效性,可以作为对等网络上计算资源高效共享与聚集的可行方案。     

Object‐oriented distributed computing based on remote class reference

Java RMI, Jini and CORBA provide effective mechanisms for implementing a distributed computing system. Recently many numeral libraries have been developed that take advantage of Java as an object‐oriented and portable language. The widely‐used client‐server method limits the extent to which the benefits of the object‐oriented approach can be exploited because of the difficulties arising when a remote object is the argument or return value of a remote or local method. In this paper this problem is solved by introducing a data object that stores the data structure of the remote object and related access methods. By using this data object, the client can easily instantiate a remote object, and use it as the argument or return value of either a local or remote method. Copyright © 2003 John Wiley & Sons, Ltd.     

光网络环境中计算任务与光路的联合调度算法

针对光网络环境下分布式计算系统的资源调度问题,提出了一种光网络计算任务和光路联合调度方案。该方案将光网络的特性加入到传统调度模型中,提出了计算任务与光路通信的联合调度模型,设计求解联合调度模型的扩展型列表算法。仿真实验验证了联合调度的有效性。     

Multihybrid job scheduling for fault-tolerant distributed computing in policy-constrained resource networks

Unpredictable fluctuations in resource availability often lead to rescheduling decisions that sacrifice a success rate of job completion in batch job scheduling. To overcome this limitation, we consider the problem of assigning a set of sequential batch jobs with demands to a set of resources with constraints such as heterogeneous rescheduling policies and capabilities. The ultimate goal is to find an optimal allocation such that performance benefits in terms of makespan and utilization are maximized according to the principle of Pareto optimality, while maintaining the job failure rate close to an acceptably low bound. To this end, we formulate a multihybrid policy decision problem (MPDP) on the primary-backup fault tolerance model and theoretically show its NP-completeness. The main contribution is to prove that our multihybrid job scheduling (MJS) scheme confidently guarantees the fault-tolerant performance by adaptively combining jobs and resources with different rescheduling policies in MPDP. Furthermore, we demonstrate that the proposed MJS scheme outperforms the five rescheduling heuristics in solution quality, searching adaptability and time efficiency by conducting a set of extensive simulations under various scheduling conditions.     

A modified genetic algorithm for distributed scheduling problems

Genetic algorithms (GAs) have been widely applied to the scheduling and sequencing problems due to its applicability to different domains and the capability in obtaining near-optimal results. Many investigated GAs are mainly concentrated on the traditional single factory or single job-shop scheduling problems. However, with the increasing popularity of distributed, or globalized production, the previously used GAs are required to be further explored in order to deal with the newly emerged distributed scheduling problems. In this paper, a modified GA is presented, which is capable of solving traditional scheduling problems as well as distributed scheduling problems. Various scheduling objectives can be achieved including minimizing makespan, cost and weighted multiple criteria. The proposed algorithm has been evaluated with satisfactory results through several classical scheduling benchmarks. Furthermore, the capability of the modified GA was also tested for handling the distributed scheduling problems.     

Self-healing of workflow activity incidents on distributed computing infrastructures

Distributed computing infrastructures are commonly used through scientific gateways, but operating these gateways requires important human intervention to handle operational incidents. This paper presents a self-healing process that quantifies incident degrees of workflow activities from metrics measuring long-tail effect, application efficiency, data transfer issues, and site-specific problems. These metrics are simple enough to be computed online and they make little assumptions on the application or resource characteristics. From their degree, incidents are classified in levels and associated to sets of healing actions that are selected based on association rules modeling correlations between incident levels. We specifically study the long-tail effect issue, and propose a new algorithm to control task replication. The healing process is parametrized on real application traces acquired in production on the European Grid Infrastructure. Experimental results obtained in the Virtual Imaging Platform show that the proposed method speeds up execution up to a factor of 4, consumes up to 26% less resource time than a control execution and properly detects unrecoverable errors.     

面向分布式实时系统的安全驱动调度算法研究

针对现有实时调度算法无法适应动态安全需求的问题,构建了一种安全驱动调度模型,该模型从系统安全级别、系统安全服务和任务安全策略三个方面描述了实时系统的动态安全需求,并设计了一种基于安全驱动的实时任务调度器框架。以该模型和框架为基础,提出了一种安全驱动调度算法(Security Driven Scheduling Algorithm,SDSA)。从全局角度对新到达任务进行可调度性检查,并将可调度任务分配到合适的处理机上运行。按照系统安全级别来动态调整已分配到各处理机上实时任务的安全策略,使其达到安全性和可调度性的最优平衡。采用优先级抢占式策略对各实时任务进行调度。仿真结果表明,SDSA算法与其他同类算法相比,在系统动态安全需求的适应性、关键任务的可调度性以及安全防危能力等方面具有较好的表现。     

计算网格中动态负载平衡的分布调度模式

网格计算下对资源进行有效的管理和调度可以提高系统的利用率.在对现有若干调度方法的研究和分析基础上,针对计算网格中的负载平衡问题,提出了一种分布式网格作业调度模型,并给出相关算法.算法通过建立主从模式的负载信息收集机制,提供给节点全局负载信息,加速重负载节点的负载转移速度.通过有效的负载平衡模式,解决资源调度中负载平衡及其可靠性问题.     

A parallel bi-objective hybrid metaheuristic for energy-aware scheduling for cloud computing systems

In this paper, we investigate the problem of scheduling precedence-constrained parallel applications on heterogeneous computing systems (HCSs) like cloud computing infrastructures. This kind of application was studied and used in many research works. Most of these works propose algorithms to minimize the completion time (makespan) without paying much attention to energy consumption.We propose a new parallel bi-objective hybrid genetic algorithm that takes into account, not only makespan, but also energy consumption. We particularly focus on the island parallel model and the multi-start parallel model. Our new method is based on dynamic voltage scaling (DVS) to minimize energy consumption.In terms of energy consumption, the obtained results show that our approach outperforms previous scheduling methods by a significant margin. In terms of completion time, the obtained schedules are also shorter than those of other algorithms. Furthermore, our study demonstrates the potential of DVS.