Integration of task abortion and security requirements in GA-based meta-heuristics for independent batch grid scheduling

Scheduling and resource allocation in large scale distributed environments, such as Computational Grids (CGs), arise new requirements and challenges not considered in traditional distributed computing environments. Among these new requirements, task abortion and security become needful criteria for Grid schedulers. The former arises due to the dynamics of the Grid systems, in which resources are expected to enter and leave the system in an unpredictable way. The latter requirement appears crucial in Grid systems mainly due to a multi-domain nature of CGs. The main aim of this paper is to develop a scheduling model that enables the aggregation of task abortion and security requirements as additional, together with makespan and flowtime, scheduling criteria into a cumulative objective function. We demonstrate the high effectiveness of genetic-based schedulers in finding near-optimal solutions for multi-objective scheduling problem, where all criteria (objectives) are simultaneously optimized. The proposed meta-heuristics are experimentally evaluated in static and dynamic Grid scenarios by using a Grid simulator. The obtained results show the fast reduction of the values of basic scheduler performance metrics, especially in the dynamic case, that confirms the usefulness of the proposed approach in real-life scenarios.     

QoS guided Min-Min heuristic for grid task scheduling

Task scheduling is an integrated component of computing.With the emergence of Grid and ubiquitous computing,new challenges appear in task scheduling based on properties such as security,quality of service,and lack of central control within distributed administrative domains.A Grid task scheduling framework must be able to deal with these issues.One of the goals of Grid task scheduling is to achivev high system throughput while matching applications with the available computing resources.This matching of resources in a non-deterministically shared heterogeneous environment leads to concerns over Quality of Service (QoS).In this paper a novel QoS guided task scheduling algorithm for Grid computing is introduced.The proposed novel algorithm is based on a general adaptive scheduling heuristics that includes QoS guidance.The algorithm is evaluated within a simulated Grid environment.The experimental results show that the nwe QoS guided Min-Min heuristic can lead to significant performance gain for a variety of applications.The approach is compared with others based on the quality of the prediction formulated by inaccurate information.     

Reputation-based dependable scheduling of workflow applications in Peer-to-Peer Grids

Grids facilitate creation of wide-area collaborative environment for sharing computing or storage resources and various applications. Inter-connecting distributed Grid sites through peer-to-peer routing and information dissemination structure (also known as Peer-to-Peer Grids) is essential to avoid the problems of scheduling efficiency bottleneck and single point of failure in the centralized or hierarchical scheduling approaches. On the other hand, uncertainty and unreliability are facts in distributed infrastructures such as Peer-to-Peer Grids, which are triggered by multiple factors including scale, dynamism, failures, and incomplete global knowledge.In this paper, a reputation-based Grid workflow scheduling technique is proposed to counter the effect of inherent unreliability and temporal characteristics of computing resources in large scale, decentralized Peer-to-Peer Grid environments. The proposed approach builds upon structured peer-to-peer indexing and networking techniques to create a scalable wide-area overlay of Grid sites for supporting dependable scheduling of applications. The scheduling algorithm considers reliability of a Grid resource as a statistical property, which is globally computed in the decentralized Grid overlay based on dynamic feedbacks or reputation scores assigned by individual service consumers mediated via Grid resource brokers. The proposed algorithm dynamically adapts to changing resource conditions and offers significant performance gains as compared to traditional approaches in the event of unsuccessful job execution or resource failure. The results evaluated through an extensive trace driven simulation show that our scheduling technique can reduce the makespan up to 50% and successfully isolate the failure-prone resources from the system.     

基于人工鱼群算法的网格任务调度策略

网格任务调度是一个NP-hard问题,而且是并行与分布式计算中一个必不可少的组成部分,特别是在网格计算环境中任务调度更加复杂。提出了一种基于人工鱼群算法的网络任务调度策略,通过鱼群的觅食、聚群、追尾等方式,实现网格任务的有效调度。     

A Model of Grid Service Capacity

Computational grids （CGs） are large scale networks of geographically distributed aggregates of resource clusters that may be contributed by distinct organizations for the provision of computing services such as model simulation, compute cycle and data mining. Traditionally, the decision-making strategies underlying the grid management mechanisms rely on the physical view of the grid resource model. This entails the need for complex multi-dimensional search strategies and a considerable level of resource state information exchange between the grid management domains. In this paper we argue that with the adoption of service oriented grid architectures, a logical service-oriented view of the resource model provides a more appropriate level of abstraction to express the grid capacity to handle incoming service requests. In this respect, we propose a quantification model of the aggregated service capacity of the hosting environment that is updated based on the monitored state of the various environmental resources required by the hosted services. A comparative experimental validation of the model shows its performance towards enabling an adequate exploitation of provisioned services.     

Bi-level fuzzy based advanced reservation of Cloud workflow applications on distributed Grid resources

The increasing demand on execution of large-scale Cloud workflow applications which need a robust and elastic computing infrastructure usually lead to the use of high-performance Grid computing clusters. As the owners of Cloud applications expect to fulfill the requested Quality of Services (QoS) by the Grid environment, an adaptive scheduling mechanism is needed which enables to distribute a large number of related tasks with different computational and communication demands on multi-cluster Grid computing environments. Addressing the problem of scheduling large-scale Cloud workflow applications onto multi-cluster Grid environment regarding the QoS constraints declared by application’s owner is the main contribution of this paper. Heterogeneity of resource types (service type) is one of the most important issues which significantly affect workflow scheduling in Grid environment. On the other hand, a Cloud application workflow is usually consisting of different tasks with the need for different resource types to complete which we call it heterogeneity in workflow. The main idea which forms the soul of all the algorithms and techniques introduced in this paper is to match the heterogeneity in Cloud application’s workflow to the heterogeneity in Grid clusters. To obtain this objective a new bi-level advanced reservation strategy is introduced, which is based upon the idea of first performing global scheduling and then conducting local scheduling. Global-scheduling is responsible to dynamically partition the received DAG into multiple sub-workflows that is realized by two collaborating algorithms: (1) The Critical Path Extraction algorithm (CPE) which proposes a new dynamic task overall critically value strategy based on DAG’s specification and requested resource type QoS status to determine the criticality of each task; and (2) The DAG Partitioning algorithm (DAGP) which introduces a novel dynamic score-based approach to extract sub-workflows based on critical paths by using a new Fuzzy Qualitative Value Calculation System to evaluate the environment. Local-scheduling is responsible for scheduling tasks on suitable resources by utilizing a new Multi-Criteria Advance Reservation algorithm (MCAR) which simultaneously meets high reliability and QoS expectations for scheduling distributed Cloud-base applications. We used the simulation to evaluate the performance of the proposed mechanism in comparison with four well-known approaches. The results show that the proposed algorithm outperforms other approaches in different QoS related terms.     

Network and data location aware approach for simultaneous job scheduling and data replication in large-scale data grid environments

Data Grid integrates graphically distributed resources for solving data intensive scientific applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. Scheduling is a traditional problem in parallel and distributed system. However, due to special issues and goals of Grid, traditional approach is not effective in this environment any more. Therefore, it is necessary to propose methods specialized for this kind of parallel and distributed system. Another solution is to use a data replication strategy to create multiple copies of files and store them in convenient locations to shorten file access times. To utilize the above two concepts, in this paper we develop a job scheduling policy, called hierarchical job scheduling strategy (HJSS), and a dynamic data replication strategy, called advanced dynamic hierarchical replication strategy (ADHRS), to improve the data access efficiencies in a hierarchical Data Grid. HJSS uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers network characteristics, number of jobs waiting in queue, file locations, and disk read speed of storage drive at data sources. Moreover, due to the limited storage capacity, a good replica replacement algorithm is needed. We present a novel replacement strategy which deletes files in two steps when free space is not enough for the new replica: first, it deletes those files with minimum time for transferring. Second, if space is still insufficient then it considers the last time the replica was requested, number of access, size of replica and file transfer time. The simulation results show that our proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, number of intercommunications, number of replications, hit ratio, computing resource usage and storage usage.     

基于自适应模糊-PID反馈模型的网格调度技术

服务调度技术是计算网格任务管理系统中的核心问题.为适应网格服务部署的需要,提出了一种基于自适应"模糊-PID"反馈控制模型的Agent技术,该技术融合了模糊理论与PID(Proportionment-Integral-Differential coefficient)技术的优点.实验证明本技术能解决网格服务部署中的动态性和不确定性,并充分发挥网格服务的虚拟执行功能,可在网络带宽效率、延时和可靠性等方面做出更好的权衡.     

A computational economy for grid computing and its implementation in the Nimrod-G resource broker

Computational grids that couple geographically distributed resources such as PCs, workstations, clusters, and scientific instruments, have emerged as a next generation computing platform for solving large-scale problems in science, engineering, and commerce. However, application development, resource management, and scheduling in these environments continue to be a complex undertaking. In this article, we discuss our efforts in developing a resource management system for scheduling computations on resources distributed across the world with varying quality of service (QoS). Our service-oriented grid computing system called Nimrod-G manages all operations associated with remote execution including resource discovery, trading, scheduling based on economic principles and a user-defined QoS requirement. The Nimrod-G resource broker is implemented by leveraging existing technologies such as Globus, and provides new services that are essential for constructing industrial-strength grids. We present the results of experiments using the Nimrod-G resource broker for scheduling parametric computations on the World Wide Grid (WWG) resources that span five continents.     

基于Web服务的网格体系结构及其支撑环境研究

网格技术是当前网络计算的前沿领域,基于Web服务技术构建网格系统有助于提高网格系统的可扩展性和互操作能力,是这一领域中的最新热点.但现有的工作尚未明确界定基于Web服务的网格的功能模型和实现机制.首先讨论了网格功能模型,基于OGSA(open grid service architecture)框架提出了基于Web服务的网格层次体系结构,并将Web服务工作流引入到网格任务描述中,给出一种Web服务与网格技术相融合的机制,介绍了基于Web服务的网格支撑环境WebSASE4G的总体结构和设计原理,为基于Web     

A New Grid Scheduler with Failure Recovery and Rescheduling Mechanisms: Discussion and Analysis

Computational Grids (CGs) have become an appealing research area. They suggest a suitable environment for developing large scale parallel applications. CGs integrate a huge mount of distributed heterogeneous resources for constituting a powerful virtual supercomputer. Scheduling is the most important issue for enhancing the performance of CGs. Various strategies have been introduced, including static and dynamic behaviors. The former maps tasks to resources at submission time, while the latter operates at run time. While static scheduling is unsuitable for the dynamic Grid environment, scheduling in CGs is still more complex than the proposed dynamic solutions. This paper introduces a decentralized Adaptive Grid Scheduler (AGS) based on a novel rescheduling mechanism. AGS has several salient properties as it is; hybrid, adaptive, decentralized, and efficient. Also, AGS is a robust mechanism as it has the ability to; (i) detect resource failures, (ii) continue its functionality in spite of the failure existence, then (iii) recover back. Moreover, it integrates both static and dynamic scheduling behaviors. An initial static scheduling map is proposed for an input Direct Acyclic Graph (DAG). However, DAG tasks may be rescheduled if the performance of the allocated resources changes in away that may affect the tasks’ response time. AGS overcomes drawbacks of traditional schedulers by utilizing the mobile agent unique features to enhance the resource discovery and monitoring processes. Experimental results have shown that AGS outperforms traditional Grid schedulers as it introduces a better scheduling efficiency.     

Static strategy and dynamic adjustment: An effective method for Grid task scheduling

Task scheduling is the key technology in Grid computing. Hierarchical organization is suitable for the computational Grid because of the dynamic, heterogeneous and autonomous nature of the Grid. Although a number of Grid systems adopt this organization, few of them has dealt with task scheduling for the hierarchical architecture. In this paper, we present an effective method, fully taking into account both historical Grid trade data and dynamic variation of the Grid market to improve the task scheduling for a hierarchical Grid market. The main idea of the proposed method is a combination of an off-line static strategy using time series prediction and an on-line dynamic adjustment using reinforcement learning. The superiority of this new scheduling algorithm, in improving the inquiry efficiency for resource consumers, getting better load balancing of the whole hierarchical Grid market, and achieving higher success rate of the Grid service request, is demonstrated by simulation experiments.     

Enhanced Dynamic Hierarchical Replication and Weighted Scheduling Strategy in Data Grid

The Data Grid provides massive aggregated computing resources and distributed storage space to deal with data-intensive applications. Due to the limitation of available resources in the grid as well as production of large volumes of data, efficient use of the Grid resources becomes an important challenge. Data replication is a key optimization technique for reducing access latency and managing large data by storing data in a wise manner. Effective scheduling in the Grid can reduce the amount of data transferred among nodes by submitting a job to a node where most of the requested data files are available. In this paper two strategies are proposed, first a novel job scheduling strategy called Weighted Scheduling Strategy (WSS) that uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers the number of jobs waiting in a queue, the location of the required data for the job and the computing capacity of the sites Second, a dynamic data replication strategy, called Enhanced Dynamic Hierarchical Replication (EDHR) that improves file access time. This strategy is an enhanced version of the Dynamic Hierarchical Replication strategy. It uses an economic model for file deletion when there is not enough space for the replica. The economic model is based on the future value of a data file. Best replica placement plays an important role for obtaining maximum benefit from replication as well as reducing storage cost and mean job execution time. So, it is considered in this paper. The proposed strategies are implemented by OptorSim, the European Data Grid simulator. Experiment results show that the proposed strategies achieve better performance by minimizing the data access time and avoiding unnecessary replication.     

Combination of data replication and scheduling algorithm for improving data availability in Data Grids

Data Grid is a geographically distributed environment that deals with large-scale data-intensive applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. Data replication is another key optimization technique for reducing access latency and managing large data by storing data in a wisely manner. In this paper two algorithms are proposed, first a novel job scheduling algorithm called Combined Scheduling Strategy (CSS) that uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers the number of jobs waiting in queue, the location of required data for the job and the computing capacity of sites. Second a dynamic data replication strategy, called the Modified Dynamic Hierarchical Replication Algorithm (MDHRA) that improves file access time. This strategy is an enhanced version of Dynamic Hierarchical Replication (DHR) strategy. Data replication should be used wisely because the storage capacity of each Grid site is limited. Thus, it is important to design an effective strategy for the replication replacement. MDHRA replaces replicas based on the last time the replica was requested, number of access, and size of replica. It selects the best replica location from among the many replicas based on response time that can be determined by considering the data transfer time, the storage access latency, the replica requests that waiting in the storage queue and the distance between nodes. The simulation results demonstrate the proposed replication and scheduling strategies give better performance compared to the other algorithms.     

Midland: a service-oriented cluster computing infrastructure

Midland is a service-oriented software infrastructure that enables the clustering of arbitrarily large collections of computing resources. The resulting clusters may be integrated to form an open, dynamically configurable computational grid system where each cluster defines a self-reliant and independent management domain. Web Services make up the primary integration mechanism both at the cluster and grid levels, respectively. This is complemented by a light XML based messaging protocol exclusively used for cluster bound interactions. The paper describes Midland’s architecture, and the service-oriented approach taken to develop the associated resource management mechanisms. It also includes an exposition of the model of service capacity which is one of the enablers of the service-centric strategy underlying the cluster management mechanisms. The operational performance of Midland is illustrated experimentally in the context of a Grid test-bed comprising three clusters. The experimental results highlight the performance of the model of service capacity as well as some aspects of Midland operational performance.     

An improved load-balancing mechanism based on deadline failure recovery on GridSim

Grid computing has emerged a new field, distinguished from conventional distributed computing. It focuses on large-scale resource sharing, innovative applications and in some cases, high performance orientation. The Grid serves as a comprehensive and complete system for organizations by which the maximum utilization of resources is achieved. The load balancing is a process which involves the resource management and an effective load distribution among the resources. Therefore, it is considered to be very important in Grid systems. For a Grid, a dynamic, distributed load balancing scheme provides deadline control for tasks. Due to the condition of deadline failure, developing, deploying, and executing long running applications over the grid remains a challenge. So, deadline failure recovery is an essential factor for Grid computing. In this paper, we propose a dynamic distributed load-balancing technique called “Enhanced GridSim with Load balancing based on Deadline Failure Recovery” (EGDFR) for computational Grids with heterogeneous resources. The proposed algorithm EGDFR is an improved version of the existing EGDC in which we perform load balancing by providing a scheduling system which includes the mechanism of recovery from deadline failure of the Gridlets. Extensive simulation experiments are conducted to quantify the performance of the proposed load-balancing strategy on the GridSim platform. Experiments have shown that the proposed system can considerably improve Grid performance in terms of total execution time, percentage gain in execution time, average response time, resubmitted time and throughput. The proposed load-balancing technique gives 7 % better performance than EGDC in case of constant number of resources, whereas in case of constant number of Gridlets, it gives 11 % better performance than EGDC.     

Flexible Grid service management through resource partitioning

In this paper, a distributed and scalable Grid service management architecture is presented. The proposed architecture is capable of monitoring task submission behaviour and deriving Grid service class characteristics, for use in performing automated computational, storage and network resource-to-service partitioning. This partitioning of Grid resources amongst service classes (each service class is assigned exclusive usage of a distinct subset of the available Grid resources), along with the dynamic deployment of Grid management components dedicated and tuned to the requirements of a particular service class introduces the concept of Virtual Private Grids. We present two distinct algorithmic approaches for the resource partitioning problem, the first based on Divisible Load Theory (DLT) and the second built on Genetic Algorithms (GA). The advantages and drawbacks of each approach are discussed and their performance is evaluated on a sample Grid topology using NSGrid, an ns-2 based Grid simulator. Results show that the use of this Service Management Architecture in combination with the proposed algorithms improves computational and network resource efficiency, simplifies schedule making decisions, reduces the overall complexity of managing the Grid system, and at the same time improves Grid QoS support (with regard to job response times) by automatically assigning Grid resources to the different service classes prior to scheduling.     

基于OGSA的网格服务管理模型研究

网格服务管理是网格计算的核心问题。通过对于目前网格服务管理体系架构的三种模型进行分析和比较,基于开放式服务体系架构(OGSA),探讨了网格服务管理系统的功能需求,进而设计了一种层次化的网格服务管理模型HGSM,描述了模型的工作流程。将网格服务管理分为任务分解、静态调度和动态调度三种层次,讨论了HGSM的各个层次的相关功能模块,以有向无环图和高级随机Petri网分别对于任务分解和服务调度提出了相关算法,算法中的可实施谓词、随机开关、实施速率等描述可以直接在SPN求解软件的编程中实现,从而为构造一种层次化的网格服务管理模型提供一个可实现的有效途径。     

An enhanced load balancing mechanism based on deadline control on GridSim

A Grid is a network of computational resources that may potentially span many continents. Load balancing in a Grid is a hot research issue which affects every aspect of the Grid, including service selection and task execution. Thus, it is necessary and significant to solve the load balancing problem in a Grid. In this paper, we propose a dynamic, distributed load balancing scheme for a Grid which provides deadline control for tasks. In our scenario, first, resources check their state and make a request to the Grid Broker according to the change of load state. Then, the Grid Broker assigns Gridlets between resources and scheduling for load balancing under the deadline request. We apply our load balancing strategy into a popular Grid simulation platform GridSim. Experimental results prove that our proposed load balancing mechanism can (1) reduce the makespan, (2) improve the finished rate of the Gridlet, and (3) reduce the resubmitted time.