Artificial Neural Network support to monitoring of the evolutionary driven security aware scheduling in computational distributed environments

Monitoring of the system performance in highly distributed computing environments is a wide research area. In cloud and grid computing, it is usually restricted to the utilization and reliability of the resources. However, in today’s Computational Grids (CGs) and Clouds (CCs), the end users may define the special personal requirements and preferences in the resource and service selection, service functionality and data access. Such requirements may refer to the special individual security conditions for the protection of the data and application codes. Therefore, solving the scheduling problems in modern distributed environments remains still challenging for most of the well known schedulers, and the general functionality of the monitoring systems must be improved to make them efficient as schedulers supporting modules.In this paper, we define a novel model of security-driven grid schedulers supported by an Artificial Neural Network (ANN). ANN module monitors the schedule executions and learns about secure task–machine mappings from the observed machine failures. Then, the metaheuristic grid schedulers (in our case—genetic-based schedulers) are supported by the ANN module through the integration of the sub-optimal schedules generated by the neural network, with the genetic populations of the schedules.The influence of the ANN support on the general schedulers’ performance is examined in the experiments conducted for four types of the grid networks (small, medium, large and very large grids), two security scheduling modes—risky and secure scenarios, and six genetic-based grid schedulers. The generated empirical results show the high effectiveness of such monitoring support in reducing the values of the major scheduling criteria (makespan and flowtime), the run times of the schedulers and the grid resource failures.     

An efficient grid scheduling strategy for data parallel applications

Scheduling large-scale application in heterogeneous grid systems is a fundamental NP-complete problem that is critical to obtain good performance and execution cost. To achieve high performance in a grid system it requires effective task partitioning, resource management and load balancing. The heterogeneous and dynamic nature of a grid, as well as the diverse demands of applications running on the grid, makes grid scheduling a major task. Existing schedulers in wide-area heterogeneous systems require a large amount of information about the application and the grid environment to produce reasonable schedules. However, this required information may not be available, may be too expensive to collect, or may increase the runtime overhead of the scheduler such that the scheduler is rendered ineffective. We believe that no one scheduler is appropriate for all grid systems and applications. This is because while data parallel applications in which further data partitioning is possible can be further improved by efficient management of resources, smart selection of resources and load balancing can be possible, in functional/not-dividable-task parallel applications such partitioning is either not possible or difficult or expensive in term of performance. In this paper, we propose a scheduler for data parallel applications (SDPA) which offers an efficient task partitioning and load balancing strategy for data parallel applications in grid environment. The proposed SDPA offers two major features: maintaining job priority even if insufficient number of free resources is available and pre-task assignment to cut the idle time of nodes. The SDPA selects nodes smartly according to the nature of task and the nodes’ resources availability. Simulation results conducted reveal that SDPA achieves performance improvement over reported strategies in the reviewed literature in terms of execution time, throughput and waiting time.     

基于层次化的网格资源三层调度模型

在分析现有的资源调度方案及模型的基础上,提出了基于层次化的网格资源三层调度模型．它由主调度器、次级调度器和计算节点组成。主调度器根据任务的性质和需求,并参考下层次级调度器的执行情况,将部分任务分发到各次级调度器上,实现了主调度器与次级调度器之间的并行工作。基于该模型提出轮循任务分发策略。通过分析和模拟．该资源调度模型及任务分发策略在调度性能上明显优于集中式调度方案。     

网格资源管理中的局部性调度

考虑通信实体之间的距离、可用带宽以及通信和资源使用费用,提出了抽象距离的数学模型,并结合网格资源和网格应用模型,设计了局部性网格资源调度算法,该算法在选择资源时首先考虑在同一节点的资源,其次通过抽象距离选择邻近的节点。实验表明,局部性调度在通信开销、成本、任务完成时间以及任务执行的成功率等方面都得到了改善。     

Automatically composed workflows for grid environments

Once the realm of high-performance computing for scientific applications, grid computing is rising as a key enabling infrastructure for resource sharing and coordinated problem solving in dynamic multiinstitutional virtual organizations. Grids build over networking technology to provide middleware support such as locating files over a network of computers, scheduling the distributed execution of jobs, and managing resource sharing and access policies.2 The need of scientific communities to interconnect applications, data, expertise, and computing resources is shared by other application areas, such as business, government, medical care, and education.     

Design of a service oriented architecture for efficient resource allocation in media environments

In many business domains, Grids and Service Oriented Architectures are considered to improve application design, integration and execution. In the audiovisual industry, applications are very data-intensive, time-constrained and computationally demanding, and design of a Service Oriented Architecture in this domain is no straightforward task. Efficient resource allocation-especially in terms of network usage-is paramount to meet users’ requirements in terms of deadlines and responsiveness, and offer high scalability at the same time. We present a resource- and network-aware management architecture addressing the issues in media environments, incorporating a number of scheduling algorithms and advance reservation systems to ensure efficient resource usage.     

基于任务-资源分配图优化选取的网格依赖任务调度

任务调度是网格应用系统获得高性能的关键.网格计算中一个大型的应用程序往往被分解为具有依赖关系的多个任务.在资源个体差异较大、广域互连的网格环境下任务间的依赖关系对传统的调度策略提出了新的挑战.任务调度的主要工作是为任务分配资源以及确定任务的执行次序,将依赖任务的可能的资源分配方案表示为任务-资源分配图（T-RAG）,在该图的基础上提出了基于T-RAG优化选取的依赖任务调度模型,将依赖任务调度问题转化为图的优化选取问题,解析最优任务-资源分配图可以同时确定资源分配方案和任务的执行次序即为最优调度方案.最后,实现了基于该模型的任务调度算法,该算法与ILHA算法的对比分析表明,在资源差异较大及任务间存在大量数据传输的情况下所提出的算法更优.     

An efficient grid-scheduling strategy based on a fuzzy matchmaking approach

Computational grids have become an appealing research area as they solve compute-intensive problems within the scientific community and in industry. A grid computational power is aggregated from a huge set of distributed heterogeneous workers; hence, it is becoming a mainstream technology for large-scale distributed resource sharing and system integration. Unfortunately, current grid schedulers suffer from the haste problem, which is the schedule inability to successfully allocate all input tasks. Accordingly, some tasks fail to complete execution as they are allocated to unsuitable workers. Others may not start execution as suitable workers are previously allocated to other peers. This paper is the first to introduce the scheduling haste problem. It also presents a reliable grid scheduler. The proposed scheduler selects the most suitable worker to execute an input grid task using a fuzzy inference system. Hence, it minimizes the turnaround time for a set of grid tasks. Moreover, our scheduler is a system-oriented one as it avoids the scheduling haste problem. Experimental results have shown that the proposed scheduler outperforms traditional grid schedulers as it introduces a better scheduling efficiency.     

Modeling and Supporting Grid Scheduling

Grid resource management systems and schedulers are important components for building Grids. They are responsible for the selection and allocation of Grid resources to current and future applications. Thus, they are important building blocks for making Grids available to user communities. In this paper we briefly analyze the requirements of Grid resource management and provide a classification of schedulers. Then, we define an extensible formal model for Grid scheduling activities, and characterize the general Grid scheduling problem. Finally, we provide a reference architecture for the support of our model and discuss different aspects of architectural implementations.     

An efficient system-oriented grid scheduler based on a fuzzy matchmaking approach

Computational grids have become an appealing research area as they solve compute-intensive problems within the scientific community and in industry. A Grid computational power is aggregated from a huge set of distributed heterogeneous workers; hence, it is becoming a mainstream technology for large-scale distributed resource sharing and system integration. Unfortunately, current grid schedulers suffer from the haste problem, which is the schedule inability to successfully allocate all input tasks. Accordingly, some tasks fail to complete execution as they are allocated to unsuitable workers. Others may not start execution as suitable workers are previously allocated to other peers. This paper is the first to introduce the scheduling haste problem. It also presents a reliable grid scheduler. The proposed scheduler selects the most suitable worker to execute an input grid task using a fuzzy inference system. Hence, it minimizes the turnaround time for a set of grid tasks. Moreover, our scheduler is a system-oriented one as it avoids the scheduling haste problem. Experimental results have shown that the proposed scheduler outperforms traditional grid schedulers as it introduces a better scheduling efficiency.     

一种基于Cache的网格任务反馈调度方法

网格调度的目标提高网格资源的利用率、改善网格应用的性能,它是网格中需着力解决的问题之一.目前,围绕着网格中的任务调度算法,国内外已做了大量的研究工作,先后提出了各种调度算法.但是,这些调度算法不能很好地适应网格环境下的自治性、动态性、分布性等特征.针对目前网格调度机制存在的问题,提出了一种动态的网格调度技术--基于Cache的反馈调度方法(cache based feedback scheduling,简称CBFS).该调度方法依据Cache中所存放的最近访问过的资源信息,如最近一次请求提交时间、任务完成时间等信息进行反馈调度,将任务提交给负载较小或性能较优的资源来完成.实验结果表明,CBFS方法不但可以有效减少不必要的延迟,而且在任务响应时间的平滑性、任务的吞吐率及任务在调度器等待调度的时间方面比随机调度等传统算法要好.     

Assignment of independent tasks to minimize completion time

The task of scheduling dynamic applications that consist of single process tasks on a non-shared memory multicomputer is examined in this paper. Each task of the application is assumed to (1) require execution on a single processor, (2) have an estimate of its maximum execution time, and (3) not wait on communications with other tasks. The objective of the studied schedulers is to map an application's tasks onto the underlying hardware in such a way that the application's completion time is minimized. Experimental evaluation of the schedulers indicate that in many situations, a more sophisticated scheduler fails to outperform simpler schedulers.     

Scheduling Task Parallel Applications for Rapid Turnaround on Enterprise Desktop Grids

Desktop Grids are popular platforms for high throughput applications, but due to their inherent resource volatility it is difficult to exploit them for applications that require rapid turnaround. Efficient desktop Grid execution of short-lived applications is an attractive proposition and we claim that it is achievable via intelligent resource selection. We propose three general techniques for resource selection: resource prioritization, resource exclusion, and task duplication. We use these techniques to instantiate several scheduling heuristics. We evaluate these heuristics through trace-driven simulations of four representative desktop Grid configurations. We find that ranking desktop resources according to their clock rates, without taking into account their availability history, is surprisingly effective in practice. Our main result is that a heuristic that uses the appropriate combination of resource prioritization, resource exclusion, and task replication can achieve performance within a factor of 1.7 of optimal in practice.     

Reputation-based dependable scheduling of workflow applications in Peer-to-Peer Grids

Grids facilitate creation of wide-area collaborative environment for sharing computing or storage resources and various applications. Inter-connecting distributed Grid sites through peer-to-peer routing and information dissemination structure (also known as Peer-to-Peer Grids) is essential to avoid the problems of scheduling efficiency bottleneck and single point of failure in the centralized or hierarchical scheduling approaches. On the other hand, uncertainty and unreliability are facts in distributed infrastructures such as Peer-to-Peer Grids, which are triggered by multiple factors including scale, dynamism, failures, and incomplete global knowledge.In this paper, a reputation-based Grid workflow scheduling technique is proposed to counter the effect of inherent unreliability and temporal characteristics of computing resources in large scale, decentralized Peer-to-Peer Grid environments. The proposed approach builds upon structured peer-to-peer indexing and networking techniques to create a scalable wide-area overlay of Grid sites for supporting dependable scheduling of applications. The scheduling algorithm considers reliability of a Grid resource as a statistical property, which is globally computed in the decentralized Grid overlay based on dynamic feedbacks or reputation scores assigned by individual service consumers mediated via Grid resource brokers. The proposed algorithm dynamically adapts to changing resource conditions and offers significant performance gains as compared to traditional approaches in the event of unsuccessful job execution or resource failure. The results evaluated through an extensive trace driven simulation show that our scheduling technique can reduce the makespan up to 50% and successfully isolate the failure-prone resources from the system.     

Adaptive checkpointing strategy to tolerate faults in economy based grid

In this paper, we develop a fault tolerant job scheduling strategy in order to tolerate faults gracefully in an economy based grid environment. We propose a novel adaptive task checkpointing based fault tolerant job scheduling strategy for an economy based grid. The proposed strategy maintains a fault index of grid resources. It dynamically updates the fault index based on successful or unsuccessful completion of an assigned task. Whenever a grid resource broker has tasks to schedule on grid resources, it makes use of the fault index from the fault tolerant schedule manager in addition to using a time optimization heuristic. While scheduling a grid job on a grid resource, the resource broker uses fault index to apply different intensity of task checkpointing (inserting checkpoints in a task at different intervals). To simulate and evaluate the performance of the proposed strategy, this paper enhances the GridSim Toolkit-4.0 to exhibit fault tolerance related behavior. We also compare “checkpointing fault tolerant job scheduling strategy” with the well-known time optimization heuristic in an economy based grid environment. From the measured results, we conclude that even in the presence of faults, the proposed strategy effectively schedules grid jobs tolerating faults gracefully and executes more jobs successfully within the specified deadline and allotted budget. It also improves the overall execution time and minimizes the execution cost of grid jobs.     

服务网格中资源可用性评估方法研究*

资源可用性的预测与评估是动态网格环境下合理资源选择和保证服务质量的前提和基础。基于相关资源和任务的历史信息,利用概率论方法对资源可用性进行了预测与评估,提出了资源离线时间、本地任务执行时间、等待队列长度、等待时间等可用性尺度,证明并给出这些尺度的分布函数。实验表明,基于相关历史信息对资源可用性进行预测方法有效,并且根据资源可用性评估及提取的相关可用性尺度来确定任务调度的候选资源,可大大减少候选资源数目,从而降低调度的时间复杂度。     

一种基于任务响应时间预测的网格调度算法的研究

针对网格环境下计算节点的自治性、异构性、分布性等特征,提出了一种动态的基于任务响应时间预测的调度算法。该调度方法依据历史数据和最近访问过计算节点的任务请求提交时间、任务完成时间、网络通信延迟等信息,预测计算节点将来的任务响应时间,将任务提交给轻负载或性能较优的计算节点完成。实验结果表明,该方法不但可以有效减少不必要的延迟,而且在任务响应时间、任务的吞吐率及任务在调度器内等待被调度的时间方面比随机调度等传统算法要优。     

Processing capacity-based decision mechanism edge computing model for IoT applications

The handling of complex tasks in IoT applications becomes difficult due to the limited availability of resources in most IoT devices. There arises a need to offload the IoT tasks with huge processing and storage to resource enriched edge and cloud. In edge computing, factors such as arrival rate, nature and size of task, network conditions, platform differences and energy consumption of IoT end devices impacts in deciding an optimal offloading mechanism. A model is developed to make a dynamic decision for offloading of tasks to edge and cloud or local execution by computing the expected time, energy consumption and processing capacity. This dynamic decision is proposed as processing capacity-based decision mechanism (PCDM) which takes the offloading decisions on new tasks by scheduling all the available devices based on processing capacity. The target devices are then selected for task execution with respect to energy consumption, task size and network time. PCDM is developed in the EDGECloudSim simulator for four different applications from various categories such as time sensitiveness, smaller in size and less energy consumption. The PCDM offloading methodology is experimented through simulations to compare with multi-criteria decision support mechanism for IoT offloading (MEDICI). Strategies based on task weightage termed as PCDM-AI, PCDM-SI, PCDM-AN, and PCDM-SN are developed and compared against the five baseline existing strategies namely IoT-P, Edge-P, Cloud-P, Random-P, and Probabilistic-P. These nine strategies are again developed using MEDICI with the same parameters of PCDM. Finally, all the approaches using PCDM and MEDICI are compared against each other for four different applications. From the simulation results, it is inferred that every application has unique approach performing better in terms of response time, total task execution, energy consumption of device, and total energy consumption of applications.     

An Integrated Approach to Locality-Conscious Processor Allocation and Scheduling of Mixed-Parallel Applications

Complex parallel applications can often be modeled as directed acyclic graphs of coarse-grained application tasks with dependences. These applications exhibit both task and data parallelism, and combining these two (also called mixed parallelism) has been shown to be an effective model for their execution. In this paper, we present an algorithm to compute the appropriate mix of task and data parallelism required to minimize the parallel completion time (makespan) of these applications. In other words, our algorithm determines the set of tasks that should be run concurrently and the number of processors to be allocated to each task. The processor allocation and scheduling decisions are made in an integrated manner and are based on several factors such as the structure of the task graph, the runtime estimates and scalability characteristics of the tasks, and the intertask data communication volumes. A locality-conscious scheduling strategy is used to improve intertask data reuse. Evaluation through simulations and actual executions of task graphs derived from real applications and synthetic graphs shows that our algorithm consistently generates schedules with a lower makespan as compared to Critical Path Reduction (CPR) and Critical Path and Allocation (CPA), two previously proposed scheduling algorithms. Our algorithm also produces schedules that have a lower makespan than pure task- and data-parallel schedules. For task graphs with known optimal schedules or lower bounds on the makespan, our algorithm generates schedules that are closer to the optima than other scheduling approaches.     

FEADS: A Framework for Exploring the Application Design Space on Network Processors

Network processors are designed to handle the inherently parallel nature of network processing applications. However, partitioning and scheduling of application tasks and data allocation to reduce memory contention remain as major challenges in realizing the full performance potential of a given network processor. The large variety of processor architectures in use and the increasing complexity of network applications further aggravate the problem. This work proposes a novel framework, called FEADS, for automating the task of application partitioning and scheduling for network processors. FEADS uses the simulated annealing approach to perform design space exploration of application mapping onto processor resources. Further, it uses cyclic and r-periodic scheduling to achieve higher throughput schedules. To evaluate dynamic performance metrics such as throughput and resource utilization under realistic workloads, FEADS automatically generates a Petri net (PN) which models the application, architectural resources, mapping and the constructed schedule and their interaction. The throughput obtained by schedules constructed by FEADS is comparable to that obtained by manual scheduling for linear task flow graphs; for more complicated task graphs, FEADS’ schedules have a throughput which is upto 2.5 times higher compared to the manual schedules. Further, static scheduling of tasks results in an increase in throughput by upto 30% compared to an implementation of the same mapping without task scheduling.