Dynamic balancing of communication and computation load for HLA-based simulations on large-scale distributed systems

Dynamic balancing of computation and communication load is vital for the execution stability and performance of distributed, parallel simulations deployed on the shared, unreliable resources of large-scale environments. High Level Architecture (HLA) based simulations can experience a decrease in performance due to imbalances that are produced initially and/or during run time. These imbalances are generated by the dynamic load changes of distributed simulations or by unknown, non-managed background processes resulting from the non-dedication of shared resources. Due to the dynamic execution characteristics of elements that compose distributed applications, the computational load and interaction dependencies of each simulation entity change during run time. These dynamic changes lead to an irregular load and communication distribution, which increases overhead of resources and latencies. A static partitioning of load is limited to deterministic applications and is incapable of predicting the dynamic changes caused by distributed applications or by external background processes. Therefore, a scheme for balancing the communication and computational load during the execution of distributed simulations is devised in a scalable hierarchical architecture. The proposed balancing system employs local and cluster monitoring mechanisms in order to observe the distributed load changes and identify imbalances, repartitioning policies to determine a distribution of load and minimize imbalances. A migration technique is also employed by this proposed balancing system to perform reliable and low-latency load transfers. Such a system successfully improves the use of shared resources and increases distributed simulations’ performance by minimizing communication latencies and partitioning the load evenly. Experiments and comparative analyses were conducted in order to identify the gains that the proposed balancing scheme provides to large-scale distributed simulations.     

An improved load-balancing mechanism based on deadline failure recovery on GridSim

Grid computing has emerged a new field, distinguished from conventional distributed computing. It focuses on large-scale resource sharing, innovative applications and in some cases, high performance orientation. The Grid serves as a comprehensive and complete system for organizations by which the maximum utilization of resources is achieved. The load balancing is a process which involves the resource management and an effective load distribution among the resources. Therefore, it is considered to be very important in Grid systems. For a Grid, a dynamic, distributed load balancing scheme provides deadline control for tasks. Due to the condition of deadline failure, developing, deploying, and executing long running applications over the grid remains a challenge. So, deadline failure recovery is an essential factor for Grid computing. In this paper, we propose a dynamic distributed load-balancing technique called “Enhanced GridSim with Load balancing based on Deadline Failure Recovery” (EGDFR) for computational Grids with heterogeneous resources. The proposed algorithm EGDFR is an improved version of the existing EGDC in which we perform load balancing by providing a scheduling system which includes the mechanism of recovery from deadline failure of the Gridlets. Extensive simulation experiments are conducted to quantify the performance of the proposed load-balancing strategy on the GridSim platform. Experiments have shown that the proposed system can considerably improve Grid performance in terms of total execution time, percentage gain in execution time, average response time, resubmitted time and throughput. The proposed load-balancing technique gives 7 % better performance than EGDC in case of constant number of resources, whereas in case of constant number of Gridlets, it gives 11 % better performance than EGDC.     

Towards a hybrid load balancing policy in grid computing system

Grid computing has become conventional in distributed systems due to technological advancements and network popularity. Grid computing facilitates distributed applications by integrating available idle network computing resources into formidable computing power. As a result, by using efficient integration and sharing of resources, this enables abundant computing resources to solve complicated problems that a single machine cannot manage. However, grid computing mines resources from accessible idle nodes and node accessibility varies with time. A node that is currently idle, may become occupied within a second of time and then be unavailable to provide resources. Accordingly, node selection must provide effective and sufficient resources over a long period to allow load assignment. This study proposes a hybrid load balancing policy to integrate static and dynamic load balancing technologies. Essentially, a static load balancing policy is applied to select effective and suitable node sets. This will lower the unbalanced load probability caused by assigning tasks to ineffective nodes. When a node reveals the possible inability to continue providing resources, the dynamic load balancing policy will determine whether the node in question is ineffective to provide load assignment. The system will then obtain a new replacement node within a short time, to maintain system execution performance.     

An efficient grid scheduling strategy for data parallel applications

Scheduling large-scale application in heterogeneous grid systems is a fundamental NP-complete problem that is critical to obtain good performance and execution cost. To achieve high performance in a grid system it requires effective task partitioning, resource management and load balancing. The heterogeneous and dynamic nature of a grid, as well as the diverse demands of applications running on the grid, makes grid scheduling a major task. Existing schedulers in wide-area heterogeneous systems require a large amount of information about the application and the grid environment to produce reasonable schedules. However, this required information may not be available, may be too expensive to collect, or may increase the runtime overhead of the scheduler such that the scheduler is rendered ineffective. We believe that no one scheduler is appropriate for all grid systems and applications. This is because while data parallel applications in which further data partitioning is possible can be further improved by efficient management of resources, smart selection of resources and load balancing can be possible, in functional/not-dividable-task parallel applications such partitioning is either not possible or difficult or expensive in term of performance. In this paper, we propose a scheduler for data parallel applications (SDPA) which offers an efficient task partitioning and load balancing strategy for data parallel applications in grid environment. The proposed SDPA offers two major features: maintaining job priority even if insufficient number of free resources is available and pre-task assignment to cut the idle time of nodes. The SDPA selects nodes smartly according to the nature of task and the nodes’ resources availability. Simulation results conducted reveal that SDPA achieves performance improvement over reported strategies in the reviewed literature in terms of execution time, throughput and waiting time.     

Efficient task migration algorithm for distributed systems

The objective of the study was to achieve balanced load among processors, reduce the communication overhead of the load balancing algorithm, and improve respource utilization, which results in better average resonse time. A communication protocol and a fully distributed algorithm for dynamic load balancing through task migration in a connected N-processor network are presented. Each processor communicates its load directly with only a subset (of the size √ N) of processors, reducing communication traffic and average response time. It is proved that the given algorithm will perform task migration even if there is only one light load processor and one heavy load processor in the system. Simulation results show that the proposed scheme can save up to 60% of the protocol messages used by the broadcast algorithms and can reduce the average response time     

Optimal Static Load Balancing in Star Network Configurations with Two-Way Traffic

We study the static load balancing problem in a distributed computer system that consists of a set of heterogeneous computer systems interconnected by a star network with two-way traffic. We formulate the static load balancing problem as a nonlinear optimization problem which minimizes the mean response time. We prove that in the optimal solution the satellite nodes in the star network can be divided into four different types: the idle source nodes, the active source nodes, the neutral nodes, and the sink nodes. The necessary and sufficient conditions for optimal solution are studied. An efficient algorithm of complexity O(n) is proposed for the static load balancing of an n-satellite system. The effects of link communication time on optimal load balancing in a star network are also studied by parametric analysis. By employing the proposed algorithm, a significant system performance improvement over that without load balancing is illustrated in a numerical example. The numerical example also shows that the effects of the link communication time in a star network are large.     

Experiences from integrating algorithmic and systemic load balancing strategies

Load balancing increases the efficient use of existing resources for parallel and distributed applications. At a coarse level of granularity, advances in runtime systems for parallel programs have been proposed in order to control available resources as efficiently as possible by utilizing idle resources and using task migration. Simultaneously, at a finer granularity level, advances in algorithmic strategies for dynamically balancing computational loads by data redistribution have been proposed in order to respond to variations in processor performance during the execution of a given parallel application. Combining strategies from each level of granularity can result in a system which delivers advantages of both. The resulting integration is systemic in nature and transfers the responsibility of efficient resource utilization from the application programmer to the runtime system. This paper presents the design and implementation of a system that combines an algorithmic fine-grained data parallel load balancing strategy with a systemic coarse-grained task-parallel load balancing strategy, and reports on recent experimental results of running a computationally intensive scientific application under this integrated system. The experimental results indicate that a distributed runtime environment which combines both task and data migration can provide performance advantages with little overhead. It also presents proposals for performance enhancements of the implementation, as well as future explorations for effective resource management. Copyright © 2001 John Wiley & Sons, Ltd.     

Task partitioning,scheduling and load balancing strategy for mixed nature of tasks

Load balancing and task partitioning are important components of distributed computing. The optimum performance from the distributed computing system is achieved by using effective scheduling and load balancing strategy. Researchers have well explored CPU, memory, and I/O-intensive tasks scheduling, and load balancing techniques. But one of the main obstacles of the load balancing technique leads to the ignorance of applications having a mixed nature of tasks. This is because load balancing strategies developed for one kind of job nature are not effective for the other kind of job nature. We have proposed a load balancing scheme in this paper, which is known as Mixed Task Load Balancing (MTLB) for Cluster of Workstation (CW) systems. In our proposed MTLB strategy, pre-tasks are assigned to each worker by the master to eliminate the worker’s idle time. A main feature of MTLB strategy is to eradicate the inevitable selection of workers. Furthermore, the proposed MTLB strategy employs Three Resources Consideration (TRC) for load balancing (CPU, Memory, and I/O). The proposed MTLB strategy has removed the overheads of previously proposed strategies. The measured results show that MTLB strategy has a significant improvement in performance.     

A Uniform Framework for Dynamic Load Balancing Strategies in Distributed Processing Systems

Load balancing plays a central role in processor utilizations in distributed systems. Several strategies have been proposed in the literature to achieve load balancing. Usually, these strategies attempt to achieve a tradeoff between reducing the execution time of an application and minimizing the synchronization and the communication overhead. In this paper, we present a general model in which load balancing decisions are reached by enforcing performance metrics which may be adapted to reflect the specific requirements of different environments. Many of the load balancing schemes that have been suggested in the literature can be viewed as specific instances of the general framework presented in this paper. The basic scheme in this framework uses a load contention number that accounts for the load of the processors, the communication cost and the distance among processors. It is meant to be adaptable to the overall load on the system, the load on the communication devices, the run time characteristics of the tasks, and the configuration of the system. Furthermore, its implementation is not computationally complex. Thus, the gains made by load balancing are not overshadowed by the load balancing cost.     

基于最小通信开销的动态负载均衡策略

针对动态负载均衡过程产生额外通信开销的问题,建立了一种基于最小通信开销的数学模型。在此基础上,提出一种利用遗传算法解决该问题的新策略。该策略可减少负载迁移次数,降低动态负载均衡过程中的网络流量。仿真实验表明,该策略可获得比贪心策略具有更小通信开销的分配方案。     

基于网格计算的混合负载均衡的改进策略

近年来,随着科学研究对计算资源的要求不断增加,结合分布式计算环境和互联网的网格计算已经得到越来越多研究者的关注。网格计算就是利用网络中的空闲计算资源来协助那些要求大量计算的复杂任务的执行。根据分布式系统的静态和动态负载均衡策略的优缺点,本文提出了在网格计算环境下的混合负载均衡策略。为了让网络中的节点在网格计算环境中有效地执行需要大量计算的复杂任务,并根据大量的实验总结,提出了新的用来评估节点效率的函数,较以前的函数执行效率有了提高。     

Game-Theoretic Approach for Load Balancing in Computational Grids

Load balancing is a very important and complex problem in computational grids. A computational grid differs from traditional high-performance computing systems in the heterogeneity of the computing nodes, as well as the communication links that connect the different nodes together. There is a need to develop algorithms that can capture this complexity yet can be easily implemented and used to solve a wide range of load-balancing scenarios. In this paper, we propose a game-theoretic solution to the grid load-balancing problem. The algorithm developed combines the inherent efficiency of the centralized approach and the fault-tolerant nature of the distributed, decentralized approach. We model the grid load-balancing problem as a noncooperative game, whereby the objective is to reach the Nash equilibrium. Experiments were conducted to show the applicability of the proposed approaches. One advantage of our scheme is the relatively low overhead and robust performance against inaccuracies in performance prediction information.     

An adaptive threshold based hybrid load balancing scheme with sender and receiver initiated approach using random information exchange

The primary objective of load balancing for distributed systems is to minimize the job execution time while maximizing the resource utilization. Load balancing on decentralized systems need effective information exchange policy so that with minimum amount of communication the nodes have up to date information about other nodes in the system. Periodic, event‐based and on‐demand information exchange are some important policies used for the same. All these approaches involve a lot of overhead and even sometime leading toward obsolete data with the nodes if there is a delay in the updation. This work presents an adaptive threshold‐based hybrid load balancing scheme with sender and receiver initiated approach (HLBWSR) using random information exchange (RIE). RIE ensures that the information is exchanged in such a way that each node in the system has up‐to‐date state of the other nodes with much reduced communication overhead. Further, the adaptive threshold ensures that almost an average numbers of jobs are executed by all the nodes in the system. The study of the effect of the use of RIE on sender initiated, receiver initiated and hybrid of sender and receiver initiated load balancing approach establishes the superior performance of HLBWSR among its RIE‐based peers. A comparative analysis of HLBWSR, with periodic information exchange strategy, modified estimated load information scheduling algorithm and load balancing on arrival reveals its effectiveness under various test conditions. Copyright © 2016 John Wiley & Sons, Ltd.     

Distributed reachability analysis in timed automata

We evaluate a distributed reachability algorithm suitable for verification of real time critical systems modeled as timed automata. It is discovered that the algorithm suffers from load balancing problems and a high communication overhead. The load balancing problems are caused by the symbolic nature of the representation of the states of a timed automaton. We propose alternative data structures for representing the state-space of a timed automaton and adding a proportional load balancing controller on top of the algorithm. We evaluate various approaches at reducing communication overhead by increasing locality and compressing states. It is experimentally evaluated that by using the techniques speedups between 50% and 90% of linear can be obtained on a 14 node Linux Beowulf cluster on medium sized examples.     

A hybrid load balancing policy underlying grid computing environment

In recent years, network bandwidth and quality has been drastically improved, even much faster than the enhancement of computer performance. The various communication and computing tasks in the fields such as telecommunication, multimedia, information technology, and construction simulation, can be integrated and applied in a distributed computing environment nowadays. However, as the demands of many researches for computing resources gradually grow, Grid Computing integrated with a distributed computing environment and the Internet (network) has gained more attention. The so-called Grid Computing is to utilize the idle computing resources (nodes) on the network to facilitate the execution of complicated tasks that require large-scale computing. In other words, the composition of Grid resources is dynamic and varies with time. Thus, when selecting nodes for executing a task, the dynamic of the nodes in the Grid must be considered, and to exploit the effectiveness of the resources, they have to be properly selected according to the properties of the task. This study proposed a hybrid load balancing policy which integrated static and dynamic load balancing technologies to assist in the selection for effective nodes. In addition, if any selected node can no longer provide resources, it can be promptly identified and replaced with a substitutive node to maintain the execution performance and the load balancing of the system.     

A load balancing scheme for massively multiplayer online games

In a distributed MMOG (massively multiplayer online game) server architecture, the server nodes may become easily overloaded by the high demand from the players for state updates. Many works propose algorithms to distribute the load on the server nodes, but this load is usually defined as the number of players on each server, what is not an ideal measure. Also, the possible heterogeneity of the system is frequently overlooked. We propose a balancing scheme with two main goals: allocate load on server nodes proportionally to each one’s power and reduce the inter-server communication overhead, considering the load as the occupied bandwidth of each server. Four algorithms were proposed, from which ProGReGA is the best for overhead reduction and ProGReGA-KF is the most suited for reducing player migrations between servers. We also make a review of related works and some comparisons were made, where our approach performed better.     

并行循环的乐观自调度模式

并行循环的自调度模式是研究以最小运行开销和最佳负载平衡将循环体分布到各处理器上做并行计算，早期的自调度模式基于悲观的思想，认为并行循环是非均匀分布的，因此为克服负载不平衡，循环体被分割成大量任务包，因而导致较大的调度开销，本文提出一类乐观自调度模式，假定循环是均匀分布的，按现有处理器数对循环做初始划分可取得较好的负载平衡，同时，乐观模式还提出克服初始划分不良引起负载不平衡的一种简单且有效的方法，模     

Task assignment in distributed computing systems

The problem is addressed of assigning a task with a precedence constraint to a distributed computing system. Task turnaround time with regard to communication overhead and idle time is adopted to measure the performance of task assignment. The assignment of the module is determined as is the sequence of message transmission to balance the processor load and reduce communication overhead. The search for the optimal task assignment with a precedence constraint is known to be NP-complete (Garey et al. 1979) in the strong sense. A heuristic algorithm with polynomial time complexity is then proposed in order to solve the task-assignment problem effectively. Experimental results show that the proposed approach produces a near-optimal or even optimal task assignment.     

Reducing Communication Overhead in Multi-GPU Hybrid Solver for 2D Laplace’s Equation

The possibility of porting algorithms to graphics processing units (GPUs) raises significant interest among researchers. The natural next step is to employ multiple GPUs, but communication overhead may limit further performance improvement. In this paper, we investigate techniques reducing overhead on hybrid CPU–GPU platforms, including careful data layout and usage of GPU memory spaces, and use of non-blocking communication. In addition, we propose an accurate automatic load balancing technique for heterogeneous environments. We validate our approach on a hybrid Jacobi solver for 2D Laplace’s Equation. Experiments carried out using various graphics hardware and types of connectivity have confirmed that the proposed data layout allows our fastest CUDA kernels to reach the analytical limit for memory bandwidth (up to 106 GB/s on NVidia GTX 480), and that the non-blocking communication significantly reduces overhead, allowing for almost linear speed-up, even when communication is carried out over relatively slow networks.     

Strategy and Simulation of Adaptive RID for Distributed Dynamic Load Balancing in Parallel Systems

Dynamic load balancing schemes are significant for efficiently executing nonuniform problems in highly parallel multicomputer systems.The objective is to minimize the total exectuion time of single applications.This paper has proposed an ARID strategy for distributed dynamic load balancing.Its principle and control protocol are described,and te communication overhead,the effect on system stability and the performance efficiency are analyzed.Finally,simulation experiments are carried out to compare the adaptive strategy with other dynamic load balancing schemes.