分布式数据库系统的动态负载平衡策略及算法设计

随着网络中数据库应用的发展，使得分布式数据库系统的负载平衡问题突显出来。目前大多数分布式数据库管理系统没有负责平衡功能，其负载平衡依赖于操作系统的负载平衡机制来解决，这样带来的问题是系统负载的评价粒度细小和负载转移的开销增加。讨论了动态负载平衡策略的要素，针对分布式数据库系统的负载平衡问题，提出了以事务队列长度作为负载评价标准，并给出了一个动态负载平衡策略及算法。     

An improved load-balancing mechanism based on deadline failure recovery on GridSim

Grid computing has emerged a new field, distinguished from conventional distributed computing. It focuses on large-scale resource sharing, innovative applications and in some cases, high performance orientation. The Grid serves as a comprehensive and complete system for organizations by which the maximum utilization of resources is achieved. The load balancing is a process which involves the resource management and an effective load distribution among the resources. Therefore, it is considered to be very important in Grid systems. For a Grid, a dynamic, distributed load balancing scheme provides deadline control for tasks. Due to the condition of deadline failure, developing, deploying, and executing long running applications over the grid remains a challenge. So, deadline failure recovery is an essential factor for Grid computing. In this paper, we propose a dynamic distributed load-balancing technique called “Enhanced GridSim with Load balancing based on Deadline Failure Recovery” (EGDFR) for computational Grids with heterogeneous resources. The proposed algorithm EGDFR is an improved version of the existing EGDC in which we perform load balancing by providing a scheduling system which includes the mechanism of recovery from deadline failure of the Gridlets. Extensive simulation experiments are conducted to quantify the performance of the proposed load-balancing strategy on the GridSim platform. Experiments have shown that the proposed system can considerably improve Grid performance in terms of total execution time, percentage gain in execution time, average response time, resubmitted time and throughput. The proposed load-balancing technique gives 7 % better performance than EGDC in case of constant number of resources, whereas in case of constant number of Gridlets, it gives 11 % better performance than EGDC.     

基于．NET Remoting的动态负载平衡模型

在对.NET Remoting技术和传统的分布式负载平衡算法深入研究的基础上,提出一种排序队列和哈希映射矩阵相结合的负载平衡策略,给出一个分布式环境下的动态负载平衡模型。一方面可提高系统吞吐量,另一方面可缩短任务请求的响应时间。模型采用模块化设计方法,使其具有部署灵活性和容错性,并应用滑动窗口机制提高模型的负载平衡指标可信度。     

计算机网络互连的路由算法的设计及仿真研究

互连网络目前应用最广泛、最流行的一种网络拓扑,广泛应用于多处理器系统、电话网络、分布式计算机系统及路由器交换机等领域。本文主要对直连网络的负载均衡路由算法进行了研究,提出了一种新的负载均衡路由算法。通过对该算法的仿真发现在相同的网络仿真环境下,该算法的性能要优于传统路由算法。     

Biased random walks on resource network graphs for load balancing

The currently emerging large-scale complex networks and networks of networks are becoming apparent in the pervasive supply of seamless and transparent access to heterogeneous resources and services such as network domains, applications, services and storage owned by multiple organizations. The dynamics and heterogeneous environments involved, however, pose many challenges for controlling and balancing resource access, composition and deployment across complex grid and network infrastructures. In this paper, a scheme is proposed that gives a distributed load-balancing scheme by generating almost regular resource allocation networks. This network system is self-organized and depends only on local information for load distribution and resource discovery. The in-degree of each node refers to its free resources, and the job assignment and resource discovery processes required for load-balancing are accomplished by using fitted random sampling. Simulation results show that the generated network system provides an effective, scalable, and reliable load-balancing scheme for the distributed resources in grids and networks. The proposed solution is tested with real world data and the performance is tested against a recently reported distributed algorithm for load balancing.     

Dynamic Load Balancing in Distributed Systems in the Presence of Delays: A Regeneration-Theory Approach

A regeneration-theory approach is undertaken to analytically characterize the average overall completion time in a distributed system. The approach considers the heterogeneity in the processing rates of the nodes as well as the randomness in the delays imposed by the communication medium. The optimal one-shot load balancing policy is developed and subsequently extended to develop an autonomous and distributed load-balancing policy that can dynamically reallocate incoming external loads at each node. This adaptive and dynamic load balancing policy is implemented and evaluated in a two-node distributed system. The performance of the proposed dynamic load-balancing policy is compared to that of static policies as well as existing dynamic load-balancing policies by considering the average completion time per task and the system processing rate in the presence of random arrivals of the external loads.     

Optimal Dispersal of Certificate Chains

A regeneration-theory approach is undertaken to analytically characterize the average overall completion time in a distributed system. The approach considers the heterogeneity in the processing rates of the nodes as well as the randomness in the delays imposed by the communication medium. The optimal one-shot load balancing policy is developed and subsequently extended to develop an autonomous and distributed load-balancing policy that can dynamically reallocate incoming external loads at each node. This adaptive and dynamic load balancing policy is implemented and evaluated in a two-node distributed system. The performance of the proposed dynamic load-balancing policy is compared to that of static policies as well as existing dynamic load-balancing policies by considering the average completion time per task and the system processing rate in the presence of random arrivals of the external loads     

A trace-driven simulation study of dynamic load balancing

A trace-driven simulation study of dynamic load balancing in homogeneous distributed systems supporting broadcasting is presented. Information about job CPU and input/output (I/O) demands collected from production systems is used as input to a simulation model that includes a representative CPU scheduling policy and considers the message exchange and job transfer cost explicitly. Seven load-balancing algorithms are simulated and their performances compared. Load balancing is capable of significantly reducing the mean and standard deviation of job response times, especially under heavy load, and for jobs with high resource demands. Algorithms based on periodic or nonperiodic load information exchange provide similar performance, and, among the periodic policies, the algorithms that use a distinguished agent to collect and distribute load information cut down the overhead and scale better. With initial job placements only, source initiative algorithms were found to perform better than server initiative algorithms. The performances of all hosts, even those originally with light loads, are generally improved by load balancing     

A Dynamic Load Balancing Framework for Real-time Applications in Message Passing Systems

Load balancing algorithms are designed essentially to equally distribute the load on processors and maximize their utilities while minimizing the total task execution time. In order to achieve these goals, the load-balancing mechanism should be “fair” in distributing the load across the different processors. This implies that the difference between the heaviest-loaded and the lightest-loaded processors should be minimized. Therefore, the load information on each processor must be updated such that the load-balancing mechanism can be more effective. In this work, we present an application independent dynamic algorithm for scheduling tasks and load- balancing in message passing systems. We propose a DAG-based Dynamic Load Balancing algorithm for Real time applications (DAG-DLBR) that is designed to work dynamically to cope with possible changes in the load that might occur during runtime. This algorithm addresses the challenge of devising a load balancing scheme which judicially deals with the hybrid execution of existing real-time application (represented by a Direct Acyclic Graph (DAG)) together with newly arriving jobs. The main objective of this algorithm is to reduce response times of the newly arriving jobs while maintaining the time constrains of the existing DAG. To evaluate the performance of the DAG-DLBR algorithm, a comparison with the performance of two common dynamic load balancing algorithms is presented. This comparison is performed by evaluating, experimentally, the execution time of different load balancing algorithms on a homogenous real parallel machine. In addition, the values of load imbalance, the execution time, and the communication overhead time are evaluated analytically using different benchmarks as test-bed workloads. These workloads cover a wide range of dynamic applications with different task types. Experimental results illustrate the improved performance of the DAG-DLBR algorithm compared to both distributed and hierarchal based algorithms by at least 12 and 19%, respectively. This improvement is true for all workloads, even with highly dependent workload. The DAG-DLBR algorithm achieves lower computation time than its corresponding values of both the distributed and the hierarchical-based algorithms for 4, 8, 12 and 16 processors.     

Considering data skew factor in multi-way join query optimization for parallel execution

A consensus on parallel architecture for very large database management has emerged. This architecture is based on a shared-nothing hardware organization. The computation model is very sensitive to skew in tuple distribution, however. Recently, several parallel join algorithms with dynamic load balancing capabilities have been proposed to address this issue, but none of them consider multi-way join problems. In this article we propose a dynamic load balancing technique for multi-way joins, and investigate the effect of load balancing on query optimization. In particular, we present a join-ordering strategy that takes load-balancing issues into consideration. Our performance study indicates that the proposed query optimization technique can provide very impressive performance improvement over conventional approaches.An earlier version of this article was presented at the 1993 International Conference on Parallel and Distributed Information Systems in San Diego, California, U.S.A.     

A distributed load-balancing policy for a multicomputer

This paper deals with the organization of a distributed load-balancing policy for a multicomputer system which consists of a cluster of independent computers that are interconnected by a local area communication network. We introduce three algorithms necessary to maintain load balancing in this system: the local load algorithm, used by each processor to monitor its own load; the exchange algorithm, for exchanging load information between the processors, and the process migration algorithm that uses this information to dynamically migrate processes from overloaded to underloaded processors. The policy that we present is distributed, i.e. each processor uses the same policy. It is both dynamic, responding to load changes without using an a priori knowledge of the resources that each process requires; and stable, unnecessary overloading of a processor is minimized. We give the essential details of the implementation of the policy and initial results on its performance. Our results confirm the feasibility of building distributed systems that are based on network communication for uniform access, resource sharing and improved reliability, as well as the use of workstations without a secondary storage device.     

网格环境中任务调度算法的研究

随着计算机网络技术的成熟,基于网络的高性能计算的进一步发展,出现了网格系统,网格是一种异构的计算环境。首先展示了在异构平台上对于常规的计算和数据,最优分配将是一个NPC问题,寻求有效的静态调度将是困难的,动态策略应当是有前途的,因为机器负载是自我管理,甚至自我均衡,尽管任一机器是异构的。然而动态策略在异构平台上真正的敌人是数据依赖,可能将导致速度被拖为最慢的处理器速度。因此,对于网格系统的异构平台,调度方案应该是动静结合,处理器数据多样性的问题能够通过在每个可识别静态相之间重映像数据和计算来实现,采用ScaLA-PACK的块模式外部乘积算法,将问题抽象为矩形块的分配,给出了完全NP启发式算法的MM乘的最优结果,并且给出研究重映像的一个统一策略。最后,做了一个10个节点的异构平台的实验,展示了结果的实用性。     

网络入侵检测系统中动态负载平衡策略的设计

随着互联网的广泛应用,网络信息量迅猛增长,网络攻击数量和方式大大增加,网络入侵检测系统需要部署多个感知器(Sensor)时网络监测和保护,通过增加Sensor,可以增强系统分析检测能力。然而,为了充分利用系统处理能力,需要动态分配处理节点任务,实现动态负载平衡。该文在分析网络入侵检测系统的基础上,提出负载值计算方法,结合通用负载平衡策略,提出了分布式入侵检测系统的动态负载平衡策略,实现了良好的动态负载平衡效果,提高了系统性能。     

Load-Balanced Join Processing in Shared-Nothing Systems

In a shared-nothing parallel database system, a join operation is split into a set of tasks that are allocated to the nodes in the system to be executed concurrently and independently. While parallel processing could greatly reduce the completion time of a join operation, the system performance may degrade because of load imbalance across the nodes caused by data skewness in the relations. Load-balanced join processing uses various techniques to evenly distribute the load among nodes in a system and hence improves the overall system performance. In this paper, the basic issues in designing load-balanced parallel join algorithms are identified. From the solutions to those issues, a large set of load-balanced join algorithms can be constructed. Performance of four representative algorithms-two dynamic load-balancing algorithms proposed in this paper and two static load-balancing algorithms adapted from similar algorithms in the literature-is studied and compared with that of a parallel join algorithm that does not balance the join load. The results of our study clearly show the benefits of load-balancing. This study also demonstrates that the dynamic load-balancing techniques proposed in this paper not only are feasible but also provide good system performance.     

带输入队列并行交换的负载平衡分析

并行交换是新兴的交换技术,基于该技术能够利用小型交换模块来构建大容量的交换系统,例如太比特或更高容量的交换机.把带输入队列的并行交换称为带缓存并行交换(buffered parallel switch,简称BPS),重点研究其中并行且独立工作的交换模块之间的负载平衡问题.从不同角度出发,提出两种负载平衡的定义.基于两种定义,分别分析了BPS负载平衡的条件并提出分布式调度算法族.最后,提出一种简单而有效的调度算法,该算法能在无加速比BPS中同时满足两种定义,仿真实验结果表明了该算法的有效性和良好性能.另外,就算法的工程实现进行了讨论.     

有效的WebGIS地图服务器场负载均衡算法

针对现有网络地理信息系统（WebGIS）负载均衡技术的缺陷,分析利用集群技术解决WebGIS负载均衡的必要性和可行性,提出基于地图服务器场的WebGIS分布式负载均衡模型,研究地图服务器场分布式动态均衡调度算法,建立试验床,通过一系列的试验结果验证了该算法良好的全局网络负载性。     

Adaptive Load-Balancing Algorithms Using Symmetric Broadcast Networks

In a distributed computing environment, it is important to ensure that the processor workloads are adequately balanced. Among numerous load-balancing algorithms, a unique approach given by Das and Prasad defines a symmetric broadcast network (SBN) that provides a robust communication pattern among the processors in a topology-independent manner. In this paper, we propose and analyze three efficient SBN-based dynamic load-balancing algorithms, and implement them on an SGI Origin2000. A thorough experimental study with Poisson-distributed synthetic loads demonstrates that our algorithms are effective in balancing system load. By optimizing completion time and idle time, the proposed algorithms are shown to compare favorably with several existing approaches.     

Scalable distributed Louvain algorithm for community detection in large graphs

Community detection (or clustering) in large-scale graphs is an important problem in graph mining. Communities reveal interesting organizational and functional characteristics of a network. Louvain algorithm is an efficient sequential algorithm for community detection. However, such sequential algorithms fail to scale for emerging large-scale data. Scalable parallel algorithms are necessary to process large graph datasets. In this work, we show a comparative analysis of our different parallel implementations of Louvain algorithm. We design parallel algorithms for Louvain method in shared memory and distributed memory settings. Developing distributed memory parallel algorithms is challenging because of inter-process communication and load balancing issues. We incorporate dynamic load balancing in our final algorithm DPLAL (Distributed Parallel Louvain Algorithm with Load-balancing). DPLAL overcomes the performance bottleneck of the previous algorithms and shows around 12-fold speedup scaling to a larger number of processors. We also compare the performance of our algorithm with some other prominent algorithms in the literature and get better or comparable performance . We identify the challenges in developing distributed memory algorithm and provide an optimized solution DPLAL showing performance analysis of the algorithm on large-scale real-world networks from different domains.

     

基于交易服务的一个动态负载平衡模型

CORBA是当前被广泛采用的分布式对象平台,它成功地解决了异种平台的互操作问题。为了提高整个基于CORBA的分布式系统的性能,负载平衡成为CORBA应当考虑的问题。本文给出了一个基于交易服务的通用动态负载平衡模型。它不仅实现了CORBA负载平衡,而且其对交易服务的扩展对用户透明,原有应用可以直接与之集成。文章最后给出了模型的性能的评价和分析。     

A dynamic resource management in mobile agent by artificial neural network

In this paper, a resource management for dynamic load balancing in mobile agent by artificial neural network scheme (ANN-DLB) is presented to maximize the number of the served tasks in developing high performance cluster. This dynamic load balance with the growth of the service type and user number in the mobile networks of the higher performance is required in service provision and throughput. Most of the conventional policies are used in load indices with the threshold value to decide the load status of the agent hosts by CPU or memory. The main factor influencing the workload is the competitions among the computing resources such as CPU, memory, I/O and network. There are certain I/O data of the intensive applications where load balancing becomes the important issue. This relationship between the computing resources is very complex to define the rules for deciding the workload. This paper proposed a new dynamic load balancing for evaluating the agent hosts’ workload with the artificial neural network (ANN). By applying the automatic learning of the back-propagation network (BPN) model can establish the ANN model and also can measure the agent host loading with five inputs: CPU, memory, I/O, network and run-queue length. The structure of the load balancing system is composed of three design agents: the load index agent (LIA), the resource management agent (RMA) and the load transfer agent (LTA). These experimental results reveal that the proposed ANN-DLB yields better performance than the other methods. These results demonstrate that the proposed method has high throughput, short response time and turnaround time, and less agent host negotiation complexity and migrating tasks than the previous methods.