An energy-efficient cuckoo search algorithm for virtual machine placement in cloud computing data centers

The Journal of Supercomputing - The advent of virtualization technology has created a huge potential application for cloud computing. In virtualization, a large hardware resource is often broken...     

ContainerCloudSim: An environment for modeling and simulation of containers in cloud data centers

Containers are increasingly gaining popularity and becoming one of the major deployment models in cloud environments. To evaluate the performance of scheduling and allocation policies in containerized cloud data centers, there is a need for evaluation environments that support scalable and repeatable experiments. Simulation techniques provide repeatable and controllable environments, and hence, they serve as a powerful tool for such purpose. This paper introduces ContainerCloudSim, which provides support for modeling and simulation of containerized cloud computing environments. We developed a simulation architecture for containerized clouds and implemented it as an extension of CloudSim. We described a number of use cases to demonstrate how one can plug in and compare their container scheduling and provisioning policies in terms of energy efficiency and SLA compliance. Our system is highly scalable as it supports simulation of large number of containers, given that there are more containers than virtual machines in a data center. Copyright © 2016 John Wiley & Sons, Ltd.     

An SLA-based cloud computing that facilitates resource allocation in the distributed data centers of a cloud provider

The number of cloud service users has increased worldwide, and cloud service providers have been deploying and operating data centers to serve the globally distributed cloud users. The resource capacity of a data center is limited, so distributing the load to global data centers will be effective in providing stable services. Another issue in cloud computing is the need for providers to guarantee the service level agreements (SLAs) established with consumers. Whereas various load balancing algorithms have been developed, it is necessary to avoid SLA violations (e.g., service response time) when a cloud provider allocates the load to data centers geographically distributed across the world. Considering load balancing and guaranteed SLA, therefore, this paper proposes an SLA-based cloud computing framework to facilitate resource allocation that takes into account the workload and geographical location of distributed data centers. The contributions of this paper include: (1) the design of a cloud computing framework that includes an automated SLA negotiation mechanism and a workload- and location-aware resource allocation scheme (WLARA), and (2) the implementation of an agent-based cloud testbed of the proposed framework. Using the testbed, experiments were conducted to compare the proposed schemes with related approaches. Empirical results show that the proposed WLARA performs better than other related approaches (e.g., round robin, greedy, and manual allocation) in terms of SLA violations and the provider’s profits. We also show that using the automated SLA negotiation mechanism supports providers in earning higher profits.     

An efficient energy-aware method for virtual machine placement in cloud data centers using the cultural algorithm

Excessive consumption of energy in cloud data centers whose number is increasing day by day has led to substantial problems. Hence, offering efficient schemes for virtual machine (VM) placement to decrease energy consumption in cloud computing environments has become a significant research field in recent years. In this paper, with the goal of reducing energy consumption in cloud data centers, we present a VM placement method using the cultural algorithm. In the proposed algorithm called balance-based cultural algorithm for virtual machine placement (BCAVMP), a new fitness function is introduced to evaluate VM allocation solutions. In this function, by using the sum of balance vector lengths for each VM placement, balanced utilization of resources is considered. Also, by applying the amount of energy usage in the fitness function, solutions with lower energy consumption are intended. The performance of the proposed method is evaluated using CloudSim simulator. The simulation results indicate that by appropriate VM assignment and resource wastage reduction, energy consumption in cloud data centers can be decreased.

     

云环境下影响数据分布并行应用执行效率的因素分析

云环境下,类似MapReduce的数据分布并行应用被广泛运用。针对此类应用执行效率低、成本高的问题,以Hadoop为例,首先,分析该类应用的执行方式,发现数据量、节点数和任务数是影响其效率的主要因素;其次,探讨以上因素对应用效率的影响;最后,通过实验得出在数据量一定的情况下,增加节点数不会明显提高应用的执行效率,反而极大地增加执行成本;当任务数接近节点数时,应用的执行效率较高、成本较低。该结论为云环境中类似MapReduce的数据分布并行应用的效率优化提供借鉴,并为用户租用云资源提供参考。     

分布式数据挖掘中间层

对如何简化机群系统上分布式数据挖掘系统的开发和维护,给出了一个完整的解决方案,并对数据挖掘系统的非算法部分进行深入的研究,给出了数据分布式存储、数据缓冲机制和负载平衡策略3个关键优化技术,并在实际应用中加以实现。     

Consensus-based data replication protocol for distributed cloud

The Journal of Supercomputing - Data availability ensures efficient data accessibility by the readers anytime and from anywhere. It can be addressed by creating multiple copies of each data file...     

Bandwidth allocation for communicating virtual machines in cloud data centers

The Journal of Supercomputing - High-performance computing in a cloud environment may require massive data transfer among some of the virtual machines (VMs). These VMs are deployed in physical...     

Synchronization methods in parallel and distributed discrete-event simulation

This work attempts to provide insight into the problem of executing discrete event simulation in a distributed fashion. The article serves as the state of the art in Parallel Discrete-Event Simulation (PDES) by surveying existing algorithms and analyzing the merits and drawbacks of various techniques. We discuss the main characteristics of existing synchronization methods for parallel and distributed discrete event simulation. The two major categories of synchronization protocols, namely conservative and optimistic, are introduced and various approaches within each category are presented. We also present the latest efforts towards PDES on emerging platforms such as heterogeneous multicore processors, Web services, as well as Grid and Cloud environment.     

GPU-aware resource management in heterogeneous cloud data centers

The Journal of Supercomputing - The power of rapid scalability and easy maintainability of cloud services is driving many high-performance computing applications from company server racks into...     

Energy-aware framework with Markov chain-based parallel simulated annealing algorithm for dynamic management of virtual machines in cloud data centers

Significant savings in the energy consumption, without sacrificing service level agreement (SLA), are an excellent economic incentive for cloud providers. By applying efficient virtual Machine placement and consolidation algorithms, they are able to achieve these goals. In this paper, we propose a comprehensive technique for optimum energy consumption and SLA violation reduction. In the proposed approach, the issues of allocation and management of virtual machines are divided into smaller parts. In each part, new algorithms are proposed or existing algorithms have been improved. The proposed method performs all steps in distributed mode and acts in centralized mode only in the placement of virtual machines that require a global vision. For this purpose, the population-based or parallel simulated annealing (SA) algorithm is used in the Markov chain model for virtual machines placement policy. Simulation of algorithms in different scenarios in the CloudSim confirms better performance of the proposed comprehensive algorithm.     

Generating data transfers for distributed GPU parallel programs

Nowadays, high performance applications exploit multiple level architectures, due to the presence of hardware accelerators like GPUs inside each computing node. Data transfers occur at two different levels: inside the computing node between the CPU and the accelerators and between computing nodes. We consider the case where the intra-node parallelism is handled with HMPP compiler directives and message-passing programming with MPI is used to program the inter-node communications. This way of programming on such an heterogeneous architecture is costly and error-prone. In this paper, we specifically demonstrate the transformation of HMPP programs designed to exploit a single computing node equipped with a GPU into an heterogeneous HMPP + MPI exploiting multiple GPUs located on different computing nodes.     

SWORD: workload-aware data placement and replica selection for cloud data management systems

Cloud computing is increasingly being seen as a way to reduce infrastructure costs and add elasticity, and is being used by a wide range of organizations. Cloud data management systems today need to serve a range of different workloads, from analytical read-heavy workloads to transactional (OLTP) workloads. For both the service providers and the users, it is critical to minimize the consumption of resources like CPU, memory, communication bandwidth, and energy, without compromising on service-level agreements if any. In this article, we develop a workload-aware data placement and replication approach, called SWORD, for minimizing resource consumption in such an environment. Specifically, we monitor and model the expected workload as a hypergraph and develop partitioning techniques that minimize the average query span, i.e., the average number of machines involved in the execution of a query or a transaction. We empirically justify the use of query span as the metric to optimize, for both analytical and transactional workloads, and develop a series of replication and data placement algorithms by drawing connections to several well-studied graph theoretic concepts. We introduce a suite of novel techniques to achieve high scalability by reducing the overhead of partitioning and query routing. To deal with workload changes, we propose an incremental repartitioning technique that modifies data placement in small steps without resorting to complete repartitioning. We propose the use of fine-grained quorums defined at the level of groups of data items to control the cost of distributed updates, improve throughput, and adapt to different workloads. We empirically illustrate the benefits of our approach through a comprehensive experimental evaluation for two classes of workloads. For analytical read-only workloads, we show that our techniques result in significant reduction in total resource consumption. For OLTP workloads, we show that our approach improves transaction latencies and overall throughput by minimizing the number of distributed transactions.     

Resource abstraction and data placement for distributed hybrid memory pool

Emerging byte-addressable non-volatile memory (NVM) technologies offer higher density and lower cost than DRAM, at the expense of lower performance and limited write endurance. There have been many studies on hybrid NVM/DRAMmemory management in a single physical server. However, it is still an open problem on how to manage hybrid memories efficiently in a distributed environment. This paper proposes Alloy, a memory resource abstraction and data placement strategy for an RDMA-enabled distributed hybrid memory pool (DHMP). Alloy provides simple APIs for applications to utilize DRAM or NVM resource in the DHMP, without being aware of the hardware details of the DHMP. We propose a hotness-aware data placement scheme, which combines hot data migration, data replication and write merging together to improve application performance and reduce the cost of DRAM. We evaluate Alloy with several micro-benchmark workloads and public benchmark workloads. Experimental results show that Alloy can significantly reduce the DRAM usage in the DHMP by up to 95%, while reducing the total memory access time by up to 57% compared with the state-of-the-art approaches.     

VMPlanner: Optimizing virtual machine placement and traffic flow routing to reduce network power costs in cloud data centers

In recent years, the power costs of cloud data centers have become a practical concern and have attracted significant attention from both industry and academia. Most of the early works on data center energy efficiency have focused on the biggest power consumers (i.e., computer servers and cooling systems), yet without taking the networking part into consideration. However, recent studies have revealed that the network elements consume 10–20% of the total power in the data center, which poses a great challenge to effectively reducing network power cost without adversely affecting overall network performance. Based on the analysis on topology characteristics and traffic patterns of data centers, this paper presents a novel approach, called VMPlanner, for network power reduction in the virtualization-based data centers. The basic idea of VMPlanner is to optimize both virtual machine placement and traffic flow routing so as to turn off as many unneeded network elements as possible for power saving. We formulate the optimization problem, analyze its hardness, and solve it by designing VMPlanner as a stepwise optimization approach with three approximation algorithms. VMPlanner is implemented and evaluated in a simulated environment with traffic traces collected from a data center test-bed, and the experiment results illustrate the efficacy and efficiency of this approach.     

Power and performance management for parallel computations in clouds and data centers

We address scheduling independent and precedence constrained parallel tasks on multiple homogeneous processors in a data center with dynamically variable voltage and speed as combinatorial optimization problems. We consider the problem of minimizing schedule length with energy consumption constraint and the problem of minimizing energy consumption with schedule length constraint on multiple processors. Our approach is to use level-by-level scheduling algorithms to deal with precedence constraints. We use a simple system partitioning and processor allocation scheme, which always schedules as many parallel tasks as possible for simultaneous execution. We use two heuristic algorithms for scheduling independent parallel tasks in the same level, i.e., SIMPLE and GREEDY. We adopt a two-level energy/time/power allocation scheme, namely, optimal energy/time allocation among levels of tasks and equal power supply to tasks in the same level. Our approach results in significant performance improvement compared with previous algorithms in scheduling independent and precedence constrained parallel tasks.