虚拟化云计算数据中心能量感知资源分配机制

针对云计算的资源管理问题,提出了云计算数据中心的能量模型以及四个虚拟机放置算法。首先计算每个机架上主机的负载并根据设定的阈值进行归类,然后采用最少迁移策略从主机上选择合适迁移的虚拟机并且接受新的虚拟机分配请求,对每个虚拟机与主机集合进行匹配,选择最优化的主机进行放置。实验结果表明,与现有的能量感知资源分配方法相比,该方法在主机、网络设备以及冷却系统方面能量利用率分别提高了2.4%,18.5%和28.1%,总的能量利用率平均提高了14.5%。     

Topology-Aware Virtual Machine Placement in Data Centers

This paper presents the Topology-aware Virtual Machine Placement algorithm, which aims at placing groups of virtual machines in data centers. It was designed to occupy small areas of the data center network in order to consolidate the network flows produced by the virtual machines. Extensive simulation is used to show that the proposed algorithm prevents the formation of network bottlenecks, therefore accepting more requests of allocation of virtual machines. Moreover, these advantages are obtained without compromising energy efficiency. The energy consumption of servers and switches are taken into account, and these are switched off whenever idle.     

云数据中心 I/O 资源池化

近年来,云计算的发展为数据中心带来了新的应用场景和需求.其中,虚拟化作为云服务的重要使能技术,对数据中心服务器I/O系统的性能、扩展性和设备种类多样性提出了更高的要求,沿用传统设备与服务器紧耦合的I/O架构将会导致资源冗余,数据中心服务器密度降低,布线复杂度增加等诸多问题.因此,文章围绕I/O资源池化架构的实现机制和方法展开研究,目标是解除设备与服务器之间的绑定关系,实现接入服务器对I/O资源的按需弹性化使用,从根本上解决云计算数据中心的I/O系统问题.同时,还提出了一种基于单根I/O虚拟化协议实现多根I/O资源池化的架构,该架构通过硬件的外设部件高速互连接口多根域间地址和标识符映射机制,实现了多个物理服务器对同一I/O设备的共享复用;通过虚拟I/O设备热插拔技术和多根共享管理机制,实现了虚拟I/O资源在服务器间的实时动态分配;采用现场可编程门阵列(Field-Programmable Gate Array)构建了该架构的原型系统.结果表明,该架构能够为各个共享服务器提供良好的I/O操作性能.     

云数据中心能效可靠的虚拟机部署

优化虚拟机部署是降低云数据中心能耗的有效方法,但是,过度对虚拟机部署进行合并可能导致主机机架出现热点,影响数据中心提供服务的可靠性。提出一种基于能效和可靠性的虚拟机部署算法。综合考虑主机利用率、主机温度、主机功耗、冷却系统功耗和主机可靠性间的相互关系,建立确保主机可靠性的冗余模型。在主动避免机架热点情况下,实现动态的虚拟机部署决策,在降低数据中心总体能耗前提下,确保主机服务可靠性。仿真结果表明,该算法不仅可以节省更多能耗,避免热点主机,而且性能保障上也更好。     

Data analytics for energy-efficient clouds: design,implementation and evaluation

ABSTRACT

The success of Cloud Computing and the resulting ever growing of large data centers is causing a huge rise in electrical power consumption by hardware facilities and cooling systems. This results in an increment of operational costs of data centres, that is becoming a crucial issue to deal with. Consolidation of virtual machines (VM) is one of the key strategies used to reduce the power consumption of Cloud servers. For this reason, it is extensively studied. Consolidation has the goal of allocating virtual machines on a few physical servers as possible while satisfying the Service Level Agreement established with users. Nevertheless, the effectiveness of a consolidation strategy strongly depends on the forecast of the VM resource needs. Predictive data mining models can be exploited for this purpose. This paper describes the design and development of a system for energy-aware allocation of virtual machines, driven by predictive data mining models. In particular, migrations are driven by the forecast of the future computational needs (CPU, RAM) of each virtual machine, in order to efficiently allocate those on the available servers. The experimental evaluation, performed on real-world Cloud data traces, reports a comparison of performance achieved by exploiting several classification models and shows good benefits in terms of energy saving.     

数据中心网络拓扑感知型能耗优化算法

针对数据中心网络中高能耗的问题,提出了一种拓扑感知型能耗优化算法。算法首先根据广义超立方体拓扑多维正交和单维全连接的结构特性,优化虚拟机的部署位置,进而提出多维最佳适应策略来充分利用服务器各维资源。然后利用虚拟机资源需求预测模型并结合迁移代价公式,均衡考虑服务器资源使用代价、虚拟机通信代价和迁移资源消耗,在合理迁移虚拟机以满足系统性能的前提下,降低了网络的能耗并且缓解了网络链路的拥塞。最终将网络的能耗优化问题转化成虚拟机在服务器上的优化配置问题。实验结果表明,与其他三种算法比较,算法在降低系统能耗和减少拥塞方面获得了良好的效果。     

More than bin packing: Dynamic resource allocation strategies in cloud data centers

Resource allocation strategies in virtualized data centers have received considerable attention recently as they can have substantial impact on the energy efficiency of a data center. This led to new decision and control strategies with significant managerial impact for IT service providers. We focus on dynamic environments where virtual machines need to be allocated and deallocated to servers over time. Simple bin packing heuristics have been analyzed and used to place virtual machines upon arrival. However, these placement heuristics can lead to suboptimal server utilization, because they cannot consider virtual machines, which arrive in the future. We ran extensive lab experiments and simulations with different controllers and different workloads to understand which control strategies achieve high levels of energy efficiency in different workload environments. We found that combinations of placement controllers and periodic reallocations achieve the highest energy efficiency subject to predefined service levels. While the type of placement heuristic had little impact on the average server demand, the type of virtual machine resource demand estimator used for the placement decisions had a significant impact on the overall energy efficiency.     

基于灰色关联度的云计算虚拟机分配方法

针对云计算系统中资源利用率提高和系统能耗降低之间的协调问题,提出了一种新的基于灰色关联度的虚拟机分配方法,应用灰色关联度的基本理论建立了基于服务层协议(SLA)违背率、系统能耗和服务器负载评价函数的虚拟机分配决策模型,构造了基于灰色关联度的虚拟机分配算法,并在CloudSim仿真平台上进行了实验。实验结果表明,与传统的基于简单线性权重的多目标优化方法相比,在不同的虚拟机选择策略下,基于灰色关联度的虚拟机分配方法在系统能耗、SLA违背率和虚拟机迁移次数上平均降低〖BP(〗是提高吗？应该是降低吧？请明确一下。〖BP)〗了6.8%、5.2%和15.5%。因此,所提方法在不同的虚拟机选择策略下能够大幅度减少虚拟机迁移次数,较好地满足系统在能耗和SLA违背率上的优化需求。     

Bayes Theorem Based Virtual Machine Scheduling for Optimal Energy Consumption

This paper proposes an algorithm for scheduling Virtual Machines (VM) with energy saving strategies in the physical servers of cloud data centers. Energy saving strategy along with a solution for productive resource utilization for VM deployment in cloud data centers is modeled by a combination of “Virtual Machine Scheduling using Bayes Theorem” algorithm (VMSBT) and Virtual Machine Migration (VMMIG) algorithm. It is shown that the overall data center’s consumption of energy is minimized with a combination of VMSBT algorithm and Virtual Machine Migration (VMMIG) algorithm. Virtual machine migration between the active physical servers in the data center is carried out at periodical intervals as and when a physical server is identified to be under-utilized. In VM scheduling, the optimal data centers are clustered using Bayes Theorem and VMs are scheduled to appropriate data center using the selection policy that identifies the cluster with lesser energy consumption. Clustering using Bayes rule minimizes the number of server choices for the selection policy. Application of Bayes theorem in clustering has enabled the proposed VMSBT algorithm to schedule the virtual machines on to the physical server with minimal execution time. The proposed algorithm is compared with other energy aware VM allocations algorithms viz. “Ant-Colony” optimization-based (ACO) allocation scheme and “min-min” scheduling algorithm. The experimental simulation results prove that the proposed combination of ‘VMSBT’ and ‘VMMIG’ algorithm outperforms other two strategies and is highly effective in scheduling VMs with reduced energy consumption by utilizing the existing resources productively and by minimizing the number of active servers at any given point of time.     

New scoring formula to rank hypervisors’ performance complementing with statistical analysis using DOE

Hypervisors enable cloud computing model to provide scalable infrastructures and on-demand access to computing resources as they support multiple operating systems to run on one physical server concurrently. This mechanism enhances utilization of physical server thus reduces server count in the data center. Hypervisors also drive the benefits of reduced IT infrastructure setup and maintenance costs along with power savings. It is interesting to know different hypervisors’ performance for the consolidated application workloads. Three hypervisors ESXi, XenServer, and KVM are carefully chosen to represent three categories full virtualized, para-virtualized, and hybrid virtualized respectively for the experiment. We have created a private cloud using CloudStack. Hypervisors are deployed as hosts in the private cloud in the respective clusters. Each hypervisor is deployed with three virtual machines. Three applications web server, application server, and database servers are installed on three virtual machines. Experiments are designed using Design of Experiments (DOE) methodology. With concurrently running virtual machines, each hypervisor is stressed with the consolidated real-time workloads (web load, application load, and OLTP) and important system information is gathered using SIGAR framework. The paper proposes a new scoring formula for hypervisors’ performance in the private cloud for consolidated application workloads and the accuracy of the results are complemented with sound statistical analysis using DOE.     

Performance-aware server architecture recommendation and automatic performance verification technology on IaaS cloud

In this paper, we propose a server architecture recommendation and automatic performance verification technology, which recommends and verifies appropriate server architecture on Infrastructure as a Service (IaaS) cloud with bare metal servers, container-based virtual servers and virtual machines. Recently, cloud services are spread, and providers provide not only virtual machines but also bare metal servers and container-based virtual servers. However, users need to design appropriate server architecture for their requirements based on three types of server performances, and users need much technical knowledge to optimize their system performance. Therefore, we study a technology that satisfies users’ performance requirements on these three types of IaaS cloud. Firstly, we measure performance and start-up time of a bare metal server, Docker containers, KVM (Kernel-based Virtual Machine) virtual machines on OpenStack with changing number of virtual servers. Secondly, we propose a server architecture recommendation technology based on the measured quantitative data. A server architecture recommendation technology receives an abstract template of OpenStack Heat and function/performance requirements and then creates a concrete template with server specification information. Thirdly, we propose an automatic performance verification technology that executes necessary performance tests automatically on provisioned user environments according to the template. We implement proposed technologies, confirm performance and show the effectiveness.     

可靠性感知下的虚拟数据中心映射算法

介绍现阶段虚拟数据中心(VDC)映射的研究进展,根据租户对VDC可靠性的需求,提出一种可靠性感知下的VDC映射启发式算法。对于每个VDC,该算法通过限制能放置在同一个服务器上的最大虚拟机数目来保证租户VDC可靠性需求,然后以降低数据中心网络带宽消耗和服务器能耗为主要目标进行VDC映射。其具体做法是:首先将相互之间带宽需求量大的虚拟机合并部署来降低数据中心网络带宽的消耗;然后把合并后的虚拟机优先部署到已开启的服务器上,从而减少开启的服务器数目,降低数据中心的服务器能耗。利用基于胖树结构的数据中心拓扑对提出的算法进行了仿真,结果表明,与2EM算法相比,该算法能够满足租户VDC的可靠性需求,能在不增加额外能耗的前提下最多减少数据中心网络约30%的带宽消耗。     

资源调度等待开销感知的虚拟机整合

近年来,数据中心庞大的能源开销问题引起广泛关注.虚拟化管理平台可以通过虚拟机迁移技术将虚拟机整合到更少的服务器上,从而提高数据中心能源有效性.对面向数据中心节能的虚拟机整合研究工作进行调研,并总结虚拟机整合研究存在的3个挑战.针对已有工作未考虑虚拟机等待资源调度带来的服务器资源额外开销这种现象,开展了资源调度等待开销感知的虚拟机整合研究.从理论和实验上证明了在具有实际意义的约束条件下,存在着虚拟机等待资源调度带来的服务器资源额外开销,且随着整合虚拟机数量的增长保持稳定.基于典型工作负载的实验结果表明,这个额外开销平均占据了11.7%的服务器资源开销.此外,提出了资源预留整合（MRC）算法,用于改进已有的虚拟机整合算法.算法模拟实验结果表明,MRC算法相比于常用的虚拟机整合算法FFD（first fit decreasing）,明显降低了服务器资源溢出概率.     

Hybrid shuffled frog leaping algorithm for energy-efficient dynamic consolidation of virtual machines in cloud data centers

Cloud computing aims to provide dynamic leasing of server capabilities as scalable virtualized services to end users. However, data centers hosting cloud applications consume vast amounts of electrical energy, thereby contributing to high operational costs and carbon footprints. Green cloud computing solutions that can not only minimize the operational costs but also reduce the environmental impact are necessary. This study focuses on the Infrastructure as a Service model, where custom virtual machines (VMs) are launched in appropriate servers available in a data center. A complete data center resource management scheme is presented in this paper. The scheme can not only ensure user quality of service (through service level agreements) but can also achieve maximum energy saving and green computing goals. Considering that the data center host is usually tens of thousands in size and that using an exact algorithm to solve the resource allocation problem is difficult, the modified shuffled frog leaping algorithm and improved extremal optimization are employed in this study to solve the dynamic allocation problem of VMs. Experimental results demonstrate that the proposed resource management scheme exhibits excellent performance in green cloud computing.     

基于轮换策略的异构云资源分配算法

云计算以其按需索取、按需付费、无需预先投资的优势给用户带来极大的便利,然而静态、单一的云计算环境容易成为网络攻击的目标,给用户带来较大的安全风险。动态的虚拟机部署策略和异构的云基础设施在提升云计算环境安全性的同时会降低资源利用率。提出一种针对虚拟机轮换时的资源分配算法,将不同类型的资源抽象成维度不同的向量,并通过求解装箱问题实现资源分配中的负载平衡,同时为每个虚拟机设定驻留时间,对当前服务器的负载状态进行轮换以提升虚拟机的安全性。实验结果表明,资源动态分配算法在提高虚拟机安全性能的同时,能够减小轮换带来的负载波动。     

Feedback control for multi-resource usage of virtualised database server

Virtualisation and cloud computing have recently received significant attention. Resource allocation and control of multiple resource usages among virtual machines in virtualised data centres remains an open problem. Therefore, in this paper, our focus is to control CPU (central processing unit) usage and memory consumption of a virtual database machine in a data centre under a time-varying heavy workload. In addition to existing work, we attempt to control multiple outputs, such as the CPU usage and memory consumption of a virtualised database server (DBVM), via changing multiple server parameters, such as the CPU allocation and memory allocation, in real time. We indicated that a virtualised database server might be modelled as a linear time-unvarying system. We obtained and compared both MIMO (multi input–multi output) and multiple SISO (single input–single output) models of that system. We designed multiple SISO feedback controllers to achieve desired CPU usages and memory consumptions under workload.     

Analysis of virtual machine live-migration as a method for power-capping

To reduce the construction cost of the power-supplying infrastructure in data centers and to increase the utilization of the existing one, many researchers have introduced software-based or hardware-based power-capping schemes. In servers with consolidated virtual machines, which can be easily found in cloud systems, exporting virtual machines to other light-loaded servers through live-migration is one of the key approaches to impose power-capping on servers. Up until now, most researchers who have tried to achieve power-capping through live-migration assumed that exporting a virtual machine instantly reduces the server power consumption. However, our analysis introduced in this paper reveals that the power consumption remains high or increases for a few seconds during a migration instance. This behavior contradicts the aim of power-capping, and may endanger the stability of servers. Based on this observation, we also propose and evaluate two power-suppressing live-migration schemes to resolve the power overshooting issue. Our evaluation shows that both approaches immediately limit the power consumption after live-migration is initiated.     

A paracasting model for concurrent access to replicated Internet content

In this paper, we develop a model to study how to effectively download a document from a set of replicated servers. We propose a generalized application-layer anycasting protocol, known as paracasting, to advocate concurrent access of a subset of replicated servers to cooperatively satisfy a client's request. Each participating server satisfies the request in part by transmitting a subset of the requested file to the client. The client can recover the complete file when different parts of the file sent from the participating servers are received. This model allows us to estimate the average time to download a file from the set of homogeneous replicated servers, and the request blocking probability when each server can accept and serve a finite number of concurrent requests. Our results show that the file download time drops when a request is served concurrently by a larger number of homogeneous replicated servers, although the performance improvement quickly saturates when the number of servers increases. If the total number of requests that a server can handle simultaneously is finite, the request blocking probability increases with the number of replicated servers used to serve a request concurrently. Therefore, paracasting is effective when a small number of servers, say, up to four, are used to serve a request concurrently.     

双层虚拟化云架构下任务调度能耗优化算法

虚拟机上部署容器的双层虚拟化云架构在云数据中心中的使用越来越广泛。为了解决该架构下云数据中心的能耗问题,提出了一种工作流任务调度算法TUMS-RTC。针对有截止时间约束的并行工作流,算法将调度过程划分为时间利用率最大化调度和运行时间压缩两个阶段。时间利用率最大化调度通过充分使用给定的时间范围减少完成工作流所需的虚拟机和服务器数量;运行时间压缩阶段通过压缩虚拟机空闲时间以缩短虚拟机和服务器的工作时间,最终达到降低能耗的目标。使用大量特征可控的随机工作流对TUMS-RTC算法的性能进行了测试。实验结果表明,TUMS-RTC算法相较于对比算法有更高的资源利用率,虚拟机数量减少率和能耗节省率,并且可以很好地处理云计算中规模大且并行度高的工作流。     

面向大规模云资源调度的可扩展分布式调度方法

云数据中心异构物理服务器的能耗优化资源分配问题是NP难的组合优化问题,当资源分配问题规模较大时,求解的空间比较大,很难在合理时间内求得最优解。基于分而治之的思想,从调度模式方面提出可扩展分布式调度方法,即当云数据中心待调度的物理服务器的数量比较大时,将待调度的服务器划分为若干个服务器集群,然后在每个服务器集群建立能耗优化的资源分配模型,并利用约束编程框架Choco求解模型,获得能耗最优的资源分配方式。将提出的基于可扩展分布式调度方法的能耗优化云资源调度算法与非扩展调度算法进行实验比较,实验结果表明,提出的基于可扩展分布式调度方法的能耗优化云资源调度算法在大规模云资源分配上有明显的性能优势。