云计算中虚拟机资源自动配置技术的研究

针对云资源管理者所面临的负载动态变化以及弹性资源需求等问题,提出一种虚拟机资源的自动配置管理技术,把强化学习技术引入云虚拟资源的管理,将虚拟机的配置管理过程建模为马尔可夫决策模型,根据系统的运行状态以及输入负载的动态变化自动决策添加或删除虚拟机的行为。实验结果验证了本技术能够根据负载的动态变化完成云虚拟资源的自动配置管理任务,及时响应终端用户的实时性任务请求,保证了云资源使用者的SLA需求。     

Energy-credit scheduler: An energy-aware virtual machine scheduler for cloud systems

Virtualization facilitates the provision of flexible resources and improves energy efficiency through the consolidation of virtualized servers into a smaller number of physical servers. As an increasingly essential component of the emerging cloud computing model, virtualized environments bill their users based on processor time or the number of virtual machine instances. However, accounting based only on the depreciation of server hardware is not sufficient because the cooling and energy costs for data centers will exceed the purchase costs for hardware. This paper suggests a model for estimating the energy consumption of each virtual machine without dedicated measurement hardware. Our model estimates the energy consumption of a virtual machine based on in-processor events generated by the virtual machine. Based on this estimation model, we also propose a virtual machine scheduling algorithm that can provide computing resources according to the energy budget of each virtual machine. The suggested schemes are implemented in the Xen virtualization system, and an evaluation shows that the suggested schemes estimate and provide energy consumption with errors of less than 5% of the total energy consumption.     

私有云平台的虚拟机内存调度策略

在私有云平台中,现有的方法无法灵活地对虚拟机内存资源进行有效的监控和分配。针对以上问题,提出了内存实时监测和动态调度(MMS)模型,利用libvirt函数库和Xen提供的libxc函数库实现了对虚拟机内存紧缺、内存空闲时的实时监测和动态调度,并且提出虚拟机迁移策略,有效地缓解宿主机的内存紧缺问题。最后选取一台物理机作为主控节点,两台物理机作为子节点,利用Eucalyptus搭建一个小型的私有云平台。结果显示,当宿主机处于内存紧缺状态时,MMS系统通过启动虚拟机迁移策略有效地释放了内存空间;当虚拟机占用内存逼近初始最大内存时,MMS为其分配新的最大内存;当占用内容降低时,MMS系统对部分空闲的内存资源进行了回收,而且释放内存不超过150MB(最大内存512MB)时,其对虚拟机性能的影响不大。结果表明该模型对私有云平台中虚拟机内存进行实时监测和动态调度是有效的。     

Network-aware virtual machine migration in an overcommitted cloud

Virtualization, which acts as the underlying technology for cloud computing, enables large amounts of third-party applications to be packed into virtual machines (VMs). VM migration enables servers to be reconsolidated or reshuffled to reduce the operational costs of data centers. The network traffic costs for VM migration currently attract limited attention.However, traffic and bandwidth demands among VMs in a data center account for considerable total traffic. VM migration also causes additional data transfer overhead, which would also increase the network cost of the data center.This study considers a network-aware VM migration (NetVMM) problem in an overcommitted cloud and formulates it into a non-deterministic polynomial time-complete problem. This study aims to minimize network traffic costs by considering the inherent dependencies among VMs that comprise a multi-tier application and the underlying topology of physical machines and to ensure a good trade-off between network communication and VM migration costs.The mechanism that the swarm intelligence algorithm aims to find is an approximate optimal solution through repeated iterations to make it a good solution for the VM migration problem. In this study, genetic algorithm (GA) and artificial bee colony (ABC) are adopted and changed to suit the VM migration problem to minimize the network cost. Experimental results show that GA has low network costs when VM instances are small. However, when the problem size increases, ABC is advantageous to GA. The running time of ABC is also nearly half than that of GA. To the best of our knowledge, we are the first to use ABC to solve the NetVMM problem.     

云计算环境下的虚拟机快速克隆技术

虚拟机克隆技术是指在云计算环境下快速复制出多个虚拟机（VM）并将这些VM分发到多台物理主机上,克隆出来的VM共享相同的初始状态然后独立运行提供服务。虚拟机克隆使得云计算提供商能够快速有效地部署系统资源。给出了一种虚拟机快速克隆方法,利用写时拷贝技术来创建虚拟磁盘和内存状态的快照,然后用按需分配内存技术和多点传送技术来请求和传输这些状态信息。在C3云平台上的实验表明,此方法在不中断源虚拟机中运行服务的情况下,实现了云计算中的快速虚拟机克隆。     

An imperialist competitive algorithm for virtual machine placement in cloud computing

Abstract

Cloud computing, the recently emerged revolution in IT industry, is empowered by virtualisation technology. In this paradigm, the user’s applications run over some virtual machines (VMs). The process of selecting proper physical machines to host these virtual machines is called virtual machine placement. It plays an important role on resource utilisation and power efficiency of cloud computing environment. In this paper, we propose an imperialist competitive-based algorithm for the virtual machine placement problem called ICA-VMPLC. The base optimisation algorithm is chosen to be ICA because of its ease in neighbourhood movement, good convergence rate and suitable terminology. The proposed algorithm investigates search space in a unique manner to efficiently obtain optimal placement solution that simultaneously minimises power consumption and total resource wastage. Its final solution performance is compared with several existing methods such as grouping genetic and ant colony-based algorithms as well as bin packing heuristic. The simulation results show that the proposed method is superior to other tested algorithms in terms of power consumption, resource wastage, CPU usage efficiency and memory usage efficiency.     

Smart elastic scheduling algorithm for virtual machine migration in cloud computing

The Journal of Supercomputing - Cloud Computing has the facility to transform a large part of information technology into services in which computer resources are virtualized and made available as...     

云环境中多目标优化的虚拟机放置算法

虚拟机放置（VMP）是虚拟机整合的核心,是一个多资源约束的多目标优化问题。高效的VMP算法不仅能显著地降低云数据中心能耗、提高资源利用率,还能保证服务质量（QoS）。针对数据中心能耗高和资源利用率低的问题,提出了基于离散蝙蝠算法的虚拟机放置（DBA-VMP）算法。首先,把最小化能耗和最大化资源利用率作为优化目标,建立多目标约束的VMP优化模型;然后,通过效仿人工蚁群在觅食过程中共享信息素的机制,将信息素反馈机制引入蝙蝠算法,并对经典蝙蝠算法进行离散化改进;最后,用改进的离散蝙蝠算法求解模型的Pareto最优解。实验结果表明,与其他多目标优化的VMP算法相比,所提算法在使用不同数据集的情况下都能有效降低能耗,提高资源利用率,实现了在保证QoS的前提下的降低能耗和提高资源利用率两者之间的优化平衡。     

Estimating null values in relational database systems using automatic clustering and multiple regression techniques

In this paper, we present a new method for estimating null values in relational database systems using automatic clustering and multiple regression techniques. First, we present a new automatic clustering algorithm for clustering numerical data. The proposed automatic clustering algorithm does not need to determine the number of clusters in advance and does not need to sort the data in the database in advance. Then, based on the proposed automatic clustering algorithm and multiple regression techniques, we present a new method to estimate null values in relational database systems. The proposed method estimating null values in relational database systems only needs to process a particular cluster instead of the whole database. It gets a higher average estimation accuracy rate than the existing methods for estimating null values in relational database systems.     

云计算环境下基于分形的聚类融合算法研究

传统的聚类算法不适用于处理海量和高维数据。针对云计算环境下,利用集群系统的并行计算能力,实现海量数据的聚类问题,给出了云计算环境下基于分形维数的聚类融合算法。该算法首先对基于分形维数的聚类算法进行改进,使之更适用于并行计算,其产生聚类作为初始聚类成员;再结合投票算法的融合策略实现融合。最后,对基于分形维数的聚类融合算法在云计算环境下实现并行计算。通过在UCI数据集上的对比实验来验证该算法的有效性。     

Detecting similarities in virtual machine behavior for cloud monitoring using smoothed histograms

The growing size and complexity of cloud systems determine scalability issues for resource monitoring and management. While most existing solutions consider each Virtual Machine (VM) as a black box with independent characteristics, we embrace a new perspective where VMs with similar behaviors in terms of resource usage are clustered together. We argue that this new approach has the potential to address scalability issues in cloud monitoring and management. In this paper, we propose a technique to cluster VMs starting from the usage of multiple resources, assuming no knowledge of the services executed on them. This innovative technique models VMs behavior exploiting the probability histogram of their resources usage, and performs smoothing-based noise reduction and selection of the most relevant information to consider for the clustering process. Through extensive evaluation, we show that our proposal achieves high and stable performance in terms of automatic VM clustering, and can reduce the monitoring requirements of cloud systems.     

FlatVC:云环境下虚拟机集群的扁平化版本控制

IaaS的发展使得云服务能够快速地部署虚拟机集群。然而,在部署过程中虚拟机群的版本控制效率不高。目前的版本控制方法存在网络负载大、操作速度慢的问题。提出一种新颖的虚拟机集群版本控制方法,叫做FlatVC。FlatVC在计算节点增量地生成虚拟机版本,以避免将版本数据传输至存储节点,并在虚拟机版本恢复时按需传输版本数据,因此减小了网络传输负载并加速了版本控制过程。通过使用缓存树结构来共享网络传输数据,FlatVC减小了根节点数据传输压力。此外,我们针对增量版本所构成的版本链进行了I/O优化,避免了版本链导致的性能下降。实验结果显示,FlatVC能有效地实施虚拟机集群版本控制,加速版本生成以及恢复过程。     

Dynamic forecast scheduling algorithm for virtual machine placement in cloud computing environment

In most cloud computing platforms, the virtual machine quotas are seldom changed once initialized, although the current allocated resources are not efficiently utilized. The average utilization of cloud servers in most datacenters can be improved through virtual machine placement optimization. How to dynamically forecast the resource usage becomes a key problem. This paper proposes a scheduling algorithm called virtual machine dynamic forecast scheduling (VM-DFS) to deploy virtual machines in a cloud computing environment. In this algorithm, through analysis of historical memory consumption, the most suitable physical machine can be selected to place a virtual machine according to future consumption forecast. This paper formalizes the virtual machine placement problem as a bin-packing problem, which can be solved by the first-fit decreasing scheme. Through this method, for specific virtual machine requirements of applications, we can minimize the number of physical machines. The VM-DFS algorithm is verified through the CloudSim simulator. Our experiments are carried out on different numbers of virtual machine requests. Through analysis of the experimental results, we find that VM-DFS can save 17.08 % physical machines on the average, which outperforms most of the state-of-the-art systems.     

A multi-objective Monarch Butterfly Algorithm for virtual machine placement in cloud computing

Neural Computing and Applications - The growing demand for cloud computing adoption presents more challenges for researchers to make cloud computing more efficient and affordable for infrastructure...     

Energy-efficiency enhanced virtual machine scheduling policy for mixed workloads in cloud environments

Virtualization technology is an effective approach to improving the energy-efficiency in cloud platforms; however, it also introduces many energy-efficiency losses especially when I/O virtualization is involved. In this paper, we present an energy-efficiency enhanced virtual machine (VM) scheduling policy, namely Share-Reclaiming with Collective I/O (SRC-I/O), with aiming at reducing the energy-efficiency losses caused by I/O virtualization. The proposed SRC-I/O scheduler allows VMs to reclaim extra CPU shares in certain conditions so as to increase CPU utilization. Meanwhile, it separates I/O-intensive VMs from CPU-intensive ones and schedules them in a collective manner, so as to reduce the context-switching cost when scheduling mixed workloads. Extensive experiments are conducted on various platforms to investigate the performance of the proposed scheduler. The results indicate that when the system is in presence of mixed workloads, SRC-I/O scheduler outperforms many existing VM schedulers in terms of energy-efficiency and I/O responsiveness.     

Layered virtual machine migration algorithm for network resource balancing in cloud computing

Due to the increasing sizes of cloud data centers, the number of virtual machines (VMs) and applications rises quickly. The rapid growth of large scale Internet services results in unbalanced load of network resource. The bandwidth utilization rate of some physical hosts is too high, and this causes network congestion. This paper presents a layered VM migration algorithm (LVMM). At first, the algorithm will divide the cloud data center into several regions according to the bandwidth utilization rate of the hosts. Then we balance the load of network resource of each region by VM migrations, and ultimately achieve the load balance of network resource in the cloud data center. Through simulation experiments in different environments, it is proved that the LVMMalgorithm can effectively balance the load of network resource in cloud computing.     

私有云平台上的虚拟机进程安全检测

针对网络防火墙在私有云平台安全防护上的单调与缺陷, 提出了一种基于进程资源监控的安全监测方法(PAMon)。首先利用虚拟机监控器获取平台上虚拟机的物理资源信息; 然后通过映射表重构进程资源信息; 再对重构的进程信息从关键进程、进程隐藏和进程占用资源异常三方面分析恶意进程; 最后对分析出的恶意进程进行了适当的处理。实验结果表明, PAMon不仅可以有效地检测出恶意程序, 而且反馈给防火墙的信息可以进一步增强网络防火墙的防御能力。     

云数据中心基于贪心算法的虚拟机迁移策略

为了节省云数据中心的能量消耗,提出了几种基于贪心算法的虚拟机（VM）迁移策略。这些策略将虚拟机迁移过程划分为物理主机状态检测、虚拟机选择和虚拟机放置三个步骤,并分别在虚拟机选择和虚拟机放置步骤中采用贪心算法予以优化。提出的三种迁移策略分别为：最小主机使用效率选择且最大主机使用效率放置算法MinMax_Host_Utilization、最大主机能量使用选择且最小主机能量使用放置算法MaxMin_Host_Power_Usage、最小主机计算能力选择且最大主机计算能力放置算法MinMax_Host_MIPS。针对物理主机处理器使用效率、物理主机能量消耗、物理主机处理器计算能力等指标设置最高或者最低的阈值,参考贪心算法的原理,在指标上超过或者低于这些阈值范围的虚拟机都将进行迁移。利用CloudSim作为云数据中心仿真环境的测试结果表明,基于贪心算法的迁移策略与CloudSim中已存在的静态阈值迁移策略和绝对中位差迁移策略比较起来,总体能量消耗少15%,虚拟机迁移次数少60%,平均SLA违规率低5%。     

高效能云计算虚拟机优化部署策略

针对云计算应用负载需求的动态变化特性,提出了一种自适应虚拟机优化部署策略。算法通过基于强局部加权回归的热点发现机制,可以根据负载所体现的资源占用历史信息动态决策主机的超载时机;通过迁移周期最优算法MPM和迁移量最少算法MNM进行超载主机的迁移虚拟机选择;提出基于功耗感知的PBFDH算法对迁移虚拟机再次优化部署。实验结果表明,算法不仅可以降低能耗,还可以降低SLA违例率。