改进粒子群算法在云计算负载均衡中的应用研究

云计算系统采用虚拟化技术可以更加灵活和高效地分配运算资源,便于管理员根据用户任务需求按需分配云计算资源。但虚拟化后的云计算中心存在种类多样、数量庞大的虚拟机资源,难以将虚拟机合理地放置到物理主机集群上并达到较好的负载均衡。为此,给出了云计算中心虚拟机放置到物理主机的负载均衡模型,采用改进后的粒子群算法（PSO）来求解最优解。最后通过和常用虚拟机放置算法的仿真对比实验,验证了所提云计算负载均衡优化算法的有效性。     

面向长作业环境中的云调度策略

随着云计算的普及,大量的数据处理选择云服务来完成。现有算法较少考虑异构型系统中虚拟机计算能力的不同,导致某些任务等待时间过长。提出了虚拟机负载大小实时调整的算法。对云计算中资源虚拟化特征,给出一种评估虚拟机计算能力的方法。根据虚拟机能力和运行过程中的状态变化,自适应进行任务量大小调整,满足实时要求。通过任务调度,协调任务完成时间,保持各虚拟机负载的动态均衡,缩短长作业的总执行时间,提高了系统的吞吐量和整体服务能力,提升了效益。实验结果表明,本文算法能自适应地调整任务量大小,进行调度,以维持虚拟机负载均衡。     

HPC Cloud环境中基于网络Ⅰ/O负载的虚拟机放置算法

随着虚拟化技术和云计算技术的发展,越来越多的高性能计算应用运行在云计算资源上.在基于虚拟化技术的高性能计算云系统中,高性能计算应用运行在多个虚拟机之中,这些虚拟机可能放置在不同的物理节点上.若多个通信密集型作业的虚拟机放置在相同的物理节点上,虚拟机之间将竞争物理节点的网络Ⅰ/O资源,如果虚拟机对网络Ⅰ/O资源的需求超过物理节点的网络Ⅰ/O带宽上限,将严重影响通信密集型作业的计算性能.针对虚拟机对网络Ⅰ/O资源的竞争问题,提出一种基于网络Ⅰ/O负载均衡的虚拟机放置算法NLPA,该算法采用网络Ⅰ/O负载均衡策略来减少虚拟机对网络Ⅰ/O资源的竞争.实验表明,与贪心算法进行比较,对于同样的高性能计算作业测试集,NLPA算法在完成作业的计算时间、系统中的网络Ⅰ/O负载吞吐率、网络Ⅰ/O负载均衡3个方面均有更好的表现.     

云环境下虚拟机集群系统动态负载均衡机制

针对传统的物理集群系统无法灵活应对大型互联网应用的问题,提出一种云环境下虚拟机集群的综合负载均衡机制。该方法首先定期地采集集群中虚拟机节点的CPU、内存、连接数、响应时间,以及所在物理主机的负载状况等指标信息,然后加权计算节点的综合负载并得出其权值,最后通过调度器进行任务请求的合理分配,从而解决了传统集群系统负载不均且不能适应多变的网络环境等诸多问题。实验结果表明,与加权轮询法(WRR)和加权最少连接法(WLC)调度方案相比,该机制能够在并发量较大时维持较低的响应时间,并能够根据集群中综合负载的状态实时地增加或减少虚拟机数量,通常在5s之内达到整体集群的负载均衡。     

Xen虚拟机资源调度优化算法

针对Xen虚拟化平台中虚拟机资源分配不合理的问题,提出了两种资源调度优化算法,即细粒度优化算法和粗粒度优化算法.细粒度优化算法主要解决单个物理节点上虚拟机资源分配不合理问题,能够根据物理节点上运行的各虚拟机的资源利用情况来调整资源分配量,适当增加利用率较高的虚拟机的资源,减少资源利用率低的虚拟机的资源,从而优化资源分配,提高资源利用效率,避免不必要的虚拟机迁移.粗粒度优化算法是针对集群中多个物理节点之间虚拟机负载不均衡问题而提出的.该算法结合粒子群优化技术,选择将集群系统中热点物理机上的部分虚拟机迁移到最适合的冷点物理机上,从而避免高载物理机宕机.实验结果表明,这两种资源调度优化算法能够有效解决虚拟机资源分配不合理的问题,具有较好的适用性和应用前景.     

基于MapReduce编程模型的节点失效处理与优化

云计算环境下对MapReduce编程模型和节点失效问题进行深入研究和实践.提出了使用基于失效规律的节点资源动态提供策略的方式,来解决任务服务器上点失效之后所带来的系统不能正常提供服务的问题,以此提高整个系统的高可用性,同时使服务器集群达到负载均衡的目的.进而实现基于MapReduce编程模型节点失效的优化.     

虚拟化环境中服务监控调度系统的设计与实现

针对虚拟化环境中因服务访问量的不同导致个别节点负载过大的现象,设计并实现了虚拟化环境中服务的监控调度系统,该系统主要包括对虚拟机的监控,对部署在虚拟机上的Web Services服务的监控以及对服务的迁移和调度。本文搭建了虚拟化实验环境并设计了整个系统的功能模块,实现了在个别虚拟机过载时,通过上报Web Services服务信息至调度中心,由调度中心决定服务迁移的目的节点,进行服务的迁移调度。     

KVM虚拟化动态迁移技术的安全防护模型

虚拟机动态迁移技术是在用户不知情的情况下使得虚拟机在不同宿主机之间动态地转移,保证计算任务的完成,具有负载均衡、解除硬件依赖、高效利用资源等优点,但此技术应用过程中会将虚拟机信息和用户信息暴露到网络通信中,其在虚拟化环境下的安全性成为广大用户担心的问题,逐渐成为学术界讨论和研究的热点问题.本文从研究虚拟化机制、虚拟化操作系统源代码出发,以虚拟机动态迁移的安全问题作为突破点,首先分析了虚拟机动态迁移时的内存泄漏安全隐患;其次结合KVM(Kernel-based Virtual Machine)虚拟化技术原理、通信机制、迁移机制,设计并提出一种新的基于混合随机变换编码方式的安全防护模型,该模型在虚拟机动态迁移时的迁出端和迁入端增加数据监控模块和安全模块,保证虚拟机动态迁移时的数据安全;最后通过大量实验,仿真测试了该模型的安全防护能力和对虚拟机运行性能的影响,仿真结果表明,该安全防护模型可以在KVM虚拟化环境下保证虚拟机动态迁移的安全,并实现了虚拟机安全性和动态迁移性能的平衡.     

TDDS:基于虚拟集群系统的任务部署与调度

利用虚拟机的灵活性和快速部署能力,设计并实现了任务部署与调度系统(task deployment and dispatch system,TDDS).TDDS能够根据用户的需求,为用户的计算任务提供可以进行资源配置的集群计算环境,满足了用户对不同操作系统、不同应用程序和不同计算资源的需求.TDDS还使用了负载均衡策略,以提高物理集群资源的利用率.提出了两种虚拟机部署策略,用以加快虚拟集群部署的速度.TDDS尽量控制虚拟机系统镜像的大小和访问频率,以提高部署的效率.实验表明,TDDS系统能够快速灵活地部署用户所需要的计算环境和计算资源,负载均衡的调度策略也切实提高了物理集群处理任务的能力,提高了集群的使用率.     

一种基于QoS的云负载均衡机制的研究

提出一种基于QoS的云负载均衡机制,即:构建QoS模型和云资源模型;建立资源度量与QoS属性之间的映射;对虚拟机实例负载状况和虚拟机集群资源利用状况进行量化评估;感知用户的QoS并对比所监控的云节点的资源度量情况,根据对比结果,通过任务调度算法和弹性伸缩算法分别实现任务的分发和虚拟机集群的弹性伸缩,最终达到优化的负载均衡的目的.通过模拟试验,结果表明本方法与Round robin算法相比,有更好的负载均衡效果.     

Effects of dynamic isolation for full virtualized RTOS and GPOS guests

Industrial systems currently include not only control processing with real-time operating system (RTOS) but also information processing with general-purpose operating system (GPOS). Multicore-based virtualization is an attractive option to provide consolidated environment when GPOS and RTOS are put in service on a single hardware platform. Researches on this technology have predominantly focused on the schedulability of RTOS virtual machines (VMs) by completely dedicated physical-CPUs (pCPUs) but have rarely considered parallelism or the throughput of the GPOS. However, it is also important that the multicore-based hypervisor adaptively selects pCPU assignment policy to efficiently manage resources in modern industrial systems. In this paper, we report our study on the effects of dynamic isolation when two mixed criticality systems are working on one platform. Based on our investigation of mutual interferences between RTOS VMs and GPOS VMs, we found explicit effects of dynamic isolation by special events. While maintaining low RTOS VMs scheduling latency, a hypervisor should manage pCPUs assignment by event-driven and threshold-based strategies to improve the throughput of GPOS VMs. Furthermore, we deal with implicit negative effects of dynamic isolation caused by the synchronization inside a GPOS VM, then propose a process of urgent boosting with dynamic isolation. All our methods are implemented in a real hypervisor, KVM. In experimental evaluation with benchmarks and an automotive digital cluster application, we analyzed that proposed dynamic isolation guarantees soft real-time operations for RTOS tasks while improving the throughput of GPOS tasks on a virtualized multicore system.     

Elastic management of web server clusters on distributed virtual infrastructures

Cluster‐based solutions are being widely adopted for implementing flexible, scalable, low‐cost and high‐performance web server platforms. One of the main difficulties to implement these platforms is the correct dimensioning of the cluster size, so as to satisfy variable and peak demand periods. In this context, virtualization is being adopted by many organizations as a solution not only to provide service elasticity, but also to consolidate server workloads, and improve server utilization rates. A virtualized web server can be dynamically adapted to the client demands by deploying new virtual nodes when the demand increases, and powering off and consolidating virtual nodes during periods of low demand. Furthermore, the resources from the in‐house infrastructure can be complemented with a cloud provider (cloud bursting), so that peak demand periods can be satisfied by deploying cluster nodes in the external cloud, on an on‐demand basis. In this paper, we analyze the scalability of hybrid virtual infrastructures for two different distributed web server cluster implementations: a simple web cluster serving static files and a multi‐tier web server platform running the CloudStone benchmark. Copyright © 2011 John Wiley & Sons, Ltd.     

Virtual machine cluster mobility in inter-cloud platforms

Modern cloud computing applications developed from different interoperable services that are interfacing with each other in a loose coupling approach. This work proposes the concept of the Virtual Machine (VM) cluster migration, meaning that services could be migrated to various clouds based on different constraints such as computational resources and better economical offerings. Since cloud services are instantiated as VMs, an application can be seen as a cluster of VMs that integrate its functionality. We focus on the VM cluster migration by exploring a more sophisticated method with regards to VM network configurations. In particular, networks are hard to managed because their internal setup is changed after a migration, and this is related with the configuration parameters during the re-instantiation to the new cloud platform. To address such issue, we introduce a Software Defined Networking (SDN) service that breaks the problem of network configuration into tractable pieces and involves virtual bridges instead of references to static endpoints. The architecture is modular, it is based on the SDN OpenFlow protocol and allows VMs to be paired in cluster groups that communicate with each other independently of the cloud platform that are deployed. The experimental analysis demonstrates migrations of VM clusters and provides a detailed discussion of service performance for different cases.     

一种灵活高效的虚拟CPU调度算法

目前,虚拟化已经广泛应用于数据中心,但主流的虚拟CPU调度策略并没有实现对I/O性能的保障,尤其当延时敏感型负载的虚拟机和计算敏感型负载的虚拟机竞争CPU资源时,其性能显著下降.针对上述问题,本文提出了一种灵活、高效的虚拟CPU调度算法（FLMS）.FLMS通过采用虚拟机分类、虚拟CPU绑定、多类时间片等技术降低了虚拟机的响应延时,同时基于多处理器架构重新设计了负载均衡策略,优化了虚拟CPU迁移.FLMS通用于目前主流的虚拟化方案,在软件虚拟化方式下相比于最新的优化方案延时降低了30%,带宽有10%的提升;在使用硬件辅助虚拟化的系统中,通过FLMS能够获得接近原生系统的I/O性能,并且保证了整个系统的公平性.     

Developing resource consolidation frameworks for moldable virtual machines in clouds

This paper considers the scenario where multiple clusters of Virtual Machines (i.e., termed Virtual Clusters) are hosted in a Cloud system consisting of a cluster of physical nodes. Multiple Virtual Clusters (VCs) cohabit in the physical cluster, with each VC offering a particular type of service for the incoming requests. In this context, VM consolidation, which strives to use a minimal number of nodes to accommodate all VMs in the system, plays an important role in saving resource consumption. Most existing consolidation methods proposed in the literature regard VMs as “rigid” during consolidation, i.e., VMs’ resource capacities remain unchanged. In VC environments, QoS is usually delivered by a VC as a single entity. Therefore, there is no reason why VMs’ resource capacity cannot be adjusted as long as the whole VC is still able to maintain the desired QoS. Treating VMs as “moldable” during consolidation may be able to further consolidate VMs into an even fewer number of nodes. This paper investigates this issue and develops a Genetic Algorithm (GA) to consolidate moldable VMs. The GA is able to evolve an optimized system state, which represents the VM-to-node mapping and the resource capacity allocated to each VM. After the new system state is calculated by the GA, the Cloud will transit from the current system state to the new one. The transition time represents overhead and should be minimized. In this paper, a cost model is formalized to capture the transition overhead, and a reconfiguration algorithm is developed to transit the Cloud to the optimized system state with low transition overhead. Experiments have been conducted to evaluate the performance of the GA and the reconfiguration algorithm.     

云计算平台中的虚拟化技术与安全机制

随着云计算的普及,其安全问题变得越来越重要。本文通过研究存储虚拟化和网络虚拟化技术,设计并部署了一个弹性和安全的虚拟化集群。该集群利用虚拟化技术把云计算用户空间在逻辑上相互隔离,有效地解决了共享性和安全性之间的矛盾。该集群也具有良好的可扩展性、高可用性和较低的成本。     

Flexible service consolidation with nested virtualization and library operating systems

In Infrastructure-as-a-service (IaaS) clouds, users can reduce costs by scale-in or scale-down when running services are underutilized. Since these optimizations of instance deployment require at least one minimum instance even for running an underutilized service, cost reduction is limited. For further optimization, multiple services can be consolidated into one instance. However, services have to be stopped temporarily at the consolidation time, and isolation between services becomes weaker after the consolidation. To solve these problems, this paper proposes FlexCapsule, which enables seamless and secure service consolidation in existing IaaS clouds. FlexCapsule runs each service in a lightweight virtual machine (VM) called an app VM, using a library operating system. An app VM runs inside an instance using a technique called nested virtualization. FlexCapsule can optimize instance deployment with negligible downtime by flexibly migrating app VMs. Due to strong isolation provided by app VMs, it can guarantee security between consolidated services. In addition, FlexCapsule provides multiprocess support using app VMs by emulating process fork and process pools. We have implemented FlexCapsule in Xen using both fully virtualized OS^v and paravirtualized MiniOS. Then, we examined the effectiveness of FlexCapsule using several applications. Due to the premature implementation of nested virtualization in Xen, the performance of app VMs largely degraded, but we believe that the performance could be improved using several existing optimizations.     

WOW: Self-organizing Wide Area Overlay Networks of Virtual Workstations

This paper describes WOW, a distributed system that combines virtual machine, overlay networking and peer-to-peer techniques to create scalable wide-area networks of virtual workstations for high-throughput computing. The system is architected to: facilitate the addition of nodes to a pool of resources through the use of system virtual machines (VMs) and self-organizing virtual network links; to maintain IP connectivity even if VMs migrate across network domains; and to present to end-users and applications an environment that is functionally identical to a local-area network or cluster of workstations. We describe IPOP, a network virtualization technique that builds upon a novel, extensible user-level decentralized technique to discover, establish and maintain overlay links to tunnel IP packets over different transports (including UDP and TCP) and across firewalls. We evaluate latency and bandwidth overheads of IPOP and also time taken for a new node to become fully-routable over the virtual network. We also report on several experiments conducted on a testbed WOW deployment with 118 P2P router nodes over PlanetLab and 33 VMware-based VM nodes distributed across six firewalled domains. Experiments show that the testbed delivers good performance for two unmodified, representative benchmarks drawn from the life-sciences domain. We also demonstrate that the system is capable of seamlessly maintaining connectivity at the virtual IP layer for typical client/server applications (NFS, SSH, PBS) when VMs migrate across a WAN.