一种基于聚类分组的虚拟机镜像去冗余方法

随着云计算的兴起,虚拟化技术使用也越来越广泛,虚拟机正逐步取代物理机,成为应用服务的部署环境.出于灵活性、可靠性等方面的需求,虚拟机镜像急剧增长,如何高效地、经济地管理这些镜像文件已成为一个很有挑战性的研究热点.由于虚拟机镜像之间存在大量重复性的数据块,高效的去冗余方法对于虚拟机镜像管理至关重要.然而,传统的去冗余方法由于需要巨大的资源开销,会对平台中托管的虚拟机性能造成干扰,因而并不适用于云环境.提出了一种局部去冗余的方法,旨在优化镜像去冗余过程.其核心思想是:将全局去冗余变成局部去冗余,从而降低去冗余算法的空间复杂度,以达到减少操作时间的目的.该方法利用虚拟机镜像相似性作为启发式规则对虚拟机镜像进行分组,当一个新的镜像到来时,通过统计抽样的方法为镜像选取最为相似的分组进行去冗余.实验结果表明:该方法可以通过牺牲1%左右的存储空间,缩短50%以上的去冗余操作时间.     

面向长作业环境中的云调度策略

随着云计算的普及,大量的数据处理选择云服务来完成。现有算法较少考虑异构型系统中虚拟机计算能力的不同,导致某些任务等待时间过长。提出了虚拟机负载大小实时调整的算法。对云计算中资源虚拟化特征,给出一种评估虚拟机计算能力的方法。根据虚拟机能力和运行过程中的状态变化,自适应进行任务量大小调整,满足实时要求。通过任务调度,协调任务完成时间,保持各虚拟机负载的动态均衡,缩短长作业的总执行时间,提高了系统的吞吐量和整体服务能力,提升了效益。实验结果表明,本文算法能自适应地调整任务量大小,进行调度,以维持虚拟机负载均衡。     

Intelligent heart disease prediction in cloud environment through ensembling

Cloud computing is the delivery of on‐demand computing resources. Cloud computing has numerous applications in fields of education, social networking, and medicine. But the benefit of cloud for medical purposes is seamless, particularly because of the enormous data generated by the health care industry. This colossal data can be managed through big data analytics, and hidden patterns can be extracted using machine learning procedures. In particular, the latest issue in the medical domain is the prediction of heart diseases, which can be resolved through culmination of machine learning and cloud computing. Hence, an attempt has been made to propose an intelligent decision support model that can aid medical experts in predicting heart disease based on the historical data of patients. Various machine learning algorithms have been implemented on the heart disease dataset to predict accuracy for heart disease. Naïve Bayes has been selected as an effective model because it provides the highest accuracy of 86.42% followed by AdaBoost and boosted tree. Further, these 3 models are being ensembled, which has increased the overall accuracy to 87.91%. The experimental results have also been evaluated using 10,082 instances that clearly validate the maximum accuracy through ensembling and minimum execution time in cloud environment.     

A security-aware virtual machine placement in the cloud using hesitant fuzzy decision-making processes

The introduction of cloud computing systems brought with itself a solution for the dynamic scaling of computing resources leveraging various approaches for providing computing power, networking, and storage. On the other hand, it helped decrease the human resource cost by delegating the maintenance cost of infrastructures and platforms to the cloud providers. Nevertheless, the security risks of utilizing shared resources are recognized as one of the major concerns in using cloud computing environments. To be more specific, an intruder can attack a virtual machine and consequently extend his/her attack to other virtual machines that are co-located on the same physical machine. The worst situation is when the hypervisor is compromised in which all the virtual machines assigned to the physical node will be under security risk. To address these issues, we have proposed a security-aware virtual machine placement scheme to reduce the risk of co-location for vulnerable virtual machines. Four attributes are introduced to reduce the aforementioned risk including the vulnerability level of a virtual machine, the importance level of a virtual machine in the given context, the cumulative vulnerability level of a physical machine, and the capacity of a physical machine for the allocation of new virtual machines. Nevertheless, the evaluation of security risks, due to the various vulnerabilities’ nature as well as the different properties of deployment environments is not quite accurate. To manage the precision of security evaluations, it is vital to consider hesitancy factors regarding security evaluations. To consider hesitancy in the proposed method, hesitant fuzzy sets are used. In the proposed method, the priorities of the cloud provider for the allocation of virtual machines are also considered. This will allow the model to assign more weights to attributes that have higher importance for the cloud provider. Eventually, the simulation results for the devised scenarios demonstrate that the proposed method can reduce the overall security risk of the given cloud data center. The results show that the proposed approach can reduce the risk of attacks caused by the co-location of virtual machines up to 41% compared to the existing approaches.

     

Using distributed raspberry PIs to enable low-cost energy-efficient machine learning algorithms for scientific articles recommendation

In recent decades, machine learning has become a crucial factor in terms of automating business operations and assisting in the decision-making process within an organization. With the huge volume of data generated at an unprecedented rate has motivated researchers and industry analysts to constantly develop effective and efficient analytical models machine learning techniques. This study adds to the data mining community by evaluating some of the most significant text mining techniques and presenting a predictive model that will supposedly ease the process of literature review for researchers. In addition, it compares the execution of the model in terms of cost, energy consumption, accuracy and scalability in three different environments, namely, google cloud instance, google cloud functions and distributed raspberry PIs. Results yielded in our study showed that distributed Raspberry PIs can have a highly positive impact in terms of lowering costs and being energy efficient. On one hand, we found out that machine learning algorithms can be adapted and run on distributed raspberry PIs with low cost and low energy consumption compared to cloud alternatives. On the other hand, this solution does not offer great scalability and it requires more time on management, deployment and configuration. The distributed Raspberry PIs also showed bad performance on execution time compared to the other alternatives when comes to high processing power.     

ICARO Cloud Simulator exploiting knowledge base

Allocation changes on cloud are complex and time consuming tasks, on cloning, scaling, etc. A solution to cope with these aspects is to perform a simulation. Cloud simulators have been proposed to assess conditions adopting specific models for energy, cloud capacity, allocations, networking, security, etc. In this paper, ICARO Cloud Simulator is proposed. It has been specifically designed for simulating the workload on the basis of real virtual machine workloads and for simulating complex business configurations and behaviours for wide temporal windows. This approach can be useful to predict and simulate the allocation of virtual machines on hosts and, thus, data centers on the basis of real business configuration behaviour for days, weeks, months, etc. (for example, to predict workloads). The proposed research has been developed in the context of the ICARO Cloud research and development project.     

Simplified cloud-oriented virtual machine management with MLN

System administrators are faced with the challenge of making their existing systems power-efficient and scalable. Although cloud computing is offered as a solution to this challenge by many, we argue that having multiple interfaces and cloud providers can result in more complexity than before. This paper addresses cloud computing from a user perspective. We show how complex scenarios, such as an on-demand render farm and scaling web-service, can be achieved utilizing clouds but at the same time keeping the same management interface as for local virtual machines. Further, we demonstrate that by enabling the virtual machine to have its policy locally instead of in the underlying framework, it can move between otherwise incompatible cloud providers and sites in order to achieve its goals more efficiently.     

云计算环境下的自适应资源管理技术综述

云环境下的自适应资源管理是当前云计算研究领域的热点问题,是云计算具备弹性扩展、动态分配和资源共享等特点的关键技术支撑,具有重要的理论意义和实用价值.其主要研究点包括:虚拟机放置优化算法,虚拟资源动态伸缩模型、多IDC间的全局云计算资源调度、全局资源配置及能力规划模型等.对云环境下自适应资源管理研究现状进行分析研究,并指出当前研究中存在的一些主要问题,同时进一步展望本领域未来的研究方向.     

Application Layer Scheduling in Cloud: Fundamentals,Review and Research Directions

The cloud computing paradigm facilitates a finite pool of on-demand virtualized resources on a pay-per-use basis. For large-scale heterogeneous distributed systems like a cloud, scheduling is an essential component of resource management at the application layer as well as at the virtualization layer in order to deliver the optimal Quality of Services (QoS). The cloud scheduling, in general, is an NP-hard problem due to large solution space, thus, it is difficult to find an optimal solution within a reasonable time. In application layer scheduling, the tasks are mapped to logical resources (i.e., virtual machines), aiming to optimize one or more QoS parameters, and conforming to several constraints. Various algorithms have been proposed in the literature for application layer scheduling, where each of them is based on some fundamental design techniques like simple heuristics, meta-heuristics, and most recently hybrid heuristics. Although ample literature survey exists for cloud scheduling algorithms, none of them present their study exclusively for the application layer. In this survey paper, we present a study on task scheduling algorithms used only at the application layer of the cloud. We classify our study according to various fundamental techniques used in designing such scheduling algorithms. One of the main features of our study is that it covers numerous application type e.g., a set of independent tasks, simple workflow, scientific workflow, and MapReduce jobs. We also provide a comparative analysis of existing algorithms on various parameters like makespan, cost, resource utilization, etc. In the end, research directions for future work have been provided.     

Achieving middleware execution efficiency: hardware-assisted garbage collection operations

Although virtualization technologies bring many benefits to cloud computing environments, as the virtual machines provide more features, the middleware layer has become bloated, introducing a high overhead. Our ultimate goal is to provide hardware-assisted solutions to improve the middleware performance in cloud computing environments. As a starting point, in this paper, we design, implement, and evaluate specialized hardware instructions to accelerate GC operations. We select GC because it is a common component in virtual machine designs and it incurs high performance and energy consumption overheads. We performed a profiling study on various GC algorithms to identify the GC performance hotspots, which contribute to more than 50% of the total GC execution time. By moving these hotspot functions into hardware, we achieved an order of magnitude speedup and significant improvement on energy efficiency. In addition, the results of our performance estimation study indicate that the hardware-assisted GC instructions can reduce the GC execution time by half and lead to a 7% improvement on the overall execution time.     

基于动态虚拟机模型的云计算调度方法

将虚拟机加入云计算环境,可充分利用云计算的资源共享优势及其并行、分布计算功能;提出了一种可根据需要动态添加或删除虚拟机的模型系统,可有效节约云计算的使用费用,提高成本效率;研究了可用于本模型系统的两种资源调度算法——自适应先到先得（Adaptive First Come First Serve,AFCFS）和最大者优先（Largest Job First Served,LJFS）算法,尽量避免不必要的延迟,最大可能地提高系统性能,因为这对于分布式系统资源调度算法十分重要;模拟实验中采用了响应时间、等待时间、到达率等性能指标及性价比这一成本指标,比较了几种算法的性能效率,研究验证了模型系统的成本效率。实验结果表明几种算法可高效地运用于云计算环境,并能提高系统性能效率和成本效率。     

基于IFWA-ABC的云计算资源任务调度算法的研究

针对云计算资源任务调度效率低,资源分配不均的情况,将改进的烟花算法和人工蜂群算法算法进行融合为IFWA-ABC。首先,对云计算资源任务调度进行描述;其次,在FWA初始化中采用混沌反向学习和柯西分布进行优化,对核心烟花和非核心烟花的半径分别进行优化,将FWA中最优个体通过改进的ABC算法进行获得;最后,将IFWA-ABC算法用于云计算任务调度。仿真实验中,通过与FWA、ABC在虚拟机、执行时间、消耗成本、能量消耗指标对比中,IFWA-ABC具有明显的优势能够有效地提高云计算资源分配效率。     

A delay-based dynamic scheduling algorithm for bag-of-task workflows with stochastic task execution times in clouds

Bag-of-Tasks (BoT) workflows are widespread in many big data analysis fields. However, there are very few cloud resource provisioning and scheduling algorithms tailored for BoT workflows. Furthermore, existing algorithms fail to consider the stochastic task execution times of BoT workflows which leads to deadline violations and increased resource renting costs. In this paper, we propose a dynamic cloud resource provisioning and scheduling algorithm which aims to fulfill the workflow deadline by using the sum of task execution time expectation and standard deviation to estimate real task execution times. A bag-based delay scheduling strategy and a single-type based virtual machine interval renting method are presented to decrease the resource renting cost. The proposed algorithm is evaluated using a cloud simulator ElasticSim which is extended from CloudSim. The results show that the dynamic algorithm decreases the resource renting cost while guaranteeing the workflow deadline compared to the existing algorithms.     

A Pliant-based Virtual Machine Scheduling Solution to Improve the Energy Efficiency of IaaS Clouds

Recently cloud computing is facing increasing attention as it is applied in many business scenarios by advertising the illusion of infinite resources towards its customers. Nevertheless, it raises severe issues with energy consumption: the higher levels of quality and availability require irrational energy expenditures. This paper proposes Pliant system-based virtual machine scheduling approaches for reducing the energy consumption of cloud datacenters. We have designed a CloudSim-based simulation environment for task-based cloud applications to evaluate our proposed solution, and applied industrial workload traces for our experiments. We show that significant savings can be achieved in energy consumption by our proposed Pliant-based algorithms, in this way a beneficial trade-off can be reached by IaaS providers between energy consumption and execution time.     

CyberGuarder: A virtualization security assurance architecture for green cloud computing

As the sizes of IT infrastructure continue to grow, cloud computing is a natural extension of virtualisation technologies that enable scalable management of virtual machines over a plethora of physically connected systems. The so-called virtualisation-based cloud computing paradigm offers a practical approach to green IT/clouds, which emphasise the construction and deployment of scalable, energy-efficient network software applications (NetApp) by virtue of improved utilisation of the underlying resources. The latter is typically achieved through increased sharing of hardware and data in a multi-tenant cloud architecture/environment and, as such, accentuates the critical requirement for enhanced security services as an integrated component of the virtual infrastructure management strategy. This paper analyses the key security challenges faced by contemporary green cloud computing environments, and proposes a virtualisation security assurance architecture, CyberGuarder, which is designed to address several key security problems within the ‘green’ cloud computing context. In particular, CyberGuarder provides three different kinds of services; namely, a virtual machine security service, a virtual network security service and a policy based trust management service. Specifically, the proposed virtual machine security service incorporates a number of new techniques which include (1) a VMM-based integrity measurement approach for NetApp trusted loading, (2) a multi-granularity NetApp isolation mechanism to enable OS user isolation, and (3) a dynamic approach to virtual machine and network isolation for multiple NetApp’s based on energy-efficiency and security requirements. Secondly, a virtual network security service has been developed successfully to provide an adaptive virtual security appliance deployment in a NetApp execution environment, whereby traditional security services such as IDS and firewalls can be encapsulated as VM images and deployed over a virtual security network in accordance with the practical configuration of the virtualised infrastructure. Thirdly, a security service providing policy based trust management is proposed to facilitate access control to the resources pool and a trust federation mechanism to support/optimise task privacy and cost requirements across multiple resource pools. Preliminary studies of these services have been carried out on our iVIC platform, with promising results. As part of our ongoing research in large-scale, energy-efficient/green cloud computing, we are currently developing a virtual laboratory for our campus courses using the virtualisation infrastructure of iVIC, which incorporates the important results and experience of CyberGuarder in a practical context.     

Energy-aware Virtual Machine Migration for Cloud Computing - A Firefly Optimization Approach

Energy efficiency has grown into a latest exploration area of virtualized cloud computing paradigm. The increase in the number and the size of the cloud data centers has propagated the need for energy efficiency. An extensively practiced technology in cloud computing is live virtual machine migration and is thus focused in this work to save energy. This paper proposes an energy-aware virtual machine migration technique for cloud computing, which is based on the Firefly algorithm. The proposed technique migrates the maximally loaded virtual machine to the least loaded active node while maintaining the performance and energy efficiency of the data centers. The efficacy of the proposed technique is exhibited by comparing it with other techniques using the CloudSim simulator. An enhancement in the average energy consumption of about 44.39 % has been attained by reducing an average of 72.34 % of migrations and saving 34.36 % of hosts, thereby, making the data center more energy-aware.     

基于负载预测的虚拟机动态调度算法研究与实现

在云计算系统中为了实现负载均衡和资源的高效利用,需要在虚拟机粒度上对云计算系统进行调度,通过热迁移技术将虚拟机从高负载物理节点迁移到低负载物理节点。把负载预测技术和虚拟机动态调度技术相结合,提出了LFS算法,通过虚拟机历史负载数据对虚拟机未来的负载变化情况进行预测,然后根据预测结果对虚拟机进行调度,能够有效地避免云计算系统中高负载物理节点出现,实现负载均衡,提高资源使用率。     

Secure multi-server-aided data deduplication in cloud computing

Cloud computing enables on-demand and ubiquitous access to a centralized pool of configurable resources such as networks, applications, and services. This makes that huge of enterprises and individual users outsource their data into the cloud server. As a result, the data volume in the cloud server is growing extremely fast. How to efficiently manage the ever-increasing datum is a new security challenge in cloud computing. Recently, secure deduplication techniques have attracted considerable interests in the both academic and industrial communities. It can not only provide the optimal usage of the storage and network bandwidth resources of cloud storage providers, but also reduce the storage cost of users. Although convergent encryption has been extensively adopted for secure deduplication, it inevitably suffers from the off-line brute-force dictionary attacks since the message usually can be predictable in practice. In order to address the above weakness, the notion of DupLESS was proposed in which the user can generate the convergent key with the help of a key server. We argue that the DupLESS does not work when the key server is corrupted by the cloud server. In this paper, we propose a new multi-server-aided deduplication scheme based on the threshold blind signature, which can effectively resist the collusion attack between the cloud server and multiple key servers. Furthermore, we prove that our construction can achieve the desired security properties.     

基于Spark的分布式大数据机器学习算法

对于大数据而言,机器学习技术是不可或缺的;对于机器学习而言,大规模的数据可以提升模型的精准度。然而复杂的机器学习算法从时间和性能上都急需分布式内存计算这种关键技术。Spark分布式内存计算可以实现算法的并行操作,有利于机器学习算法处理大数据集。因此本文提出在Spark分布式内存环境下实现非线性机器学习算法,其中包括多层可变神经网络、BPPGD SVM、K-means,并在实现的基础上进行数据压缩、数据偏向抽样或者数据加载等方面的优化。为了实现充分配置资源批量运行脚本,本文也实现SparkML调度框架来调度以上优化算法。实验结果表明,优化后的3种算法平均误差降低了40%,平均时间缩短了90%。     

Efficient image restoration of virtual machines with reference count based rewriting and caching

Virtual machine (VM) image backups have duplicate data blocks distributed in different physical addresses, which occupy a large amount of storage space in a cloud computing platform (Choo et al.,  [1] and González-Manzano et al.,  [2]). Deduplication is a widely used technology to reduce the redundant data in a VM backup process. However, deduplication always causes the fragmentation of data blocks, which seriously affects the VM restoration performance. Current approaches often rewrite data blocks to accelerate image restoration, but rewriting could cause significant performance overhead because of frequent I/O operations. To address this issue, we have found that the reference count is a key to the fragmentation degree from a series of experiments. Thus, we propose a reference count based rewriting method to defragment VM image backups, and a caching method based on the distribution of rewritten data blocks to restore VM images. Compared with existing studies, our approach has no interfere to the deduplication process, needs no extra storage, and efficiently improves the performance of VM image restoration. We have implemented a prototype to evaluate our approach in our real cloud computing platform OnceCloud. Experimental results show that our approach can reduce about 57% of the dispersion degree of data blocks, and accelerate about 51% of the image restoration of virtual machines.