DAC: Improving storage availability with Deduplication-Assisted Cloud-of-Clouds

With the increasing popularity and rapid development of the cloud storage technology, more and more users are beginning to upload their data to the cloud storage platform. However, solely depending on a particular cloud storage provider has a number of potentially serious problems, such as vendor lock-in, availability and security. To address these problems, we propose a Deduplication-Assisted primary storage system in Cloud-of-Clouds (short for DAC) in this paper. DAC eliminates the redundant data blocks in the cloud computing environment and distributes the data among multiple independent cloud storage providers by exploiting the data reference characteristics. In DAC, the data blocks are stored in multiple cloud storage providers by combing the replication and erasure code schemes. To better utilize the advantages of both replication and erasure code schemes and exploit the reference characteristics in data deduplication, the high referenced data blocks are stored with the replication scheme while the other data blocks are stored with the erasure code scheme. The experiments conducted on our lightweight prototype implementation show that DAC improves the performance and cost efficiency significantly, compared with the existing schemes.     

一种基于聚类分组的虚拟机镜像去冗余方法

随着云计算的兴起,虚拟化技术使用也越来越广泛,虚拟机正逐步取代物理机,成为应用服务的部署环境.出于灵活性、可靠性等方面的需求,虚拟机镜像急剧增长,如何高效地、经济地管理这些镜像文件已成为一个很有挑战性的研究热点.由于虚拟机镜像之间存在大量重复性的数据块,高效的去冗余方法对于虚拟机镜像管理至关重要.然而,传统的去冗余方法由于需要巨大的资源开销,会对平台中托管的虚拟机性能造成干扰,因而并不适用于云环境.提出了一种局部去冗余的方法,旨在优化镜像去冗余过程.其核心思想是:将全局去冗余变成局部去冗余,从而降低去冗余算法的空间复杂度,以达到减少操作时间的目的.该方法利用虚拟机镜像相似性作为启发式规则对虚拟机镜像进行分组,当一个新的镜像到来时,通过统计抽样的方法为镜像选取最为相似的分组进行去冗余.实验结果表明:该方法可以通过牺牲1%左右的存储空间,缩短50%以上的去冗余操作时间.     

Recent advancement in VM task allocation system for cloud computing: review from 2015 to2021

Cloud computing is new technology that has considerably changed human life at different aspect over the last decade. Especially after the COVID-19 pandemic, almost all life activity shifted into cloud base. Cloud computing is a utility where different hardware and software resources are accessed on pay per user ground base. Most of these resources are available in virtualized form and virtual machine (VM) is one of the main elements of visualization.VM used in data center for distribution of resource and application according to benefactor demand. Cloud data center faces different issue in respect of performance and efficiency for improvement of these issues different approaches are used. Virtual machine play important role for improvement of data center performance therefore different approach are used for improvement of virtual machine efficiency (i-e) load balancing of resource and task. For the improvement of this section different parameter of VM improve like makespan, quality of service, energy, data accuracy and network utilization. Improvement of different parameter in VM directly improve the performance of cloud computing. Therefore, we conducting this review paper that we can discuss about various improvements that took place in VM from 2015 to 20,201. This review paper also contain information about various parameter of cloud computing and final section of paper present the role of machine learning algorithm in VM as well load balancing approach along with the future direction of VM in cloud data center.

     

Secure multi-server-aided data deduplication in cloud computing

Cloud computing enables on-demand and ubiquitous access to a centralized pool of configurable resources such as networks, applications, and services. This makes that huge of enterprises and individual users outsource their data into the cloud server. As a result, the data volume in the cloud server is growing extremely fast. How to efficiently manage the ever-increasing datum is a new security challenge in cloud computing. Recently, secure deduplication techniques have attracted considerable interests in the both academic and industrial communities. It can not only provide the optimal usage of the storage and network bandwidth resources of cloud storage providers, but also reduce the storage cost of users. Although convergent encryption has been extensively adopted for secure deduplication, it inevitably suffers from the off-line brute-force dictionary attacks since the message usually can be predictable in practice. In order to address the above weakness, the notion of DupLESS was proposed in which the user can generate the convergent key with the help of a key server. We argue that the DupLESS does not work when the key server is corrupted by the cloud server. In this paper, we propose a new multi-server-aided deduplication scheme based on the threshold blind signature, which can effectively resist the collusion attack between the cloud server and multiple key servers. Furthermore, we prove that our construction can achieve the desired security properties.     

Secure proof of storage with deduplication for cloud storage systems

Explosion of multimedia content brings forth the needs of efficient resource utilization using the state of the arts cloud computing technologies such as data deduplication. In the cloud computing environments, achieving both data privacy and integrity is the challenging issue for data outsourcing service. Proof of Storage with Deduplication (POSD) is a promising solution that addresses the issue for the cloud storage systems with deduplication enabled. However, the validity of the current POSD scheme stands on the strong assumption that all clients are honest in terms of generating their keys. We present insecurity of this approach under new attack model that malicious clients exploit dishonestly manipulated keys. We also propose an improved POSD scheme to mitigate our attack.     

Anomaly detection and identification scheme for VM live migration in cloud infrastructure

Virtual machines (VM) offer simple and practical mechanisms to address many of the manageability problems of leveraging heterogeneous computing resources. VM live migration is an important feature of virtualization in cloud computing: it allows administrators to transparently tune the performance of the computing infrastructure. However, VM live migration may open the door to security threats. Classic anomaly detection schemes such as Local Outlier Factors (LOF) fail in detecting anomalies in the process of VM live migration. To tackle such critical security issues, we propose an adaptive scheme that mines data from the cloud infrastructure in order to detect abnormal statistics when VMs are migrated to new hosts. In our scheme, we extend classic Local Outlier Factors (LOF) approach by defining novel dimension reasoning (DR) rules as DR-LOF to figure out the possible sources of anomalies. We also incorporate Symbolic Aggregate ApproXimation (SAX) to enable timing information exploration that LOF ignores. In addition, we implement our scheme with an adaptive procedure to reduce chances of performance instability. Compared with LOF that fails in detecting anomalies in the process of VM live migration, our scheme is able not only to detect anomalies but also to identify their possible sources, giving cloud computing operators important clues to pinpoint and clear the anomalies. Our scheme further outperforms other classic clustering tools in WEKA (Waikato Environment for Knowledge Analysis) with higher detection rates and lower false alarm rate. Our scheme would serve as a novel anomaly detection tool to improve security framework in VM management for cloud computing.     

An improved secure file deduplication avoidance using CKHO based deep learning model in a cloud environment

Data deduplication is a process that gets rid of excessive duplicates of data and minimizes the storage capacity to a large extent. This process mainly optimizes redundancies without compromising the data fidelity or integrity. However, the major challenge faced by most data deduplication systems is secure cloud storage. Cloud computing relies on the ability and security of all information. In the case of distributed storage, data protection and security are critical. This paper presents a Secure Cloud Framework for owners to effectively handle cloud-based information and provide high security for information (SCF). Weaknesses, Cross-Site Scripting (XSS), SQL perfusion, adverse processing, and wrapping are all examples of significant attacks in the cloud. This paper proposes an improved Secure File Deduplication Avoidance (SFDA) algorithm for block-level deduplication and security. The deduplication process allows cloud customers to adequately manage the distributed storage space by avoiding redundant information and saving transfer speed. A deep learning classifier is used to distinguish the familiar and unfamiliar data. A dynamic perfect hashing scheme is used in the SFDA approach to perform convergent encryption and offer secure storage. The Chaotic krill herd optimization (CKHO) algorithm is used for the optimal secret key generation process of the Advanced Encryption Standard (AES) algorithm. In this way, the unfamiliar data are encrypted one more time and stored in the cloud. The efficiency of the results is demonstrated via the experiments conducted in terms of computational cost, communication overhead, deduplication rate, and attack level. For file sizes of 8 MB, 16 MB, 32 MB, and 64 MB, the proposed methodology yields a deduplication rate of 53%, 62%, 54%, and 44%, respectively. The dynamic perfect hashing and the optimal key generation using the CKHO algorithm minimizes the data update time and the time taken to update a total of 1024 MB data is 341.5 ms. The improved SFDA algorithm's optimal key selection approach reduces the impact of an attack by up to 12% for a data size of 50 MB, whereas the existing system is mostly impacted by data size, and its attack level rises by up to 19 percent for the same data size.

     

A new content-defined chunking algorithm for data deduplication in cloud storage

Chunking is a process to split a file into smaller files called chunks. In some applications, such as remote data compression, data synchronization, and data deduplication, chunking is important because it determines the duplicate detection performance of the system. Content-defined chunking (CDC) is a method to split files into variable length chunks, where the cut points are defined by some internal features of the files. Unlike fixed-length chunks, variable-length chunks are more resistant to byte shifting. Thus, it increases the probability of finding duplicate chunks within a file and between files. However, CDC algorithms require additional computation to find the cut points which might be computationally expensive for some applications. In our previous work (Widodo et al., 2016), the hash-based CDC algorithm used in the system took more process time than other processes in the deduplication system. This paper proposes a high throughput hash-less chunking method called Rapid Asymmetric Maximum (RAM). Instead of using hashes, RAM uses bytes value to declare the cut points. The algorithm utilizes a fix-sized window and a variable-sized window to find a maximum-valued byte which is the cut point. The maximum-valued byte is included in the chunk and located at the boundary of the chunk. This configuration allows RAM to do fewer comparisons while retaining the CDC property. We compared RAM with existing hash-based and hash-less deduplication systems. The experimental results show that our proposed algorithm has higher throughput and bytes saved per second compared to other chunking algorithms.     

An efficient energy-aware method for virtual machine placement in cloud data centers using the cultural algorithm

Excessive consumption of energy in cloud data centers whose number is increasing day by day has led to substantial problems. Hence, offering efficient schemes for virtual machine (VM) placement to decrease energy consumption in cloud computing environments has become a significant research field in recent years. In this paper, with the goal of reducing energy consumption in cloud data centers, we present a VM placement method using the cultural algorithm. In the proposed algorithm called balance-based cultural algorithm for virtual machine placement (BCAVMP), a new fitness function is introduced to evaluate VM allocation solutions. In this function, by using the sum of balance vector lengths for each VM placement, balanced utilization of resources is considered. Also, by applying the amount of energy usage in the fitness function, solutions with lower energy consumption are intended. The performance of the proposed method is evaluated using CloudSim simulator. The simulation results indicate that by appropriate VM assignment and resource wastage reduction, energy consumption in cloud data centers can be decreased.

     

面向云平台的二代测序数据近似去重方法研究

新一代测序因其数据量大、数据处理过程复杂、对计算资源要求高等特点,需要通过云计算进行处理。然而,云计算的处理方式要求先将测序数据上传到云平台中。但由于测序过程的随机性,使得同一样本的两次测序、两个相似样本分别测序后所产生的文件在二进制层面会有较大差别。目前已有的去重方法无法有效识别出这样的“重复”测序文件和测序结果中的“重复”内容。重复上传和存储这些重复数据,不仅消耗网络带宽,而且浪费存储空间。针对现存的重复数据删除方法仅仅基于文件的二进制特征,并未有效利用测序结果数据相似性特点的问题,提出一种面向云平台的海量高通量测序数据近似去重方法NPD（Near Probability Deduplication）。该方法对FastQ中的序列和质量信息,使用SimHash计算分块指纹,采用客户端与云平台双布谷过滤器（Cukoo Filter）对指纹值进行快速存在性检测,最后由云平台使用近似算法对指纹值近似去重。实验结果表明,NPD方法在保证高效的同时,大幅提升了去重率,进而减少了网络流量,缩短了数据上传时间,能够支撑海量数据处理,具有良好的实用价值。     

基于离线密钥分发的加密数据重复删除方法

重复数据删除技术受到工业界和学术界的广泛关注.研究者致力于将云服务器中的冗余数据安全的删除,明文数据的重复删除方法较为简单.而用户为了保护隐私,会使用各自的密钥将数据加密后上传至云服务器,形成不同的加密数据.在保证安全性的前提下,加密数据的重复删除较难实现.目前已有的方案较多依赖在线的可信第三方.提出一种基于离线密钥分发的加密数据重复删除方案,通过构造双线性映射,在不泄露数据隐私的前提下,验证加密数据是否源自同一明文.利用广播加密技术实现加密密钥的安全存储与传递.任意数据的初始上传者能够借助云服务器,以离线方式验证后继上传者的合法性并传递数据加密密钥.无需可信第三方在线参与,实现云服务器对加密数据的重复删除.分析并证明了方案的安全性.仿真实验验证了方案的可行性与高效性.     

Entangled cloud storage

Entangled cloud storage (Aspnes et al., ESORICS 2004) enables a set of clients to “entangle” their files into a single clew to be stored by a (potentially malicious) cloud provider. The entanglement makes it impossible to modify or delete significant part of the clew without affecting all files encoded in the clew. A clew keeps the files in it private but still lets each client recover his own data by interacting with the cloud provider; no cooperation from other clients is needed. At the same time, the cloud provider is discouraged from altering or overwriting any significant part of the clew as this will imply that none of the clients can recover their files.We put forward the first simulation-based security definition for entangled cloud storage, in the framework of universal composability (Canetti, 2001). We then construct a protocol satisfying our security definition, relying on an entangled encoding scheme based on privacy-preserving polynomial interpolation; entangled encodings were originally proposed by Aspnes et al. as useful tools for the purpose of data entanglement. As a contribution of independent interest we revisit the security notions for entangled encodings, putting forward stronger definitions than previous work (that for instance did not consider collusion between clients and the cloud provider).Protocols for entangled cloud storage find application in the cloud setting, where clients store their files on a remote server and need to be ensured that the cloud provider will not modify or delete their data illegitimately. Current solutions, e.g., based on Provable Data Possession and Proof of Retrievability, require the server to be challenged regularly to provide evidence that the clients’ files are stored at a given time. Entangled cloud storage provides an alternative approach where any single client operates implicitly on behalf of all others, i.e., as long as one client’s files are intact, the entire remote database continues to be safe and unblemished.     

基于Bloom Filter的混合云存储安全去重方案

针对现有云存储系统中数据去重采用的收敛加密算法容易遭到暴力破解以及猜测攻击等不足,提出一种基于布隆过滤器的混合云存储安全去重方案BFHDedup,改进现有混合云存储系统模型,私有云部署密钥服务器Key Server支持布隆过滤器认证用户的权限身份,实现了用户的细粒度访问控制。同时使用双层加密机制,在传统收敛加密算法基础上增加额外的加密算法并且将文件级别去重和块级别去重相结合实现细粒度去重。此外,BFHDedup采用密钥加密链机制应对去重带来的密钥管理难题。安全性分析及仿真实验结果表明,该方案在可容忍的时间开销代价下实现了较高的数据机密性,有效抵抗暴力破解以及猜测攻击,提高了去重比率并且减少了存储空间。     

Integrating QoS awareness with virtualization in cloud computing systems for delay-sensitive applications

Cloud computing provides scalable computing and storage resources over the Internet. These scalable resources can be dynamically organized as many virtual machines (VMs) to run user applications based on a pay-per-use basis. The required resources of a VM are sliced from a physical machine (PM) in the cloud computing system. A PM may hold one or more VMs. When a cloud provider would like to create a number of VMs, the main concerned issue is the VM placement problem, such that how to place these VMs at appropriate PMs to provision their required resources of VMs. However, if two or more VMs are placed at the same PM, there exists certain degree of interference between these VMs due to sharing non-sliceable resources, e.g. I/O resources. This phenomenon is called as the VM interference. The VM interference will affect the performance of applications running in VMs, especially the delay-sensitive applications. The delay-sensitive applications have quality of service (QoS) requirements in their data access delays. This paper investigates how to integrate QoS awareness with virtualization in cloud computing systems, such as the QoS-aware VM placement (QAVMP) problem. In addition to fully exploiting the resources of PMs, the QAVMP problem considers the QoS requirements of user applications and the VM interference reduction. Therefore, in the QAVMP problem, there are following three factors: resource utilization, application QoS, and VM interference. We first formulate the QAVMP problem as an Integer Linear Programming (ILP) model by integrating the three factors as the profit of cloud provider. Due to the computation complexity of the ILP model, we propose a polynomial-time heuristic algorithm to efficiently solve the QAVMP problem. In the heuristic algorithm, a bipartite graph is modeled to represent all the possible placement relationships between VMs and PMs. Then, the VMs are gradually placed at their preferable PMs to maximize the profit of cloud provider as much as possible. Finally, simulation experiments are performed to demonstrate the effectiveness of the proposed heuristic algorithm by comparing with other VM placement algorithms.     

云存储安全网关关键技术研究

云存储安全网关能够提供安全、高效的数据存储备份服务,克服传统存储备份服务的不足。阐述了云存储网关的研究现状。总结了现有云存储网关研究相关的一些关键技术,其中包括多租户下的数据隔离和隐私保护,访问性能优化和重复数据删除技术,数据访问管理技术和透明加密技术。最后,总结全文并指出云存储网关未来的研究方向。     

A secure cloud storage system supporting privacy-preserving fuzzy deduplication

Deduplication is an important technology in the cloud storage service. For protecting user privacy, sensitive data usually have to be encrypted before outsourcing. This makes secure data deduplication a challenging task. Although convergent encryption is used to securely eliminate duplicate copies on the encrypted data, these secure deduplication techniques support only exact data deduplication. That is, there is no tolerance of differences in traditional deduplication schemes. This requirement is too strict for multimedia data including image. For images, typical modifications such as resizing and compression only change their binary presentation but maintain human visual perceptions, which should be eliminated as duplicate copies. Those perceptual similar images occupy a lot of storage space on the remote server and greatly affect the efficiency of deduplication system. In this paper, we first formalize and solve the problem of effective fuzzy image deduplication while maintaining user privacy. Our solution eliminates duplicated images based on the measurement of image similarity over encrypted data. The robustness evaluation is given and demonstrates that this fuzzy deduplication system is able to duplicate perceptual similar images, which optimizes the storage and bandwidth overhead greatly in cloud storage service.     

Adaptive Markov-based approach for dynamic virtual machine consolidation in cloud data centers with quality-of-service constraints

Dynamic virtual machine (VM) consolidation is one of the emerging technologies that has been considered for low-cost computing in cloud data centers. Quality-of-service (QoS) assurance is one of the challenging issues in the VM consolidation problem since it is directly affected by the increase of resource utilization due to the consolidations. In this paper, we take advantage of Markov chain models to propose a novel approach for VM consolidation that can be used to explicitly set a desired level of QoS constraint in a data center to ensure the QoS goals while improving system utilization. For this purpose, an energy-efficient and QoS-aware best fit decreasing algorithm for VM placement is proposed, which considers QoS objective when determining the location of a migrating VM. This algorithm employs an online transition matrix estimator method to deal with the nonstationary nature of real workload data. We also propose new policies for detecting overloaded and underloaded hosts. The performance of our proposed algorithms is evaluated through simulations. The results show that the proposed VM consolidation algorithms in this paper outperforms the benchmark algorithms in terms of energy consumption, service-level agreement violations, and other cost factors.     

一种混合云环境下基于Merkle哈希树的数据安全去重方案

重复数据删除技术是云存储系统中一种高效的数据压缩和存储优化技术,能够通过检测和消除冗余数据来减少存储空间、降低传输带宽消耗。针对现有的云存储系统中数据安全去重方案所采用的收敛加密算法容易遭受暴力攻击和密文计算时间开销过大等问题,提出了一种混合云环境下基于Merkle哈希树的数据安全去重方案MTHDedup。该方案通过引入权限等级函数和去重系数来计算去重标签,高效地实现了支持访问控制的数据安全去重系统;同时通过执行额外的加密算法,在文件级和数据块级的数据去重过程中构造Merkle哈希树来生成加密密钥,保证了生成的密文变得不可预测。安全性分析表明,该方案能够有效地抵制内部和外部攻击者发起的暴力攻击,从而提高数据的安全性。仿真实验结果表明,MTHDedup方案能有效地降低密文生成的计算开销,减少密钥的存储空间,而且随着权限集数目的增加,性能优势将更加明显。     

Application-specific architectures for energy-efficient database query processing and optimization

Data processing on a continuously growing volume of data and the increasing power restrictions have become an ubiquitous challenge in our world today. Besides parallel computing, a promising approach to improve the energy efficiency of current systems is to integrate specialized hardware. This paper presents two application-specific architectures to accelerate basic database operators frequently used in modern database systems: an extended instruction set based on a given Cadence Tensilica processor (ASIP) and a comparable application-specific integrated circuit (ASIC). The ASIP is implemented in a system-on-chip and manufactured in a 28 nm CMOS technology to realize measurements of performance and power consumption. Furthermore, the comparison with the ASIC blocks allows to quantify the results with the ASIP approach in terms of throughput, area, and energy efficiency as well as to discuss the capabilities and limitations when accelerating selected database operators.     

DFA-VMP: An efficient and secure virtual machine placement strategy under cloud environment

The problem of Virtual Machine (VM) placement is critical to the security and efficiency of the cloud infrastructure. Nowadays most research focuses on the influences caused by the deployed VM on the data center load, energy consumption, resource loss, etc. Few works consider the security and privacy issues of the tenant data on the VM. For instance, as the application of virtualization technology, the VM from different tenants may be placed on one physical host. Hence, attackers may steal secrets from other tenants by using the side-channel attack based on the shared physical resources, which will threat the data security of the tenants in the cloud computing. To address the above issues, this paper proposes an efficient and secure VM placement strategy. Firstly, we define the related security and efficiency indices in the cloud computing system. Then, we establish a multi-objective constraint optimization model for the VM placement considering the security and performance of the system, and find resolution towards this model based on the discrete firefly algorithm. The experimental results in OpenStack cloud platform indicates that the above strategy can effectively reduce the possibility of malicious tenants and targeted tenants on the same physical node, and reduce energy consumption and resource loss at the data center.