基于压缩近邻的查重元数据去冗算法设计

随着重复数据删除次数的增加,系统中用于存储指纹索引的清单文件等元数据信息会不断累积,导致不可忽视的存储资源开销。因此,如何在不影响重复数据删除率的基础上,对重复数据删除过程中产生的元数据信息进行压缩,从而减小查重索引,是进一步提高重复数据删除效率和存储资源利用率的重要因素。针对查重元数据中存在大量冗余数据,提出了一种基于压缩近邻的查重元数据去冗算法Dedup2。该算法先利用聚类算法将查重元数据分为若干类,然后利用压缩近邻算法消除查重元数据中相似度较高的数据以获得查重子集,并在该查重子集上利用文件相似性对数据对象进行重复数据删除操作。实验结果表明,Dedup2可以在保持近似的重复数据删除比的基础上,将查重索引大小压缩50%以上。     

Research progress on secure data deduplication in cloud

In order to improve the efficiency of cloud storage and save the communication bandwidth, a deduplication mechanism for multi-duplicate of the same data in cloud environment was needed. However, the implement of the secure data deduplication was seriously hindered by the ciphertext in cloud. This issue has quickly aroused wide attention of academia and industry, and became a research hotspot. From a security standpoint, firstly the primary cause and the main challenges of secure data deduplication in cloud environment was analyzed, and then the deduplication system model as well as its security model was described. Furthermore, focusing on the realization mechanism of secure data deduplica-tion, the thorough analyses were carried on and reviews for the related research works in recent years from content-based encryption, proof of ownership and privacy protection for secure deduplication, then the advantages and common prob-lems of various key technologies and methods were summed up. Finally, the future research directions and development trends on secure data deduplication in cloud was given.     

云存储安全技术研究进展综述

云存储服务涉及大量的用户隐私数 据,因此其安全性和隐私性至关重要。为此,本文针对云存储的重复数据删除、隐藏存储、 数据加密与密文搜索以及数据完整性审计４个方面,介绍当前云存储安全技术研究的进 展, 并探讨提高商业云存储系统的安 全性和隐私性的未来研究方向。 其中,云存储的可重复数据删除技术旨在同时对抗文件识别攻击等多种攻击,减少额外 服务器开支,并提高用户收益。隐藏存储技术采用模块化结构和异步机制等方法降低 云存储系统的计算量,并提高其安全性和可扩展性。云存储的数据加密和密文搜索在传统加 密的基础上,基于密文策略属性的加密技术实现,提供密文搜索,防止用户身份隐私的泄露 。数据完整性审计系统针对用户密钥泄露等安全问题,旨在为用户提供安全、高效和支持审 计的存储服务。     

基于分片复用的多版本容器镜像加载方法

容器将应用和支持软件、库文件等封装为镜像,通过发布新版本镜像实现应用升级,导致不同版本之间存在大量相同数据.镜像加载消耗大量时间,使容器启动时间从毫秒级延迟为秒级甚至是分钟级.复用不同版本之间的相同数据,有利于减少容器加载时间.当前,容器镜像采用继承和分层加载机制,有效实现了支持软件、库文件等数据的复用,但对于应用内部数据还没有一种可靠的复用机制.提出一种基于分片复用的多版本容器镜像加载方法,通过复用不同版本镜像之间的相同数据,提升镜像加载效率.方法的核心思想是:利用边界匹配数据块切分方法将容器镜像切分为细粒度数据块,将数据块哈希值作为唯一标识指纹,借助B-树搜索重复指纹判断重复数据块,减少数据传输.实验结果表明,该方法可以提高5.8X以上容器镜像加载速度.     

一种基于语义及统计分析的Deep Web实体识别机制

分析了常见的实体识别方法,提出了一种基于语义及统计分析的实体识别机制(deep Web entity identification mechanism based on semantics and statistical analysis,简称SS-EIM),能够有效解决Deep Web数据集成中数据纠错、消重及整合等问题.SS-EIM主要由文本匹配模型、语义分析模型和分组统计模型组成,采用文本粗略匹配、表象关联关系获取以及分组统计分析的三段式逐步求精策略,基于文本特征、语义信息及约束规则来不断精化识别结果;根据可获取的有限的实例信息,采用静态分析、动态协调相结合的自适应知识维护策略,构建和完善表象关联知识库,以适应Web数据的动态性并保证表象关联知识的完备性.通过实验验证了SS-EIM中所采用的关键技术的可行性和有效性.     

基于零参考深度曲线估计的图像增强网络改进

随着多云存储市场的快速发展, 越来越多的用户选择将数据存储在云上, 随之而来的是云环境中的重复数据也呈爆炸式增长. 由于云服务代理是相互独立的, 因此传统的数据去重只能消除代理本身管理的几个云服务器上的冗余数据. 为了进一步提高云环境中数据去重的力度, 本文提出了一种多代理联合去重方案. 通过区块链技术促成云服务代理间的合作, 并构建代理联盟, 将数据去重的范围从单个代理管理的云扩大到多代理管理的多云. 同时, 能够为用户、云服务代理和云服务提供商带来利益上的共赢. 实验表明, 多代理联合去重方案可以显著提高数据去重效果、节约网络带宽.     

PoSW:novel secure deduplication scheme for collaborative cloud applications

In order to solve the problems of secure access and deduplication to the shared file in the cloud environment,a novel notion called proof of shared ownership (PoSW) was formalized,and a formal definition of the PoSW was given.Furthermore,a PoSW scheme and an enhanced version of that were proposed.In the PoSW scheme,secure shared file dispersal,convergent encryption and secret sharing algorithm were employed to transform the shared file realize the sharing and authorization for the shared file,and then a novel challenge-response protocol was proposed to achieve the proof of shared ownership and the foundation for the secure deduplication of the shared file was provided.An enhanced PoSW scheme was designed to improve the availability and reliability for different kinds of the shared files by introducing the multi-cloud server providers and using the strategies of both data duplication and secret file dispersal.Security analysis and performance evaluation show the security and efficiency of the proposed scheme.     

基于重复数据删除的快速文件归档方法

为了提高归档系统的存储效率及性能,提出了一种基于重复数据删除的快速文件归档方法（RAMBDD）,利用文件分块、比较数据块指纹、删除重复数据,实现了文件的数据块级归档。RAMBDD中给出了一种基于winnow-ing的重复数据删除方法LMCA,它在提高文件冗余检测率的同时也保证了文件分块的效率,并通过使用指纹快速检索方法和局部指纹缓存方法,减少了在查找不存在的数据块指纹时的磁盘读取次数,加速了查找重复数据块的过程。实验结果表明,与传统的文件归档方法相比,本方法大大节省了归档数据的存储空间和网络传输带宽,缩短了归档时间,提高了文件归档的效率。     

Secured Data Storage Using Deduplication in Cloud Computing Based on Elliptic Curve Cryptography

The tremendous development of cloud computing with related technologies is an unexpected one. However, centralized cloud storage faces few challenges such as latency, storage, and packet drop in the network. Cloud storage gets more attention due to its huge data storage and ensures the security of secret information. Most of the developments in cloud storage have been positive except better cost model and effectiveness, but still data leakage in security are billion-dollar questions to consumers. Traditional data security techniques are usually based on cryptographic methods, but these approaches may not be able to withstand an attack from the cloud server's interior. So, we suggest a model called multi-layer storage (MLS) based on security using elliptical curve cryptography (ECC). The suggested model focuses on the significance of cloud storage along with data protection and removing duplicates at the initial level. Based on divide and combine methodologies, the data are divided into three parts. Here, the first two portions of data are stored in the local system and fog nodes to secure the data using the encoding and decoding technique. The other part of the encrypted data is saved in the cloud. The viability of our model has been tested by research in terms of safety measures and test evaluation, and it is truly a powerful complement to existing methods in cloud storage.     

一种基于重复数据删除的网络文件备份系统设计与实现

针对广域网中网络备份系统存在备份速度低、网络带宽需求高和数据冗余等问题,提出并实现一种具有重复数据删除功能的网络文件备份系统。通过在客户端实现重复数据删除技术,对文件进行分块和在备份过程中去除重复数据块,减少客户端与服务期间需要传输的数据量,使得文件备份的速度获得较大提高,也较大地降低网络带宽要求,显著提高了网络备份系统的性能。此外,备份过程中客户端去除了重复的数据块使得服务器中存储的数据块都是唯一的,极大地降低了系统所需要的存储空间,同时,服务器端的数据块引用管理的设计很好地解决了空闲数据块耗费存储空间的问题。