A new content-defined chunking algorithm for data deduplication in cloud storage

Chunking is a process to split a file into smaller files called chunks. In some applications, such as remote data compression, data synchronization, and data deduplication, chunking is important because it determines the duplicate detection performance of the system. Content-defined chunking (CDC) is a method to split files into variable length chunks, where the cut points are defined by some internal features of the files. Unlike fixed-length chunks, variable-length chunks are more resistant to byte shifting. Thus, it increases the probability of finding duplicate chunks within a file and between files. However, CDC algorithms require additional computation to find the cut points which might be computationally expensive for some applications. In our previous work (Widodo et al., 2016), the hash-based CDC algorithm used in the system took more process time than other processes in the deduplication system. This paper proposes a high throughput hash-less chunking method called Rapid Asymmetric Maximum (RAM). Instead of using hashes, RAM uses bytes value to declare the cut points. The algorithm utilizes a fix-sized window and a variable-sized window to find a maximum-valued byte which is the cut point. The maximum-valued byte is included in the chunk and located at the boundary of the chunk. This configuration allows RAM to do fewer comparisons while retaining the CDC property. We compared RAM with existing hash-based and hash-less deduplication systems. The experimental results show that our proposed algorithm has higher throughput and bytes saved per second compared to other chunking algorithms.     

Entangled cloud storage

Entangled cloud storage (Aspnes et al., ESORICS 2004) enables a set of clients to “entangle” their files into a single clew to be stored by a (potentially malicious) cloud provider. The entanglement makes it impossible to modify or delete significant part of the clew without affecting all files encoded in the clew. A clew keeps the files in it private but still lets each client recover his own data by interacting with the cloud provider; no cooperation from other clients is needed. At the same time, the cloud provider is discouraged from altering or overwriting any significant part of the clew as this will imply that none of the clients can recover their files.We put forward the first simulation-based security definition for entangled cloud storage, in the framework of universal composability (Canetti, 2001). We then construct a protocol satisfying our security definition, relying on an entangled encoding scheme based on privacy-preserving polynomial interpolation; entangled encodings were originally proposed by Aspnes et al. as useful tools for the purpose of data entanglement. As a contribution of independent interest we revisit the security notions for entangled encodings, putting forward stronger definitions than previous work (that for instance did not consider collusion between clients and the cloud provider).Protocols for entangled cloud storage find application in the cloud setting, where clients store their files on a remote server and need to be ensured that the cloud provider will not modify or delete their data illegitimately. Current solutions, e.g., based on Provable Data Possession and Proof of Retrievability, require the server to be challenged regularly to provide evidence that the clients’ files are stored at a given time. Entangled cloud storage provides an alternative approach where any single client operates implicitly on behalf of all others, i.e., as long as one client’s files are intact, the entire remote database continues to be safe and unblemished.     

一个网络文件存储系统TNS关键技术

基于多服务器架构、为多用户服务的网络文件存储系统普遍存在资源分配不均,重复文件多,存储空间浪费严重的问题。设计并实现了TNS网络文件存储系统,该系统基于多服务器存储架构,分别由用户服务器、索引服务器、数据服务器、共享服务器、管理服务器和登录服务器组成,为多用户服务,采用一致性Hash实现负载均衡,支持在客户端进行文件粒度的重复数据删除。经过实际生产环境运行测试,具有良好的负载均衡能力和重复数据删除功能,可以有效节省存储空间,提高存储设备利用率。     

改进的高效云存储数据去重方案

针对Chen等人提出的云存储数据去重方案BL-MLE的计算开销过大的问题,对其方案进行了改进,提出了一种更高效的数据去重方案。首先对BL-MLE方案进行了分析,指出其在计算效率等方面的不足;随后通过使用hash函数和标签决策树对BL-MLE的块标签生成过程以及块标签比较过程进行改进;最后,实验仿真了改进的方案。结果表明,改进后的方案在块标签比较所需次数更少,且块标签生成上时间开销更低,能更好地适应当前的云存储环境。     

Achieving convergent causal consistency and high availability for cloud storage

The tradeoff between consistency and availability is inevitable when designing distributed data stores, and today’s cloud services often choose high availability instead of strong consistency, leading to visible inconsistencies for clients. Convergent causal consistency is one of the strongest consistency model that still remains available during system partitions, and it can also satisfy human perception of causality between events. In this paper, we present CoCaCo, a distributed key-value store that provides convergent causal consistency with asynchronous replication, since it is able to provide cloud services’ desired properties including high performance and availability. Moreover, CoCaCo can efficiently guarantee causal consistency by performing dependency checking only during handling read operations. We implement CoCaCo based on Cassandra and our experimental results indicate that CoCaCo provides performance comparable to eventually consistent Cassandra.     

Secured Data Storage Using Deduplication in Cloud Computing Based on Elliptic Curve Cryptography

The tremendous development of cloud computing with related technologies is an unexpected one. However, centralized cloud storage faces few challenges such as latency, storage, and packet drop in the network. Cloud storage gets more attention due to its huge data storage and ensures the security of secret information. Most of the developments in cloud storage have been positive except better cost model and effectiveness, but still data leakage in security are billion-dollar questions to consumers. Traditional data security techniques are usually based on cryptographic methods, but these approaches may not be able to withstand an attack from the cloud server's interior. So, we suggest a model called multi-layer storage (MLS) based on security using elliptical curve cryptography (ECC). The suggested model focuses on the significance of cloud storage along with data protection and removing duplicates at the initial level. Based on divide and combine methodologies, the data are divided into three parts. Here, the first two portions of data are stored in the local system and fog nodes to secure the data using the encoding and decoding technique. The other part of the encrypted data is saved in the cloud. The viability of our model has been tested by research in terms of safety measures and test evaluation, and it is truly a powerful complement to existing methods in cloud storage.     

Improved security of a dynamic remote data possession checking protocol for cloud storage

Cloud storage offers the users with high quality and on-demand data storage services and frees them from the burden of maintenance. However, the cloud servers are not fully trusted. Whether the data stored on cloud are intact or not becomes a major concern of the users. Recently, Chen et al. proposed a remote data possession checking protocol to address this issue. One distinctive feature of their protocol support data dynamics, meaning that users are allowed to modify, insert and delete their outsourced data without the need to re-run the whole protocol. Unfortunately, in this paper, we find that this protocol fails to achieve its purpose since it is vulnerable to forgery attack and replace attack launched by a malicious server. Specifically, we show how a malicious cloud server can deceive the user to believe that the entire file is well-maintained by using the meta-data related to the file alone, or with only part of the file and its meta-data. Then, we propose an improved protocol to fix the security flaws and formally proved that our proposal is secure under a well-known security model. In addition, our improvement keeps all the desirable features of the original protocol.     

基于Merkle哈希树的云存储加密数据去重复研究

收敛加密可以有效地解决数据加密和去重的矛盾,实现安全去重复,但是收敛加密仍然面临许多安全问题。针对传统的收敛加密容易遭受字典攻击的问题,提出基于Merkle哈希树的收敛加密方案实现数据去重复,通过执行额外的加密操作,加强数据的机密性,有效地避免字典攻击。为了克服传统的收敛加密方案的收敛密钥随着用户数量线性增长的问题,设计收敛密钥共享机制,进一步节省了收敛密钥的存储空间。     

面向去中心化存储的数据流行度去重模型

数据流行度去重方案中存在检测机构不诚实、数据存储不可靠等问题,提出一种面向去中心化存储的数据流行度去重模型。针对检测机构不诚实,模型结合区块链的不可篡改性与智能合约的不可抵赖性,将智能合约作为检测机构执行数据的重复性检测和流行度检测,保障了检测结果的真实性。针对数据存储不可靠问题,提出一种文件链存储结构,该结构满足数据流行度去重的要求,并通过添加辅助信息的方式,建立分布在不同存储节点中实现物理/逻辑上传的分片之间的逻辑关系,为流行度数据去中心化网络存储提供基础;同时,在数据块信息中添加备份标识,借助备份标识将存储网络划分为两个虚拟存储空间,分别实现数据和备份数据的检测与存储,满足了用户备份需求。安全性分析和性能分析表明,该方案具有可行性,保障了检测结果的真实性,并提高了数据存储的可靠性。     

A-SATCHMORE: SATCHMORE with availability checking

We present an improvement of SATCHMORE, calledA-SATCHMORE, by incorporating availability checking into relevancy. Because some atoms unavailable to the further computation are also marked relevant, SATCHMORE suffers from a potential explosion of the search space. Addressing this weakness of SATCHMORE, we show that an atom does not need to be marked relevant unless it is available to the further computation and no non-Horn clause needs to be selected unless all its consequent atoms are marked availably relevant, i.e., unless it is totally availably relevant. In this way,A-SATCHMORE is able to further restrict the ues of non-Horn clauses (therefore to reduce the search space) and makes the proof more goal-oriented. Our theorem prover,A-SATCHMORE, can be simply implemented in PROLOG based on SATCHMORE. We discuss how to incorporate availability cheeking into relevancy, describe our improvement and present the implementation. We also prove that our theorem prover is sound and complete, and provide examples to show the power of our availability approach. This research is supported in part by the Japanese Ministry of Education and the Artificial Intelligence Research Promotion Foundation. Lifeng He, Ph.D: He received the B. E. degree from Northwest Institute of Light Industry, China, in 1982, the M. S. and Ph.D. degrees in AI and computer science from Nagoya Institute of Technology, Japan, in 1994 and 1997, respectively. He currently works at the Institute of Open System in Nagoya, Japan. His research interests include automated reasoning, theorem proving, logic programming, knowledge bases, multi-agent cooperation and modal logic. Yuyan Chao, M. S.: She received the B. E. degree from Northwest Institute of Light Industry, China, in 1984, and the M. S. degree from Nagoya University, Japan, in 1997. She is currently a doctoral candidate in the Department of Human Information, Nagoya University. Her research interests include image processing, graphic understanding, CAD and theorem proving. Yuka Shimajiri, M. S.: She currently works as a Assistant Professor in Department of Artificial Intelligence and Computer Science at the Nagoya Institute of Technology. She received her B.Eng. and M.Eng. from the Nagoya Institute of Technology in 1994 and 1996, respectively. Her current research interests include logic programming and automated deduction. She is a member of IPSJ and JSAI. Hirohisa Seki, Ph.D.: He received the B. E., M. E. and Ph.D degrees from the University of Tokyo in 1979, 1981 and 1991 respectively. He joined the Central Research Laboratory of Mitsubishi Electric Corporation in 1981. From 1985 to 1989, he was with the Institute for New Generation Computer Technology (ICOT). Since 1992, he has been an Associate Professor in the Department of AI and Computer Science at Nagoya Institute of Technology. His current research interests include logic programming, deductive databases and automated deduction. He is a member of ACM, IEEE, IPSJ and JSAI. Hidenori Itoh, Ph.D.: He received the B. S. degree from Fukui University, in 1969, the M. S. degree and Ph.D degree from Nagoya University, Japan, in 1971 and 1974, respectively. From 1974 to 1985, he worked at Nippon Telephone and Telegraph Laboratories, developing operating systems. From 1985 to 1989, he was with the Institute for New Generation Computer Technology, developing knowledge base systems. Since 1989, he has become a professor at the Nagoya Institute of Technology. His current research interests include image processing, parallel computing, fuzzy logic and knowledge processing.     

基于哈希树的云存储完整性检测算法

云存储服务使得用户无需大量软硬件投入即可享受大容量、高规格的存储服务,但是同时也带来了云环境下数据机密性、完整性和可用性等安全问题。针对云存储中的完整性问题,利用哈希树结构和大数模运算,提出了一种新的基于哈希树结构的数据完整性检测算法。分析结果表明,该算法使得用户只需在常量的存储、计算和网络资源下就能高概率地、正确地检测远端服务器数据文件的完整性,且支持文件数据的动态更新。     

高性能的云存储安全网关设计与实现

随着互联网应用的迅速发展,数据量呈指数级增长趋势。越来越多的用户选择将数据存储备份到云端,但同时数据安全隐患也越发突出。用户对海量数据的存储备份和安全性需求越来越高。为了方便用户将本地数据迁移到云存储服务端,简化远程云端资源的访问,提供隐私保护、数据安全等附加价值,设计并实现了一个安全、高性能、大容量的混合云存储安全网关。该云存储安全网关能够提供安全、高效的数据存储备份服务,克服传统存储备份服务的不足。具有灵活的数据机密性和用户隐私管理、安全的数据存储和传输等特点。     

Increasing availability of industrial systems through data stream mining

Improving industrial product reliability, maintainability and thus availability is a challenging task for many industrial companies. In industry, there is a growing need to process data in real time, since the generated data volume exceeds the available storage capacity. This paper consists of a review of data stream mining and data stream management systems aimed at improving product availability. Further, a newly developed and validated grid-based classifier method is presented and compared to one-class support vector machine (OCSVM) and a polygon-based classifier.     

Engineering of secure multi-cloud storage

This article addresses security and privacy issues associated with storing data in public cloud services. It presents an architecture based on a novel secure cloud gateway that allows client systems to store sensitive data in a semi-trusted multi-cloud environment while providing confidentiality, integrity, and availability of data. This proxy system implements a space-efficient, computationally-secure threshold secret sharing scheme to store shares of a secret in several distinct cloud datastores. Moreover, the system integrates a comprehensive set of security measures and cryptographic protocols to mitigate threats induced by cloud computing. Performance in practice and code quality of the implementation are analyzed in extensive experiments and measurements.     

基于多目标优化的云存储副本分布策略的研究

针对现有云存储副本分布策略优化目标比较单一的不足,提出了局部最佳分布策略(Local Optimum Distribution,LODS).LODS策略通过给出一系列新定义并利用一致性哈希函数来缩小副本分布的节点选择范围,进一步结合层次分析法,将一定决策半径内的节点作为方案层中的候选对象,通过更深入地研究云存储多目标优化准则对其优化从而最终选择出当前候选方案中的最佳目标节点.实验结果表明,通过优化的最优决策半径取值相对稳定,不随云存储系统规模的扩展和数据的增多而剧烈变化,并且当取值最佳决策半径时,LODS策略的存储负载平衡、热度负载平衡、等待时间性能高于HDFS、Amazon S3等系统中所采用的副本分布策略.     

保证云存储的数据安全

近几年来,云存储的使用越来越广泛,不仅是中小型企业,而且针对个体用户也可以购买云存储服务,但如何在云存储中保证数据的安全性和机密性,单靠云存储服务供应商的承诺是难以让人信服的。本文提出了一种保证数据安全的云存储框架,并且使用服务等级协议（SLA）作为用户和供应商之间的共同标准。此外,本文还提及了几种保证云存储数据安全的技术,这些技术可以被分成三类：存储保护,传输保护和授权保护。     

Proofs of retrievability with tag outsourcing based on Goppa codes

In cloud storage, because the data owner loses the physical control of the data, the data may be tampered with or deleted. Although, it has been proposed to adopt provable data possession (PDP) or proofs of retrievability (POR) mechanism to ensure the integrity of cloud storage data. However, at present, most PDP/POR schemes are based on traditional cryptographic mechanisms and cannot resist quantum computer attacks. For this reason, the first POR scheme based on coding mechanism (BC-POR) is proposed in this paper. The scheme is constructed based on the difficulty assumptions of 2-regular word syndrome decoding (2-RWSD) problem and Goppa code distinguishing problem. Moreover, considering the low computing power of lightweight users, this paper adopts an audit scheme that supports the outsourcing of data tag calculation on the client side, that is, the calculation of data tag generation is outsourced to a third-party institution for execution. First of all, this scheme can prevent third-party institution from obtaining the real content of the data in the process of calculation tags and realize the privacy protection of user data. Secondly, the scheme uses a FSB hash function to generate a decodable syndrome, and this algorithm does not require iterative operations during the tagging process, thereby reducing the computational overhead of the tag. Finally, the provable security method is used to prove the security of the proposed scheme, and the performance of the proposed audit scheme is evaluated to prove the effectiveness of the scheme.     

Secure administration of cryptographic role-based access control for large-scale cloud storage systems

Cloud systems provide significant benefits by allowing users to store massive amount of data on demand in a cost-effective manner. Role-based access control (RBAC) is a well-known access control model which can be used to protect the security of cloud data storage. Although cryptographic RBAC schemes have been developed recently to secure data outsourcing, these schemes assume the existence of a trusted administrator managing all the users and roles, which is not realistic in large-scale systems. In this paper, we introduce a cryptographic administrative model AdC-RBAC for managing and enforcing access policies for cryptographic RBAC schemes. The AdC-RBAC model uses cryptographic techniques to ensure that the administrative tasks are performed only by authorised administrative roles. Then we propose a role-based encryption (RBE) scheme and show how the AdC-RBAC model decentralises the administrative tasks in the RBE scheme thereby making it practical for security policy management in large-scale cloud systems.     

浅谈数据存储的结合运用

本文通过对存储系统的硬件技术性能分析,结合作者实际工作经验,阐述电子信息数据安全在当前企业信息化建设中越来越重要,探讨如何选择先进的数据存储方法,对企业数据进行主动安全防护,从而保证数据的安全性、可靠性、稳定性.     

云存储安全网关关键技术研究

云存储安全网关能够提供安全、高效的数据存储备份服务,克服传统存储备份服务的不足。阐述了云存储网关的研究现状。总结了现有云存储网关研究相关的一些关键技术,其中包括多租户下的数据隔离和隐私保护,访问性能优化和重复数据删除技术,数据访问管理技术和透明加密技术。最后,总结全文并指出云存储网关未来的研究方向。