网格中基于访问频率的数据复制管理策略

网格中涉及到大量数据文件的复制和传输，数据的有效复制可以节省带宽、减少时延、均衡负载、改善系统可靠性。文中提出的采用基于访问频率的复制管理技术可以使用户有效地获得所需的数据，并采用一种改进的Fast Upload方法来更新文件的副本，减少了复制和更新所用的时间，提高系统的效率。     

异步复制流程的研究与性能改进

数据更新流程是远程复制系统的框架。针对不同的应用需求,系统可以采用不同的复制模式。文中主要讨论异步模式下的数据更新流程。在分析借鉴了已有的异步复制协议基础上,提出了一种改进性能的异步复制流程。原型实验表明,此数据更新流程的设计在保持数据的一致性以及系统简单、可靠性的同时,也降低了复制过程中对网络带宽的需求和I/O操作的数量。     

多复制服务器间无阻塞的数据更新

在Client/Server系统中,服务器的可用性是提高整个系统可用性的关键,采用多复制服务器是提高系统可用性的最有希望的手段.但是,复制数据更新过程中的阻塞问题是整个系统性能的一个瓶颈.本文提出一种无阻塞的多服务器独立提交的复制数据更新方法,对于因失效不能完成更新的服务器采用协调机制使其达到相同的最终状态.     

Linux平台远程逻辑卷复制系统的设计

提出了一种Linux平台上远程逻辑卷复制系统的设计方案,论述了它的系统结构及实现方法。该系统不依赖于特殊的存储设备或网络,在复制过程中通过保持主、从两端逻辑卷上数据块的更新顺序来保证数据一致性。系统支持同步和异步两种复制模式,可以适合不同的应用环境,为实现通用、经济的远程数据容灾系统提供了可能。     

分布式数据库数据复制的实现：SYBASE的复制服务器概念

近几年来,分布式数据库已成为信息处理的一个重要领域。其中,分布式数据库的数据共享技术更是分布式数据库实现的一个关键。过去,通过网络连接的集中式系统,已经证明在数据的共享性,存取速度以及系统开销上存在问题。以往的分布式数据库也有类似的限制,待别是当相同数据在不同的操作环境间共享时,环境之间可能相互影响。对于这类问题,复制数据是唯一可以提供高性能、高可靠性的方法,复制数据的优点还在于通过使用本地数据而减少网络负担,提高整个系统性能;通过每一个节点对数据输入、输出、更新的     

数据网格的动态读／写复制策略研究

探讨了在网格环境中数据复制的相关策略，包括动态只读复制策略和动态读/写复制策略。动态复制策略可以用来减少带宽消耗和访问延迟，不同的复制策略在不同的访问方式下有不同的性能表现。同时，提出了动态读/写复制策略的Fast Upload算法，算法支持数据传输过程中的并发复制，通过复制锁机制来维护复制与更新数据的一致性。     

远程复制系统数据一致性研究与实现

针对远程数据复制容灾系统中的数据一致性问题,提出了保证数据一致性和视图一致性并优化系统性能的实现思想,即使用日志卷记录用户请求顺序,通过保持主、从两端数据卷上数据块的更新顺序来保证数据一致性,对于不同的数据类型采用不同的传输方法。论述了它的组织结构、实现流程以及关键技术。     

劳动保障容灾系统解决方案分析

劳动保障容灾系统通过自动、连续、安全的数据复制进行备份,精简了复杂的数据、系统和网络的管理流程。容灾系统的业务连续性解决方案可以帮助简化运营、减少人为错误和保护重要的应用程序和数据。容灾系统的解决方案通过了可靠性测试。运行实践证明,该系统可以显著减少突发故障、群集升级和系统维护,且不会中断业务的正常运行。     

多副本动态更新数据无重复挖掘仿真

网络动态更新时,需要存储和传输大量数据,有时还会对网络更新数据重复复制,产生副本,浪费大量存储空间、减缓传输速度,产生网络拥堵、数据泄露等现象。针对这些问题提出一种多副本动态更新数据无重复挖掘。先依据网络数据特性构建多副本动态更新数据模型,再采用密文方法对更新数据做加密处理,并存储到网络服务器中,最后利用密文防御特性将该数据重复部分挖掘,并做解密处理,保证了网络服务器中的动态更新数据都是无重复的。实验结果表明,多副本动态更新数据无重复挖掘,可以有效避免数据泄露、传输速度慢、拥挤等问题,实现了多副本动态更新数据的无重复。     

双机热备系统中的数据同步技术研究

双机热备系统中数据的及时同步在很大程度上影响系统的性能。针对传统双机热备系统中数据复制的连续性以及检索更新文件时间长的不足,提出了基于链路延迟的文件复制和基于单向哈希函数的更新文件检索。通过对这两种方法的分析,说明该改善在很大程度上提高了双机对等节点间数据同步的效率。     

Network and data location aware approach for simultaneous job scheduling and data replication in large-scale data grid environments

Data Grid integrates graphically distributed resources for solving data intensive scientific applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. Scheduling is a traditional problem in parallel and distributed system. However, due to special issues and goals of Grid, traditional approach is not effective in this environment any more. Therefore, it is necessary to propose methods specialized for this kind of parallel and distributed system. Another solution is to use a data replication strategy to create multiple copies of files and store them in convenient locations to shorten file access times. To utilize the above two concepts, in this paper we develop a job scheduling policy, called hierarchical job scheduling strategy (HJSS), and a dynamic data replication strategy, called advanced dynamic hierarchical replication strategy (ADHRS), to improve the data access efficiencies in a hierarchical Data Grid. HJSS uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers network characteristics, number of jobs waiting in queue, file locations, and disk read speed of storage drive at data sources. Moreover, due to the limited storage capacity, a good replica replacement algorithm is needed. We present a novel replacement strategy which deletes files in two steps when free space is not enough for the new replica: first, it deletes those files with minimum time for transferring. Second, if space is still insufficient then it considers the last time the replica was requested, number of access, size of replica and file transfer time. The simulation results show that our proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, number of intercommunications, number of replications, hit ratio, computing resource usage and storage usage.     

A novel dynamic network data replication scheme based on historical access record and proactive deletion

Data replication is becoming a popular technology in many fields such as cloud storage, Data grids and P2P systems. By replicating files to other servers/nodes, we can reduce network traffic and file access time and increase data availability to react natural and man-made disasters. However, it does not mean that more replicas can always have a better system performance. Replicas indeed decrease read access time and provide better fault-tolerance, but if we consider write access, maintaining a large number of replications will result in a huge update overhead. Hence, a trade-off between read access time and write updating cost is needed. File popularity is an important factor in making decisions about data replication. To avoid data access fluctuations, historical file popularity can be used for selecting really popular files. In this research, a dynamic data replication strategy is proposed based on two ideas. The first one employs historical access records which are useful for picking up a file to replicate. The second one is a proactive deletion method, which is applied to control the replica number to reach an optimal balance between the read access time and the write update overhead. A unified cost model is used as a means to measure and compare the performance of our data replication algorithm and other existing algorithms. The results indicate that our new algorithm performs much better than those algorithms.     

基于副本索引的P2P副本一致性维护策略

P2P中副本应用从静态逐渐转变到动态更新,研究如何管理动态改变数据的一致性问题成为关键。该文提出一种基于副本索引的无结构P2P副本一致性维护策略,在创建副本时将副本所在节点的索引发送至种子节点保存,同时自身保存种子节点的索引。当副本节点发生更新后,主动发送信息通知种子节点,种子节点根据副本索引表进行副本的一致性维护。仿真实验和分析表明,该副本一致性维护策略能有效地提高副本的一致性,减少冗余消息数目。     

基于访问频率的网格副本创建策略

提出了一种基于访问频率的副本创建策略。该策略主要依据网格用户对文件副本的访问频率进行副本创建,在替换副本时也依据频率值,将不经常访问的副本删除。这种策略能够很好地满足用户访问所需副本的要求,并能提高副本的传输速率与带宽的利用率。文章根据网格结构的特点和算法的环境要求对网格模拟器OptorSim的模块进行了改进,并对该算法进行了测试。测试结果表明,基于访问频率的副本创建算法提高了用户访问副本的效率。     

PDDRA: A new pre-fetching based dynamic data replication algorithm in data grids

In recent years, grid technology has had such a fast growth that it has been used in many scientific experiments and research centers. A large number of storage elements and computational resources are combined to generate a grid which gives us shared access to extra computing power. In particular, data grid deals with data intensive applications and provides intensive resources across widely distributed communities. Data replication is an efficient way for distributing replicas among the data grids, making it possible to access similar data in different locations of the data grid. Replication reduces data access time and improves the performance of the system. In this paper, we propose a new dynamic data replication algorithm named PDDRA that optimizes the traditional algorithms. Our proposed algorithm is based on an assumption: members in a VO (Virtual Organization) have similar interests in files. Based on this assumption and also file access history, PDDRA predicts future needs of grid sites and pre-fetches a sequence of files to the requester grid site, so the next time that this site needs a file, it will be locally available. This will considerably reduce access latency, response time and bandwidth consumption. PDDRA consists of three phases: storing file access patterns, requesting a file and performing replication and pre-fetching and replacement. The algorithm was tested using a grid simulator, OptorSim developed by European Data Grid projects. The simulation results show that our proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, effective network usage, total number of replications, hit ratio and percentage of storage filled.     

Usability of a cloud-based collaborative learning framework to improve learners’ experience

Computer-Supported Collaborative Learning (CSCL) is concerned with how Information and Communication Technology (ICT) might facilitate learning in groups which can be co-located or distributed over a network of computers such as Internet. CSCL supports effective learning by means of communication of ideas and information among learners, collaborative access of essential documents, and feedback from instructors and peers on learning activities. As the cloud technologies are increasingly becoming popular and collaborative learning is evolving, new directions for development of collaborative learning tools deployed on cloud are proposed. Development of such learning tools requires access to substantial data stored in the cloud. Ensuring efficient access to such data is hindered by the high latencies of wide-area networks underlying the cloud infrastructures. To improve learners’ experience by accelerating data access, important files can be replicated so a group of learners can access data from nearby locations. Since a cloud environment is highly dynamic, resource availability, network latency, and learner requests may change. In this paper, we present the advantages of collaborative learning and focus on the importance of data replication in the design of such a dynamic cloud-based system that a collaborative learning portal uses. To this end, we introduce a highly distributed replication technique that determines optimal data locations to improve access performance by minimizing replication overhead (access and update). The problem is formulated using dynamic programming. Experimental results demonstrate the usefulness of the proposed collaborative learning system used by institutions in geographically distributed locations.     

网格虚拟组织副本协作预取机制

副本复制是数据网格中提高数据访问效率的有效方法,如何提高副本复制的效率是一个关键性问题.现有的复制策略大多基于文件访问历史选择高价值副本进行复制,但其针对的都是节点已经访问过的文件.通过对虚拟组织文件访问特性进行深入分析,引入隐性高价值文件概念,提出虚拟组织副本协作预取机制(cooperative replica prefetching mechanism,简称CoRPM),使得本地节点通过与虚拟组织中其他节点进行协作来获取隐性高价值文件副本.该机制首先给出了副本协作预取架构,各个虚拟组织节点上的文件预取模块以协作的方式为虚拟组织内节点提供文件预取服务;然后,在副本协作预取架构的基础上设计了副本协作预取流程,其核心算法包括以作业类型为中心的本地文件预取算法和预取文件选择算法.模拟实验结果表明,CoRPM与已有的基于文件访问历史的副本复制策略相结合,可以更加有效地降低数据访问延迟.     

基于hybrid拓扑的数据网格副本创建策略*

数据复制技术被广泛应用于数据网格中,以缩短数据访问时间和传输时间、降低网络带宽消耗.针对包含树型拓扑和环型拓扑的混合式网格拓扑结构,提出了一种考虑网络带宽、网络传输延迟、用户请求频率和站点可用存储空间大小等因素的副本创建策略,并引入评估函数衡量各因素的影响大小,具有良好的可靠性、可扩展性和自适应性.模拟实验的结果显示此副本创建策略可以有效降低数据平均访问时间.     

Adaptive popularity-driven replica placement in hierarchical data grids

Data grids support access to widely distributed storage for large numbers of users accessing potentially many large files. Efficient access is hindered by the high latency of the Internet. To improve access time, replication at nearby sites may be used. Replication also provides high availability, decreased bandwidth use, enhanced fault tolerance, and improved scalability. Resource availability, network latency, and user requests in a grid environment may vary with time. Any replica placement strategy must be able to adapt to such dynamic behavior. In this paper, we describe a new dynamic replica placement algorithm, Popularity Based Replica Placement (PBRP), for hierarchical data grids which is guided by file “popularity”. Our goal is to place replicas close to clients to reduce data access time while still using network and storage resources efficiently. The effectiveness of PBRP depends on the selection of a threshold value related to file popularity. We also present Adaptive-PBRP (APBRP) that determines this threshold dynamically based on data request arrival rates. We evaluate both algorithms using simulation. Results for a range of data access patterns show that our algorithms can shorten job execution time significantly and reduce bandwidth consumption compared to other dynamic replication methods.     

基于热点数据块的动态副本调整策略

副本管理策略对于分布式存储系统的可用性、可靠性和系统整体性能有至关重要的作用。本文针对基于文件的动态副本调整策略的不足,提出了一种基于热点数据块的动态副本调整策略。根据时间局部性原理和数据访问规律,通过对历史访问周期和当前周期赋予不同的权重,数据块下一周期的预测进行访问频率计算,接下来基于计算出的预测访问频率对数据块进行热点判定。结合HDFS中数据访问规律近似二八定律的特点和热点数据块的判定结果,来确定数据块的调整阈值。最后,分3个步骤对基于热点数据块的动态副本调整策略进行性详细设计。实验结果表明,本文提出的基于热点数据块的动态副本调整策略在数据访问效率和集群存储资源利用率两方面有了明显提升。