信息网格中基于Agent的数据复制方法探讨

说明了有效的数据复制机制和副本管理策略是信息网格研究的重要内容.概述了应用Agent技术实现信息网格中数据复制的基本方法.探讨了在全国人口信息系统中基于Agent的数据复制方法的设计和实现,利用Agent的自主性、移动性和交互性等特点,实现上层创建下层副本,及同层间副本的创建.采用数据格式转换系统解决底层数据库的异构问题.重点分析了基于Agent的推动和拉动复制方法的实现特点,说明了基于Agent的数据复制方法在信息网格中的合理性和高效性.     

教育资源网格模型及副本创建策略

教育资源网格是解决目前分布式教育资源共享问题的有效手段.针对中小学教育资源共享问题,提出了层次式的教育资源网格模型,定义了各层节点的功能.通过与欧洲数据网格对比,分析了教育资源网格的特点.基于层次式的教育资源网格,对影响副本创建策略性能的因素进行了分析,然后引入网络带宽和文件大小两个参数,提出了一种动态副本创建策略(dynamic replica creation strategy,简称EDRS).利用数据网格模拟工具OptorSim构建了教育资源网格虚拟环境,分析比较了EDRS策略与Caching-LRU策略、Caching-LFU策略和基于经济模型的副本创建策略的性能.最后,综合各项指标分析了不同策略对教育资源网格系统性能的影响.结果表明,EDRS策略在教育资源网格应用中有着更好的系统性能.     

数据网格中的副本定位

在数据网格中,数据副本的定位是副本管理的一个关键问题,是数据网格环境中数据访问和管理的基础。本文分析了当前对数据副本定位所采用的集中式和分布式的定位方法中存在的不足,提出了一种混合式的副本定位策略,将数据副本的定位信息分散地存储于不同的域定位节点中,通过逻辑上集中的逻辑文件定位索引实现了对数据副本的准确定位。     

多媒体组通信机制——服务质量受控型组管道

提出一种服务质量受控型组管道,作为基于服务质量的平台层多媒体组通信抽象,以支持分布式多媒体组通信机制的描述与开发.给出了服务质量受控型组管道的形式定义,详细讨论了其工作过程.该抽象已被成功地用于指导分布式多媒体信息系统、计算机会议系统、远程教学系统等系统中的组通信机制的开发工作.实验结果表明,服务质量受控型组管道可以作为分布式多媒体组通信机制的开发指南.     

基于分类广告的网格副本选择

在分布式网格环境中，Globus数据网格为其中资源、数据的存储管理提供了一种可扩展的机制。网格中的科学应用一般都需要以不同的服务方式来访问大量的数据。文章详细阐述了网格中ClassAds的匹配机制和资源描述。以及非集中化的存储代理副本选择过程。     

Elsa:一种面向跨区域架构的无协调分布式键值存储系统

作为具备高性能和高可伸缩性的分布式存储解决方案,键值存储系统近年来被广泛采用,例如Redis、MongoDB、Cassandra等.分布式存储系统中广泛使用的多副本机制一方面提高了系统吞吐量和可靠性,但同时也增加了系统协调和副本一致性的额外开销.对于跨域分布式系统来说,远距离的副本协调开销甚至可能成为系统的性能瓶颈,降低系统的可用性和吞吐量.提出分布式键值存储系统Elsa,这是一种面向跨区域架构的无协调键值存储系统. Elsa在保证高性能和高可拓展性的基础上,采用无冲突备份数据结构(CRDT)技术来无协调的保证副本间的强最终一致性,降低了系统节点间的协调开销.在阿里云上构建了跨4数据中心8节点的跨区域分布式环境,进行了大规模分布式性能对比实验,实验结果表明:在跨域的分布式环境下,对于高并发争用的负载, Elsa系统的性能具备明显的优势,最高达到MongoDB集群的7.37倍, Cassandra集群的1.62倍.     

基于区域性分割索引节点的分布式副本定位方法

提出了一种在数据网格环境下的基于区域性分割索引节点的分布式副本定位方法。利用将副本定位索引信息按着区域特点或访问的频繁程度分布存储在多个索引节点上,同时利用本地副本定位节点完成本地副本的查询。此方法实现简单且具有区域特性。     

P2P分布式存储系统

设计了一个基于P2P的分布式存储系统.该存储系统采用高可扩展的P2P体系结构,将大量分散的节点组织成一个逻辑网络,充分利用原先被忽视的端系统资源,构建大规模分布式存储系统.该存储系统采用高效的结构化P2P路由机制、动态自适应的副本管理、信任机制和激励机制为用户提供高效、可靠的分布式存储服务.     

一种多媒体通信机制--服务质量受控型管道

提出一种多媒体通信机制--服务质量受控型管道,作为基于服务质量的平台层点对点多媒体通信抽象,以支持分布式多媒体应用的描述与开发.给出了服务质量受控型管道的形式定义,详细讨论了其工作过程.该抽象已经成功地应用于分布式多媒体信息点播系统和分布式多媒体信息系统等.实践表明,服务质量受控型管道既可以直接支持点对点多媒体通信,也可以作为多媒体组通信抽象的基础.     

一种基于文档副本局部性的分布式检索算法研究

摘 要: 分布式信息检索是信息检索领域的重要研究内容。为了提高分布式信息检索的性能,提出了一种基于文档副本局部性的分布式检索方法。对于任一站点,如果将查询结果中的非本地文档建立本地副本,那么可以减少查询处理中站点之间的查询转发,从而相应的提高信息检索的性能。基于该思想,将分布式信息检索中的副本放置转化为查询的局部性问题,建立了相应的优化模型,并针对不同的副本放置模型提出了相应的副本选择及放置策略。最后通过模拟实验验证表明,本文提出的方法与相关方法相比较既提高了查询结果的准确性,又减小了查询的响应时间。     

数据网格中基于MAS的复制管理研究

数据网格是网格环境下的一种数据管理和存储架构,通常使用数据复制技术来获得更好的数据访问效率和容错性能,提出了一种基于MAS的复制管理模型,解决数据网格中数据高度自治和动态带来的管理难题,探讨了基于MAS的实现架构,给出了Agent的结构和协作过程,将复制管理和复制优化策略封装于Agent智能模块中,结合实际应用使用Optorsim仿真器对模型和复制优化策略进行分析,并对经济模型的基于二项分布的估价函数进行了改进,仿真结果表明模型能够提供高效的复制管理服务.     

Towards a framework for large-scale multimedia data storage and processing on Hadoop platform

Cloud computing techniques take the form of distributed computing by utilizing multiple computers to execute computing simultaneously on the service side. To process the increasing quantity of multimedia data, numerous large-scale multimedia data storage computing techniques in the cloud computing have been developed. Of all the techniques, Hadoop plays a key role in the cloud computing. Hadoop, a computing cluster formed by low-priced hardware, can conduct the parallel computing of petabytes of multimedia data. Hadoop features high-reliability, high-efficiency, and high-scalability. The numerous large-scale multimedia data computing techniques include not only the key core techniques, Hadoop and MapReduce, but also the data collection techniques, such as File Transfer Protocol and Flume. In addition, distributed system configuration allocation, automatic installation, and monitoring platform building and management techniques are all included. As a result, only with the integration of all the techniques, a reliable large-scale multimedia data platform can be offered. In this paper, we introduce how cloud computing can make a breakthrough by proposing a multimedia social network dataset on Hadoop platform and implementing a prototype version. Detailed specifications and design issues are discussed as well. An important finding of this article is that we can save more time if we conduct the multimedia social networking analysis using Cloud Hadoop Platform rather than using a single computer. The advantages of cloud computing over the traditional data processing practices are fully demonstrated in this article. The applicable framework designs and the tools available for the large-scale data processing are also proposed. We show the experimental multimedia data including data sizes and processing time.     

Vega: A Computer Systems Approach to Grid Computing

In this paper, we contrast four approaches for Grid computing, and discuss a computer systems approach in detail. This approach views a Grid as a distributed computer system, and its main concerns are systems abstractions and constructs, such as the Grid equivalents of computer architecture, address space, process, device, file system, user/developers interface. Following this methodology, we identified several essential issues, developed a Vega Grid systems architecture, and proposed several systems techniques such as Grid routers, Grid address spaces, Grid process (grip), Grid community (agora), and a Grid Service Markup Language (GSML) software suite. We evaluated the computer systems approach and the associated techniques by discussing an OSGA-compliant Grid system software implementation and a travel agency example.     

Job scheduling and dynamic data replication in data grid environment

Data Grid is a geographically distributed environment that deals with large-scale data-intensive applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. Data replication is another key optimization technique for reducing access latency and managing large data by storing data in a wisely manner. In this paper, two algorithms are proposed: first, a novel job scheduling algorithm called Combined Scheduling Strategy (CSS) that considers the number of jobs waiting in queue, the location of required data for the job, and computational capability; second, a dynamic data replication strategy called Dynamic Hierarchical Replication Algorithm (DHRA) that improves file access time. DHRA stores each replica in an appropriate site, i.e., appropriate site in the requested region that has the highest number of access for that particular replica. Also, it can minimize access latency by selecting the best replica when various sites hold replicas of datasets. The simulation results demonstrate the proposed replication and scheduling strategies give better performance compared to the other algorithms.     

Fault-tolerant grid architecture and practice

Grid computing emerges as effective technologies to couple geographically dis-tributed resources and solve large-scale computational problems in wide area networks. The fault tolerance is a significant and complex issue in grid computing systems. Various techniques have been investigated to detect and correct faults in distributed computing systems. Unreliable fault detection is one of the most effective techniques. Globus as a grid middleware manages resources in a wide area network. The Globns fault detection service uses the well-known techniques basedon unreliable fault detectors to detect and report component failures. However, more powerful techniques are required to detect and correct both system-level and application-level faults in agrid system, and a convenient toolkit is also needed to maintain the consistency in the grid. Afault-tolerant grid platform (FTGP) based on an unreliable fault detector and the Globus faultdetection service is presented in this paper. The platform offers effective strategies in such threeaspects as grid key components, user tasks, and high-level applications.     

面向知识网格的本体学习研究

网格计算正在从单纯的面向大型计算的分布式资源共享发展为一种面向服务的架构,以实现透明而可靠的分布式系统集成。网格智能是指如何获取、预处理、表示和集成不同层次的网格服务(如HTML/XML/RDF/OWL文档、服务响应时间和服务质量等)的数据和信息,并最终转换为有用的智能(知识)。因为高层知识将在未来的网格应用起到越来越重要的作用,本体是知识网格实现的关键。文章提出了一种实现从Web文档中本体(半)自动构建的本体学习框架WebOntLearn,并讨论了本体学习中领域概念的抽取、概念之间关系的抽取和分类体系的自动构建等关键技术。     

Combination of data replication and scheduling algorithm for improving data availability in Data Grids

Data Grid is a geographically distributed environment that deals with large-scale data-intensive applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. Data replication is another key optimization technique for reducing access latency and managing large data by storing data in a wisely manner. In this paper two algorithms are proposed, first a novel job scheduling algorithm called Combined Scheduling Strategy (CSS) that uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers the number of jobs waiting in queue, the location of required data for the job and the computing capacity of sites. Second a dynamic data replication strategy, called the Modified Dynamic Hierarchical Replication Algorithm (MDHRA) that improves file access time. This strategy is an enhanced version of Dynamic Hierarchical Replication (DHR) strategy. Data replication should be used wisely because the storage capacity of each Grid site is limited. Thus, it is important to design an effective strategy for the replication replacement. MDHRA replaces replicas based on the last time the replica was requested, number of access, and size of replica. It selects the best replica location from among the many replicas based on response time that can be determined by considering the data transfer time, the storage access latency, the replica requests that waiting in the storage queue and the distance between nodes. The simulation results demonstrate the proposed replication and scheduling strategies give better performance compared to the other algorithms.     

面向资源广域分布环境的数据复制定位

在资源广域分布的环境中，数据复制和定位是提高数据访问性能的关键技术之一。该文提出了一种分布式、自适应复制定位机制(ALM)，该机制基于复制定位信息分发和网络缓存思想，采用分层的分布式复制目录系统和软状态协议实现。其中，复制目录系统分布存储复制定位信息，软状态协议分发复制定位信息及其改变。因此，ALM具有很好的分布性和自适应性。ALM在复制目录的设计中引入了Web缓存的功能，使得ALM较其它机制具有更广泛的适用性。     

Modeling Replica Availability in Large Data Grids

Large Grid systems not only provide massive aggregated computing power but also an unprecedented amount of distributed storage space. Unfortunately, the dynamic behavior of the Grid, caused by varying resource availability, unpredictable data updates, and the impact of local site policies makes it difficult to exploit the full capabilities of Data Grids.We present an analytical model for determining the optimal number of replica servers, catalog servers, and catalog sizes to guarantee a given overall reliability in the face of unreliable components. Our model captures the characteristics of peer-to-peer-like environments as well as that of Grid systems. Empirical simulations confirm the accuracy of our analytical model.     

On Fairness, Optimizing Replica Selection in Data Grids

The emergence of scientific applications that produce huge volume of data files requires special attention, and leads to the problem of how to manage and share such data files in wide area properly. In large-scale Grid, data replication provides a suitable solution for managing data files where data reliability and data availability are enhanced. Replica selection is one of the major functions of data replication that decides which replica location is the best for Grid users. In this paper, we address the replica selection problem in a Grid environment where the users are competing for the limited data resource. Thus, our aim is to establish fairness among the users in the selection decisions. Since the criteria that play a role in the selection process conflict with each other and produce heterogeneous values, the Analytical Hierarchy Process is used to solve this optimization problem. The proposed system validity and performance are evaluated by using a simulation. The simulation results were produced and discussed in this paper.