Scalable consistency protocols for distributed services

A common way to address scalability requirements of distributed services is to employ server replication and client caching of objects that encapsulate the service state. The performance of such a system could depend very much on the protocol implemented by the system to maintain consistency among object copies. We explore scalable consistency protocols that never require synchronization and communication between all nodes that have copies of related objects. We achieve this by developing a novel approach called local consistency (LC). LC based protocols can provide increased flexibility and efficiency by allowing nodes control over how and when they become aware of updates to cached objects. We develop two protocols for implementing strong consistency using this approach and demonstrate that they scale better than a traditional invalidation based consistency protocol along the system load and geographic distribution dimensions of scale     

分布式关系型数据库研究与金融行业应用

数据库作为金融信息化建设的重要组成部分, 需要面对持续的业务量增长、高度可用性和扩展性等挑战,而以MySQL、Oracle等为代表的传统数据库单点架构, 在可用性、扩展性和存储能力上已经无法满足当前的金融服务要求. 分布式数据库的出现, 旨在解决单机数据库所面临的各种挑战, 提供更加灵活的架构, 保障系统稳定运行. 为此, 本文在结合实际的金融业务需求下, 研究实现了具有分布式事务支持、分布式SQL引擎、混合事务分析处理等特点的分布式数据库. 系统采用全组件的冗余设计, 通过类Raft增强一致性算法保证了存储层高可用和数据强一致, 同时利用基于Zookeeper的集群调度方案保证调度层的高可用.     

Clustering support and replication management for scalable network services

The ubiquity of the Internet and various intranets has brought about widespread availability of online services and applications accessible through the network. Cluster-based network services have been rapidly emerging due to their cost-effectiveness in achieving high availability and incremental scalability. We present the design and implementation of the Neptune middleware system that provides clustering support and replication management for scalable network services. Neptune employs a loosely connected and functionally symmetric clustering architecture to achieve high scalability and robustness. It shields the clustering complexities from application developers through simple programming interfaces. In addition, Neptune provides replication management with flexible replication consistency support at the clustering middleware level. Such support can be easily applied to a large number of applications with different underlying data management mechanisms or service semantics. The system has been implemented on Linux and Solaris clusters, where a number of applications have been successfully deployed. Our evaluations demonstrate the system performance and smooth failure recovery achieved by proposed techniques.     

Remote service discovery and binding architecture for soft real-time QoS in indoor location-based service

Indoor location-based service (LBS) is generally distinguished from web services that have no physical location and user context. In particular, various resources have dynamic and frequent mobility in indoor environments. In addition, an indoor LBS includes numerous service lookups being requested concurrently and frequently from several locations, even through a network infrastructure requiring high scalability in indoor environments. The traditional centralized LBS approach needs to maintain a geographical map of the entire building or complex in its central server, which can cause low scalability and traffic congestion. This paper presents a self-organizing and fully distributed indoor LBS platform with regional cooperation among devices. A service lookup algorithm based on the proposed distributed architecture searches for the shortest physical path to the nearest service resource. A continuous service binding mechanism guarantees a probabilistic real-time QoS regardless of dynamic and frequent mobility in a soft real-time system such as an indoor LBS. Performance evaluation of the proposed algorithm and platform is compared to the traditional centralized architecture in the experimental evaluation of scalability and real test bed environments.     

基于SOA架构的数据服务设计方法与实现

随着企业级应用的不断发展,传统应用架构已经无法满足可重用和可维护性的问题,文章深入分析了面向服务的体系结构(SOA)的相关理论,提出了一种基于SOA架构的数据服务设计方法,此服务组件架构和服务数据对象架构相结合的方式对外提供数据访问服务,在服务端实现了服务组件和数据对象的集成,既屏蔽了数据层的内部结构,又做到了业务层和数据层设计的分离。     

MaDViWorld: a software framework for massively distributed virtual worlds

The MaDViWorld project represents an original attempt to define an appropriate software architecture for supporting massively distributed virtual world systems. A non‐massively distributed virtual world system is typically engineered as a client–server application for which a single server or more rarely a small cluster of servers contain all the world pertinent data and assume the world accessibility, consistency and persistence. On the client side, many of them enable interaction with the other users and the various objects of the world. The main originality of our approach resides in the fact that the server part of the proposed system is no more limited to a few centralized servers, but can be distributed on arbitrarily many of them. Indeed, MaDViWorld, the prototypal software framework already implemented using Java and RMI by our group, allows for creating the rooms of a given world on several machines, each running the server application. It is then possible to connect the rooms by way of simple doors and to populate them with active objects. Finally, avatars managed by the client application visit the rooms and interact with the active objects either directly on the remote host or locally by cloning or transporting them first to the client machine. This paper draws from the experience gained with the development of our first prototype in order to discuss, both at the user's and the vi‐world developer's level, the main software engineering issues related to the implementation of such massively distributed virtual world systems. Copyright © 2002 John Wiley & Sons, Ltd.     

An LDAP-based User Modeling Server and its Evaluation

Representation components of user modeling servers have been traditionally based on simple file structures and database systems. We propose directory systems as an alternative, which offer numerous advantages over the more traditional approaches: international vendor-independent standardization, demonstrated performance and scalability, dynamic and transparent management of distributed information, built-in replication and synchronization, a rich number of pre-defined types of user information, and extensibility of the core representation language for new information types and for data types with associated semantics. Directories also allow for the virtual centralization of distributed user models and their selective centralized replication if better performance is needed. We present UMS, a user modeling server that is based on the Lightweight Directory Access Protocol (LDAP). UMS allows for the representation of different models (such as user and usage profiles, and system and service models), and for the attachment of arbitrary components that perform user modeling tasks upon these models. External clients such as user-adaptive applications can submit and retrieve information about users. We describe a simulation experiment to test the runtime performance of this server, and present a theory of how the parameters of such an experiment can be derived from empirical web usage research. The results show that the performance of UMS meets the requirements of current small and medium websites already on very modest hardware platforms, and those of very large websites in an entry-level business server configuration.The UMUAI managing editor for this paper was Sandra Carberry, University of Delaware.     

分布式系统中数据复制的研究与应用

维持数据的一致性是复制数据库并发执行的关键。介绍分析了MS SQL Server 2000中内置的数据复制构件以及MS SQL Server 2000的事务模型,最后结合实际的应用软件开发对数据复制的并发控制原理与措施进行了相关实现。     

一种复制数据库恢复协议研究

数据复制是提高数据库系统性能和可用性的重要技术。近年来出现的基于组通信技术的数据库复制协议较之传统的数据库复制协议因其实现简单灵活、性能较优,在构建实用复制数据库系统时得到广泛应用,维持各节点副本一致性是数据复制技术研究的核心问题。本文讨论了当故障节点恢复后重新加入系统或增加新节点后如何恢复、维护系统节节点副本间的一致性的问题。本文提出了一个针对分布复制数据库系统的恢复协议,该协议结合基于组通信技术的复制协议,可在不影响系统正常事务处理的情况下,实现故障恢复后节点或全新节点重新加入系统时的系统正确恢复,并给出了理论证明。同时还证明,该恢复协议对于分布在不可靠网络上的复制数据库的恢复问题同样适用。     

数据复制与一致性

数据复制是分布式数据库和数据仓库中常用的方法,以提高数据的可用性和系统的性能。文中介绍了在数据库系统中广泛使用的数据复制方法,分析了其优缺点和适用的范围,重点讨论了异步复制方法中的一致性恢复、一致性偏离的度量和数据存取的限制等问题,给出了数据复制策略的评价目标。     

Ceph分布式系统的ISCSI高可用集群

在分布式存储集群环境中,为了兼容现有存储协议,提高集群可扩展性,通常都会支持ISCSI存储协议.而现有ISCSI服务的高可用主要通过主从备份的方式实现,该方式会导致资源利用不充分,容易造成单节点负载过重等情况.本文基于Ceph分布式存储系统的优势,采用raft一致性算法等设计并实现ISCSI高可用集群.ISCSI集群使用自定义的节点选择策略实现服务负载均衡,并通过raft分布式协议实现服务故障迁移.实验表明,本文提出的方案是有效的,能够实现服务故障迁移,保证集群负载均衡.     

On the Design of Distributed Object Placement and Load Balancing Strategies in Large-Scale Networked Multimedia Storage Systems

In a large-scale multimedia storage system (LMSS) where client requests for different multimedia objects may have different demands, the placement and replication of the objects is an important factor, as it may result in an imbalance in server loading across the system. Since replication management and load balancing are all the more crucial issues in multimedia systems, in the literature, these problems are handled by centralized servers. Each object storage server (OSS) responds to the requests that come from the centralized servers independently and has no communication with other OSSs in the system. In this paper, we design a novel distributed load balancing strategy for LMSS, in which OSSs can cooperate to achieve higher performance. Such OSS modeled as an M/G/m system can replicate the objects to and balance the requests among other servers to achieve a near-optimal average waiting time (AWT) of the requests in the system. We validate the performance of the system via rigorous simulations with respect to several influencing factors and prove that our proposed strategy is scalable, flexible, and efficient for real-life applications.     

Design and implementation of a secure wide-area object middleware

Wide-area service replication is becoming increasingly common, with the emergence of new operational models such as content delivery networks and computational grids. This paper describes the security architecture for Globe, an object-based middleware specifically designed to support dynamic replication of services over wide-area networks. Replication introduces a series of new security issues, including the need to restrict replica privileges with respect to method execution, and protection of distributed objects against malicious hosts running instances of their code. Our modular security design addresses these new threats, as well as a broad range of traditional ones, and is validated through a series of performance measurements. Additional contributions include a novel authentication mechanism specifically designed for wide-area deployment, which combines some of the best features of public key authentication protocols (reliance on an offline trusted third party in particular) with the computational efficiency characteristic to symmetric key schemes.     

Java based replicated server objects and their remote method invocations in a distributed environment

A distributed system is said to be fault‐tolerant if it is able to provide important services despite partial failures of the computers or software objects in the system. These systems are needed to support applications such as remote access and control, virtual mobile offices and wide area collaborative systems where there are chances of failures in the network and software objects. Fault‐tolerance is usually achieved by replicating the objects in the system. Traditional distributed applications constructed using Java RMI (remote method invocation) are not fault‐tolerant because of the lack of support of object replication. The objective of the present work is to design a remote method invocation that supports server object replication. To provide a fault‐tolerant service to the remote client objects, server objects are actively replicated. The problems associated with the method invocation in the context of active server object replication are presented and solutions are discussed and implemented. Copyright © 2001 John Wiley & Sons, Ltd.     

A design of RFTOG model for distributed real-time applications

In this paper, we design the Real-Time Fault-Tolerant Object Group (RFTOG) model that supports the grouping of distributed objects that are required for distributed application. The proposed model basically provides two services. One is the group management service, which supports both consistency maintenance and transparency of the replicated objects with a variety of replication mechanisms. It also provides the load balancing of distributed applications. The other is real-time service in an object group. When the clients request the service to the service object selected through the load balance, this service guarantees the service execution within deadline for the clients’ requests. We develop the Naval Air Defense System (NADS) simulator for verifying the effectiveness of the services proposed by the RFTOG model.     

关于多层分布式系统体系技术的探讨

传统的计算机应用系统体系是Client/Server结构模式,这种多对一的信息共享方式,造成了服务器系统的异常复杂和难以维护。随着网络服务需求的剧增,传统的信息共享模式已无法再满足新业务的需要,于是分布式应用系统技术(B/S结构)慢慢成为网络服务系统的首选结构。     

分布式架构在银行核心业务系统的应用

相对于主机集中式架构,以X86和云计算为基础的分布式架构在扩展性、低成本方面的优势明显,随着技术的进步,其可用性也在逐步提升,已经成为主流的架构方案.针对在数据库层应用分布式架构,本文从分库分表、读写分离、数据共享和访问性能优化、高效运维等方面,提出了一套适用于银行核心业务系统的分布式架构解决方案,并已取得应用实践的成功.