移动P2P数据分发技术研究*

移动P2P数据分发技术将P2P模型应用到移动网络中,通过节点间的相互配合来提高系统的可靠性、传输速度和扩展性,目前已成为无线通信的重点研究领域。但是由于移动网络的复杂性,现有的移动P2P数据分发技术在实际应用中仍然存在很多问题。对近年来该领域的一些重点技术如Gossip算法、网络编码、纠错码进行了介绍,并在可靠性、传输速度和扩展性方面对它们进行了分析,针对其在网络动态适应性、网络融合、节点合作度等方面的不足提出了今后的研究方向。     

基于网络编码的对等网络互惠资源共享方法

传统网络消息传播基于存储转发路由机制,网络节点对于网络消息不进行任何处理,网络编码理论允许节点对传播的信息进行编码处理.基于此,利用网络编码数据包能携带更多网络信息的特点,提出一种对等网络环境下的互惠资源共享方法,保证对等网络资源共享的高可靠性和鲁棒性,并通过仿真实验加以实现.仿真实验结果表明,网络编码互惠资源共享方法...     

DiVers: An erasure code based storage architecture for versioning exploiting sparsity

We propose a differential versioning based data storage (DiVers) architecture for distributed storage systems, which relies on a novel erasure coding technique that exploits sparsity across versions. The emphasis of this work is to demonstrate how sparsity exploiting codes (SEC), originally designed for I/O optimization, can be extended to significantly reduce storage overhead in a repository of versioned data. In addition to facilitating reduced storage, we address some key reliability aspects for DiVers such as (i) mechanisms to deploy the coding technique with arbitrarily varying size of data across versions, and (ii) investigating the right allocation strategy for the encoded blocks over a network of distributed nodes across different versions so as to achieve the best fault tolerance. We also discuss system issues related to the management of data structures for accessing and manipulating the files over the differential versions.     

P2P storage systems: Study of different placement policies

In a P2P storage system using erasure codes, a data block is encoded in many redundancy fragments. These fragments are then sent to distinct peers of the network. In this work, we study the impact of different placement policies of these fragments on the performance of storage systems. Several practical factors (easier control, software reuse, latency) tend to favor data placement strategies that preserve some degree of locality. We compare three policies: two of them are local, in which the data are stored in logical neighbors, and the other one, global, in which the data are spread randomly in the whole system. We focus on the study of the probability to lose a data block and the bandwidth consumption to maintain such redundancy. We use simulations to show that, without resource constraints, the average values are the same no matter which placement policy is used. However, the variations in the use of bandwidth are much more bursty under the local policies. When the bandwidth is limited, these bursty variations induce longer maintenance time and henceforth a higher risk of data loss. We then show that a suitable degree of locality could be introduced in order to combine the efficiency of the global policy with the practical advantages of a local placement. Additionally, we propose a new external reconstruction strategy that greatly improves the performance of local placement strategies. Finally, we give analytical methods to estimate the mean time to the occurrence of data loss for the three policies.     

Hierarchical codes: A flexible trade-off for erasure codes in peer-to-peer storage systems

Redundancy is the basic technique to provide reliability in storage systems consisting of multiple components. A redundancy scheme defines how the redundant data are produced and maintained. The simplest redundancy scheme is replication, which however suffers from storage inefficiency. Another approach is erasure coding, which provides the same level of reliability as replication using a significantly smaller amount of storage. When redundant data are lost, they need to be replaced. While replacing replicated data consists in a simple copy, it becomes a complex operation with erasure codes: new data are produced performing a coding over some other available data. The amount of data to be read and coded is d times larger than the amount of data produced, where d, called repair degree, is larger than 1 and depends on the structure of the code. This implies that coding has a larger computational and I/O cost, which, for distributed storage systems, translates into increased network traffic. Participants of Peer-to-Peer systems often have ample storage and CPU power, but their network bandwidth may be limited. For these reasons existing coding techniques are not suitable for P2P storage. This work explores the design space between replication and the existing erasure codes. We propose and evaluate a new class of erasure codes, called Hierarchical Codes, which allows to reduce the network traffic due to maintenance without losing the benefits given by traditional erasure codes.     

P2P存储网络中的冗余维持

在P2P存储网络中,为保证数据安全可靠,当网络中数据丢失后需生成新的冗余数据.因此,如何快速生成新的冗余数据并保持数据的可靠性就成为评价存储网络性能的关键.网络编码作为一项新兴技术,已被证明可以提高网络吞吐率和鲁棒性.将网络编码思想引入冗余数据维持,可以加快冗余生成速度,从而提高数据可靠性.通过理论分析和仿真给出了快速修复冗余数据的方法、开销和相应的传输策略.仿真结果表明,较传统修复方式所提出的快速修复方法可以节省60%以上的传输时间.     

一种基于语义P2P的Web服务发现模型

提出了一种基于语义P2P的Web服务发现和选择模型，引入一种新的本体编码方式，根据分类服务注册节点的本体编码对其在P2P网络内进行组织，实现了P2P层内的语义支持发现。本文创新性的在P2P发现方面加入了本体信息，使得Web服务在P2P层内支持语义发现，提出的模型结合了集中式Web服务发现方式和分布式Web服务方式发现的优点，提高了服务发现的准确性和即时性。     

一种分层P2 P流媒体系统重叠网的构建策略

基于Peer-to-Peer(P2P)技术的流媒体应用具有部署效率高及可扩展性好等突出优势。而采用分层视频编码技术的P2P流媒体系统把原视频流分解为多个视频层数据进行分发,让节点能够选择与自己带宽资源相匹配的视频质量,使其很好地适应节点的异构性。但是各分层视频数据传输的路径存在较大差异,使重叠网构建策略面临更大的挑战。因此定义了基于分层视频编码技术环境下的P2P流媒体重叠网络构建问题,并证明该问题是一个NP难问题。提出了一种构建重叠网的集中启发式算法,同时还提出了一种基于视频组(Streaming Group)的分布式重叠网络构建策略。通过大规模网络仿真实验验证了基于该分布式重叠网构建策略的分层流媒体系统具备低服务器带宽占用、高数据获取率等优点。     

Robust Redundancy Scheme for the Repair Process: Hierarchical Codes in the Bandwidth-Limited Systems

High performance computing can be well supported by the Grid or cloud computing systems. However, these systems have to overcome the failure risks, where data is stored in the “unreliable” storage nodes that can leave the system at any moment and the nodes’ network bandwidth is limited. In this case, the basic way to assure data reliability is to add redundancy using either replication or erasure codes. As compared to replication, erasure codes are more space efficient. Erasure codes break data into blocks, encode these blocks and distribute them into different storage nodes. When storage nodes permanently or temporarily abandon the system, new redundant blocks must be created to guarantee the data reliability, which is referred to as repair. Later when the churn nodes rejoin the system, the blocks stored in these nodes can reintegrate the data group to enhance the data reliability. For “classical” erasure codes, generating a new block requires to transmit a number of k blocks over the network, which brings lots of repair traffic, high computation complexity and high failure probability for the repair process. Then a near-optimal erasure code named Hierarchical Codes, has been proposed that can significantly reduce the repair traffic by reducing the number of nodes participating in the repair process, which is referred to as the repair degree d. To overcome the complexity of reintegration and provide an adaptive reliability for Hierarchical Codes, we refine two concepts called location and relocation, and then propose an integrated maintenance scheme for the repair process. Our experiments show that Hierarchical Code is the most robust redundancy scheme for the repair process as compared to other famous coding schemes.     

Insight into redundancy schemes in DHTs

In order to provide high data availability in peer-to-peer (P2P) DHTs, proper data redundancy schemes are required. This paper compares two popular schemes: replication and erasure coding. Unlike previous comparison, we take user download behavior into account. Furthermore, we propose a hybrid redundancy scheme, which shares user downloaded files for subsequent accesses and utilizes erasure coding to adjust file availability. Comparison experiments of three schemes show that replication saves more bandwidth than erasure coding, although it requires more storage space, when average node availability is higher than 47%; moreover, our hybrid scheme saves more maintenance bandwidth with acceptable redundancy factor.