面向大规模二部图的分布式Tip分解算法

Tip分解作为图数据管理领域的热点研究问题,已被广泛应用于文档聚类和垃圾邮件组检测等实际场景中.随着图数据规模的爆炸式增长,单机内存已无法满足其存储需求,亟需研究分布式环境下Tip分解技术.现有分布式图计算系统的通信模式无法适用于二部图,为此,首先提出一种基于中继的通信模式,以实现分布式环境下处理二部图时消息的有效传递...     

Linear structure of bipartite permutation graphs and the longest path problem

The class of bipartite permutation graphs is the intersection of two well known graph classes: bipartite graphs and permutation graphs. A complete bipartite decomposition of a bipartite permutation graph is proposed in this note. The decomposition gives a linear structure of bipartite permutation graphs, and it can be obtained in O(n) time, where n is the number of vertices. As an application of the decomposition, we show an O(n) time and space algorithm for finding a longest path in a bipartite permutation graph.     

基于顶点组重分配的动态增量图划分算法

图划分是分布式图计算中的一项基础工作, 其作用是将大规模图进行划分并分配到集群中的不同机器上. 图划分的质量对分布式图计算的性能有很大的影响, 其目标是降低负载平衡和最小化边割. 如今, 现实中的图数据通常呈动态增长态势, 这就需要一种能够处理动态增量图的划分方法, 在图数据动态增长的过程中确保划分的质量不受影响. 目前虽然有一些动态图划分算法被提出, 但它们不能同时专注于实时处理动态变化和获得高质量的划分结果. 提出基于顶点组重分配的动态增量图划分算法(ED-IDGP)来解决大规模动态增量图的划分问题. 在ED-IDGP算法中, 设计实时处理4种不同单元更新类型的动态处理器, 并在每次处理完单元更新后通过在分区发生动态变化的附近执行局部优化器进一步提高图划分的质量. 在ED-IDGP的局部优化器中, 利用基于改进标签传播算法的顶点组搜索策略搜索顶点组, 并利用提出的顶点组移动增益公式衡量最有益的顶点组, 将该顶点组移动到目标分区中做优化. 在真实数据集上从不同的角度和度量指标评估了ED-IDGP算法的性能和效率.     

Adaptive Algorithms for Join Processing in Distributed Database Systems

Distributed query processing algorithms usually perform data reduction by using a semijoin program, but the problem with these approaches is that they still require an explicit join of the reduced relations in the final phase. We introduce an efficient algorithm for join processing in distributed database systems that makes use of bipartite graphs in order to reduce data communication costs and local processing costs. The bipartite graphs represent the tuples that can be joined in two relations taking also into account the reduction state of the relations. This algorithm fully reduces the relations at each site. We then present an adaptive algorithm for response time optimization that takes into account the system configuration, i.e., the additional resources available and the data characteristics, in order to select the best strategy for response time minimization. We also report on the results of a set of experiments which show that our algorithms outperform a number of the recently proposed methods for total processing time and response time minimization.     

基于加权二部图匹配的中文段落相似度计算

为了改进传统以向量空间模型（VSM）为代表的基于词频统计的方法在中文段落相似度计算时存在的精度不高问题,在基于加权二部图匹配的思想上提出了一种计算中文段落之间相似度的方法。该方法将相似度计算分为段落和句子两个层次,将句子作为简单段落看待,也使用二部图匹配进行相似度计算。首先利用句子主干词汇提取算法来提取句子的主干词汇,将主干词汇作为二部图的顶点,把主干词汇之间的相似度作为二部图顶点之间的权值系数,进行句子相似度的计算。其次,将句子作为加权二部图的顶点,把句子之间的相似度作为二部图顶点之间的权值系数,进行段落之间的相似度计算。实验结果表明,该方法与VSM相比,由于它能准确识别同义词,自动匹配两个在段落中不同位置的相似词语,因而在准确度上有了很大的提高。     

Single-edge monotonic sequences of graphs and linear-time algorithms for minimal completions and deletions

We study graph properties that admit an increasing, or equivalently decreasing, sequence of graphs on the same vertex set such that for any two consecutive graphs in the sequence their difference is a single edge. This is useful for characterizing and computing minimal completions and deletions of arbitrary graphs into having these properties. We prove that threshold graphs and chain graphs admit such sequences. Based on this characterization and other structural properties, we present linear-time algorithms both for computing minimal completions and deletions into threshold, chain, and bipartite graphs, and for extracting a minimal completion or deletion from a given completion or deletion. Minimum completions and deletions into these classes are NP-hard to compute.     

L-PowerGraph: a lightweight distributed graph-parallel communication mechanism

In order to process complex and large-scale graph data, numerous distributed graph-parallel computing platforms have been proposed. PowerGraph is an excellent representative of them. It has exhibited better performance, such as faster graph-processing rate and higher scalability, than others. However, like in other distributed graph computing systems, unnecessary and excessive communications among computing nodes in PowerGraph not only aggravate the network I/O workload of the underlying computing hardware systems but may also cause a decrease in runtime performance. In this paper, we propose and implement a mechanism called L-PowerGraph, which reduces the communication overhead in PowerGraph. First, L-PowerGraph identifies and eliminates the avoidable communications in PowerGraph. Second, in order to further reduce the required communications L-PowerGraph proposes an edge direction-aware master appointment strategy, in which L-PowerGraph appoints the replica with both incoming and outgoing edges as master. Third, L-PowerGraph proposes an edge direction-aware graph partition strategy, which optimally isolates the outgoing edges from the incoming edges of a vertex during the graph partition process. We have conducted extensive experiments using real-world datasets, and our results verified the effectiveness of the proposed mechanism. For example, compared with PowerGraph under Random partition scenario L-PowerGraph can not only reduce up to 30.5% of the communication overhead but also cut up to 20.3% of the runtime for PageRank algorithm while processing Live-journal dataset. The performance improvement achieved by L-PowerGraph over our precursor work, LightGraph, which only reduces the synchronizing communication overhead, is also verified by our experimental results.

     

采用图分解的特征识别算法研究

CAD/CAE模型转换,其关键在于如何将模型分解为最简单元,这些单元往往具有相近的网格划分属性,可以方便估计计算误差和计算时间。基于此提出了基于图分解的特征识别算法,对属性邻接图进行分解,根据分解后的属性邻接图中的连通分量生成体特征。该算法不再局限于特征类型,只要合理控制顶点的可分解性判断就可以得到期望的模型分解结果;同时该算法可以获得体特征,使得可以在特征这一粒度上进行特征删除和替换,以方便地完成模型的简化。     

加权最大频繁子图挖掘算法的研究

如何从大量的图中挖掘出令人感兴趣的子图模式已经成为数据挖掘领域研究的热点之一。传统的频繁子图挖掘方法对满足最小支持度阈值的子图同等对待,但在真实数据库中不同的子图往往具有不同的重要程度。为解决上述问题,提出了一种深度优先的挖掘加权最大频繁子图的新算法。首先给出了一种新的用于计算图的邻接矩阵规范编码的结点排序策略,大大降低了求图规范编码的复杂度,并可以加速子图规范编码匹配的速度。其次,给出了加权最大频繁子图的定义,不仅可以找出较为重要的最大频繁子图,而且可以使挖掘结果同样具有反单调性,从而可加速剪枝。实验结果表明,提出的算法不仅可以有效地减少挖掘结果的数量,而且具有较高的效率。     

面向图计算的运行时系统的消息聚合技术

图计算应用的通信模式以时空随机的点对点细粒度通信为主,但现有高性能计算机的网络系统应对大量细粒度通信时表现不佳,进而影响整体性能。虽然在应用层进行通信优化可以有效提升图计算应用性能,但这会给应用开发人员带来很大的负担,因此提出并实现结构动态的消息聚合技术,通过构建虚拟拓扑的方法在通信路径上增加中间点从而提升消息聚合的效果。传统的消息聚合策略一般仅在通信源或者目的地上进行,聚合机会有限,而所提技术通过灵活调整虚拟拓扑的结构和配置适应了不同硬件条件和应用特征。同时,还提出并实现了面向图计算的有消息聚合的运行时系统,这使得在程序迭代执行时可以动态选择参数,从而减少开发人员负担。在256节点规模的系统上实验的结果显示,使用所提消息聚合技术优化后的典型图计算应用的性能可得到100%以上的提升。     

k-tuple domination in graphs

In a graph G, a vertex is said to dominate itself and all of its neighbors. For a fixed positive integer k, the k-tuple domination problem is to find a minimum sized vertex subset in a graph such that every vertex in the graph is dominated by at least k vertices in this set. The current paper studies k-tuple domination in graphs from an algorithmic point of view. In particular, we give a linear-time algorithm for the k-tuple domination problem in strongly chordal graphs, which is a subclass of chordal graphs and includes trees, block graphs, interval graphs and directed path graphs. We also prove that the k-tuple domination problem is NP-complete for split graphs (a subclass of chordal graphs) and for bipartite graphs.     

A distributed low tree-depth decomposition algorithm for bounded expansion classes

We study the distributed low tree-depth decomposition problem for graphs restricted to a bounded expansion class. Low tree-depth decomposition have been introduced in 2006 and have found quite a few applications. For example it yields a linear-time model checking algorithm for graphs in a bounded expansion class. Recall that bounded expansion classes cover classes of graphs of bounded degree, of planar graphs, of graphs of bounded genus, of graphs of bounded treewidth, of graphs that exclude a fixed minor, and many other graphs. There is a sequential algorithm to compute low tree-depth decomposition (with bounded number of colors) in linear time. In this paper, we give the first efficient distributed algorithm for this problem. As it is usual for a symmetry breaking problem, we consider a synchronous model, and as we are interested in a deterministic algorithm, we use the usual assumption that each vertex has a distinct identity number. We consider the distributed message-passing \(\mathcal {CONGEST}_\mathrm{BC}\) model, in which messages have logarithmic length and only local broadcast are allowed. In this model, we present a logarithmic time distributed algorithm for computing a low tree-depth decomposition of graphs in a fixed bounded expansion class. In the sequential centralized case low tree-depth decomposition linear time algorithm are used as a core procedure in several non-trivial linear time algorithms. We believe that, similarly, low tree-depth decomposition could be at the heart of several non-trivial logarithmic time algorithms.     

基于边权约束法实现接触问题多约束图剖分

为提高接触问题并行计算的效率,分析内力计算和接触计算过程的并行性,提出基于边权约束法构造接触多约束图的方法,对比和分析多约束图剖分算法和双重区域剖分算法的负载平衡和通信性能.数值实验表明,在典型二维模型中多约束图剖分算法的负载平衡性能略低于双重区域剖分算法,但仍可将负载不平衡度控制在较好的范围内,简化并行计算的通信过程,减少总通信量并降低动态通信量比例.     

一种基于树分解的图上点区间编码方法及应用

根据图上节点所在位置与邻居节点特征,可以使用不同策略为每个图上节点进行区间编码,基于区间编码,许多在大型图上的应用如知识图谱查询、智能问答等的处理可以加速或得到准确性上的提升。针对此种情况,提出一种基于树分解算法的图上点区间编码方法,并在大型知识图谱上通过智能问答歧义消除的应用验证该方法的有效性。实验结果表明,该方法能够有效地表达出图上节点的位置特征,并帮助智能问答中的实体消除歧义。     

基于二分图的协作通信中继选择技术*

合理的中继选择对于提升协作通信系统的能效具有重要意义。针对多用户多中继协作通信系统,为获得较高的系统能效并降低系统的复杂度,提出了基于二分图的中继选择策略,将协作通信系统中继选择问题转化为带权二分图的最大匹配问题。首先将用户节点与待选择的中继节点建模为二分图的顶点,根据中继的协作范围确定二分图的边集,然后将不同的协作组合所产生的能效给各边赋权,最后通过KM算法求解该带权二分图的最大匹配。仿真结果表明,相比于其它中继选择算法,该策略能够有效提高系统能效,同时具有低复杂度的优点。     

并行LU分解的通信模式在WDM环网上的波长分配算法

波长分配是光网络设计的基本问题，设计波长分配算法是洞察光网络通信能力的基本方法．不同的并行算法具有不同的通信模式，如何在光互连网上实现这些通信模式，是当前一个颇受关注的研究领域．本文基于WDM环网络，针对矩阵的并行LU分解，构造了一种并行LU分解的通信模式，讨论了将该通信模式嵌入在环形光网络中的波长分配问题．在解决该问题的过程中，得到了将一种特殊的二分图结构的通信模式嵌入在环网中的波长分配算法．通过分析和证明得到了在WDM环网上实现该并行LU分解通信模式所需的最小波长数．     

An optimal bit complexity randomized distributed MIS algorithm

We present a randomized distributed maximal independent set (MIS) algorithm for arbitrary graphs of size n that halts in time O(log n) with probability 1 ? o(n ^?1), and only needs messages containing 1 bit. Thus, its bit complexity par channel is O(log n). We assume that the graph is anonymous: unique identities are not available to distinguish between the processes; we only assume that each vertex distinguishes between its neighbours by locally known channel names. Furthermore we do not assume that the size (or an upper bound on the size) of the graph is known. This algorithm is optimal (modulo a multiplicative constant) for the bit complexity and improves the best previous randomized distributed MIS algorithms (deduced from the randomized PRAM algorithm due to Luby (SIAM J. Comput. 15:1036?C1053, 1986)) for general graphs which is O(log² n) per channel (it halts in time O(log n) and the size of each message is log n). This result is based on a powerful and general technique for converting unrealistic exchanges of messages containing real numbers drawn at random on each vertex of a network into exchanges of bits. Then we consider a natural question: what is the impact of a vertex inclusion in the MIS on distant vertices? We prove that this impact vanishes rapidly as the distance grows for bounded-degree vertices and we provide a counter-example that shows this result does not hold in general. We prove also that these results remain valid for Luby??s algorithm presented by Lynch (Distributed algorithms. Morgan Kaufman 1996) and by Wattenhofer (http://dcg.ethz.ch/lectures/fs08/distcomp/lecture/chapter4.pdf, 2007). This question remains open for the variant given by Peleg (Distributed computing??a locality-sensitive approach 2000).     

Achieving Optimal Inter-Node Communication in Graph Partitioning Using Random Selection and Breadth-First Search

Processing large graph datasets represents an increasingly important area in computing research and applications. The size of many graph datasets has increased well beyond the processing capacity of a single computing node, thereby necessitating distributed approaches. As these datasets are processed over a distributed system of nodes, this leads to an inter-node communication cost problem that negatively affects system performance. Previously proposed algorithms implemented breadth-first search (BFS) for graph searching and focused on the execution, parallel performance and not the communication. In this paper a new methodology is proposed that combines BFS with random selection in order to partition large graph datasets and effectively minimize inter-node communication. The new method is discussed and applied to the single-source shortest path and PageRank algorithms using three graphs that are representative of real-world scenarios. Experimental results show that graph inter-node communication for canonical graphs representative of real-world data is improved up to 42 % in case of Powerlaw graph, up to 27 % in case of Random near K-regular graph (with low degree), and up to 7 % in case of Random near K-regular graph (with high degree).     

D-VSSP:分布式社会网络隐私保护算法

针对传统社会网络隐私保护技术对大规模社会网络数据处理效率较低的问题,提出一种分布式结点分裂匿名社会网络隐私保护算法(Distributed-Vertex Splitting Social Network Privacy Preserving,D-VSSP)。D-VSSP算法利用MapReduce和Pregel-like分布式计算模型处理社会网络图数据。首先基于MapReduce分布式计算模型对大图中的结点的标签信息进行标签平凡化、标签平凡化分组和精确分组处理;然后基于Pregel-like的消息传递机制,选举结点分裂,进行分布式结点分裂匿名。实验结果表明,在 对大规模社会网络数据的处理效率上, D-VSSP算法优于传统算法。     

IEH graphs

We propose a new family of interconnection topology that can be used to design communication architecture for distributed systems with an arbitrary number of computing nodes. The design is based on a novel generalization of the well known hypercube graphs. The proposed topology is shown to be incrementally extensible in steps of 1 (cost of restructuring or adding edges is very low), optimally fault tolerant and its diameter is logarithmic in the number of nodes. Moreover, for any given number of nodes, the difference of the maximum and the minimum degree of a node in the graph is ≤1, i.e., the graph is almost regular. A shortest routing algorithm for the proposed family of graphs has been developed.