面向异质图的在线图划分算法

图划分算法是分布式图计算系统里的重要组成部分, 它将一个图划分为若干子图以便在分布式系统中运行, 并将子图上的点和边数据及子图上的计算任务分配到各分区. 异质图是现实世界中广泛存在的一种图, 它是指具有多种节点类型或边类型的图, 在针对异质图的计算过程中, 现有的图划分算法对于异质图的处理没有考虑到以下问题: 在图计算过程中, 不同类型的节点和边携带的数据量可能不同; 不同的节点和边类型, 可能会采用不同的处理算法, 其计算时间也会不同. 针对现有图划分方法的不足, 本文提出一种面向异质图的在线图划分算法OGP-HG算法, 并对现有的GraphX图计算引擎进行改进, 将OGP-HG算法在改进后的图计算引擎中实现. 本文提出的OGP-HG算法通过计算节点划分到不同分区上的负载均衡得分和边划分到不同分区上的数据均衡得分, 得到使异质图负载和内存占用均衡的划分结果. 实验表明, 与传统图划分算法相比, 该算法提高异质图计算效率1.05–1.4倍.     

异构集群下基于标签传播的大规模图划分算法

近年来,大规模图数据处理在众多领域得到广泛应用,图划分算法是分布式图计算系统的基础,但大规模图在异构集群中的划分尚未得到充分研究。为此,针对异构集群,提出基于标签传播的大规模图划分算法(heterogeneous label propagation, HLP),根据计算节点负载能力进行图划分,以实现负载均衡和边割率最小化为目标。HLP算法规避了传统标签传播中顶点迁移的步骤,提高了算法效率。实验结果表明,HLP算法在分区质量以及划分效率方面均有较好表现。     

基于通信负载均衡的社交网络图分割方法

海量社交网络数据中蕴含着丰富的信息,图论是挖掘这些信息的重要方法之一。面对日益增多的图数据,分布式计算成为处理大规模图数据的有效手段。在分布式图计算中,通信所消耗的时间占有很大的比例,通过图分割算法的设计可以有效地降低通信量并实现负载均衡,从而提高分布式图计算的效率,典型的例子包括Metis图分割算法。但是,用现有的图分割算法处理非均衡图数据会造成各个子图之间通信量不均衡,从而影响了计算效率。为了解决这一问题,提出一种新的图分割方法：通信均衡标签交换方法。该方法在保持子图规模一致的基础上,既降低了全图计算所需的通信量,又使各个子图之间的通信量达到均衡。实验结果表明,与Metis等典型的图分割算法相比,提出的图分割方法在各种数据集和集群配置情况下,能降低6%~30%的图计算时间,充分显示了该方法的有效性。     

一种松弛的优化均衡流式图划分算法研究

在大规模图数据的分布式处理中,往往需要将图数据进行划分并放置在不同的节点上。如果数据划分得不均衡,那么部分节点可能会成为分布式系统的瓶颈。为了提高图数据划分的均衡性,并且有效地应对图数据的快速更新,提出了一种松弛的优化均衡流式图划分算法。首先,定义了一种同时包含划分内部代价和划分之间的割的代价的目标函数作为图划分的整体框架。然后,在图划分框架的基础上通过最大化和最小化两种优化函数分析了均衡图划分问题,并给出了二者之间的关系。最后,针对流式图数据,提出一种贪婪的图最优k划分算法。该划分算法以最大化优化函数为基础,通过最大化顶点放置产生的目标函数增加值进行节点划分块的选取。实验表明,提出的图划分算法与相关算法相比,不仅均衡性好,而且通信开销小,在基于该算法进行图划分时上层应用的计算性能得到了明显的提高。     

网络化分布式凸优化算法研究进展

分布式优化是指利用网络化多自主体之间的协作来求解的一类优化问题,其在大规模数值计算、机器学习、资源分配、传感器网络等方面具有重要的研究意义和应用价值.自主体之间的协作通常基于代数图来描述,且图的结构对分布式优化算法的设计与性能有显著影响.本文针对凸优化问题,基于平衡图和非平衡图的情形,简要讨论了分布式优化算法的最新研究进展,并对今后的发展趋势和应用进行展望.     

基于邻域的大规模图数据动态分割算法

随着图数据的规模日益增大,出现大量以动态图数据为基础的分布式处理需求,划分问题在动态图数据分布式处理领域尤为重要. 对大规模动态图数据上的划分问题进行研究,根据图结构性质及动态图特点,提出并实现基于邻域的动态图分割算法. 算法分为静态切分和动态调整两个阶段,其中基于割边算法整合现有最优化策略提出了大规模图数据的静态切割算法. 在优化后的静态切割算法的基础上,根据图数据的动态扩张的特性提出动态分割算法. 根据迁移顶点所达到的最小负载值进行顶点迁移,并在此基础上进行性能及割边控制优化操作. 最后,改进算法在各类图数据集上进行了验证,验证的结果显示在平衡度和割边等指标上优化后的算法效果显著,提高了划分的合理性,并且在保证割边不增加的情况下提高了图分割的平衡度.     

基于图划分的全基因组并行拼接算法

提出了一种基于图划分的全基因组并行拼接算法．该算法巧妙地将数据划分问题转化成图划分的问题,解决了传统数据划分算法中存在的节点负载不平衡的问题．同时,算法在建立关系图时有效地利用了WGS测序中所提供reads之间的长度信息和配对信息,使reads关系图能更准确地反映出数据之间的关系特性,从而提高了数据划分的准确性．实验结果表明,该算法可以准确地划分各种模拟数据、真实数据的数据集,相对于传统数据划分算法划分质量有了明显改善．     

动态图划分算法研究综述

图划分是大规模分布式图处理的首要工作,对图应用的存储、查询、处理和挖掘起基础支撑作用.随着图数据规模的不断扩大,真实世界中的图表现出动态性.如何对动态图进行划分,已成为目前图划分研究的热点问题.从不同动态图划分算法的关注点和特点出发,系统性地介绍当前可用于解决动态图划分问题的各类算法,包括流式图划分算法、增量式图划分算法和图重划分算法.首先介绍图划分的3种不同的划分策略及问题定义、图的两种不同的动态性来源以及动态图划分问题;然后介绍3种不同的流式图划分算法,包括基于Hash的划分算法、基于邻居分布的划分算法以及基于流的优化划分算法;其次介绍单元素增量式划分和批量增量式划分这两种不同的增量式图划分算法;再次,分别介绍针对图结构动态的重划分算法和针对图计算动态的重划分算法;最后,在对已有方法分析和比较的基础上,总结目前动态图划分面临的主要挑战,提出相应的研究问题.     

基于GraphX的分布式幂迭代聚类

为解决幂迭代聚类算法并行实现中存在的编程繁琐、效率低下等问题,基于Spark大规模数据通用计算引擎及其GraphX组件,提出了一种在分布式环境下实现幂迭代聚类的方法。首先,利用某种相似性度量方法,将原始数据转换成一个可以视为图的亲和矩阵;然后,通过顶点切割,把行归一化后的亲和矩阵切分成若干个小图,分别存储在不同的机器上;最后,利用Spark基于内存计算的特点,对存储在集群中的图进行多次迭代计算,得到这个图的一个切割,图的每一个划分子图对应一个类簇。在不同规模的数据集和不同executor个数下进行的实验结果表明,基于GraphX的分布式幂迭代聚类算法具有良好的可扩展性,算法运行时间与executor个数呈负相关的线性关系,在6个executor下,与单个executor相比,算法的加速比达到了2.09到3.77。同时,通过与基于Hadoop的幂迭代聚类进行对比,在新闻数量为40000篇时,运行时间降低了61%。     

异构计算环境中图划分算法的研究

复杂网络的研究已经广泛地应用到生物、计算机等各个学科领域.如今,网络规模十分巨大,如何对这些大规模图数据进行有效率的挖掘计算,是研究复杂网络的首要任务.并行计算技术是现在最成熟、应用最广、最可行的计算加速技术之一.而图划分技术是提高并行计算性能的有效手段.图划分问题的研究是随着实际应用的需求而驱动.针对异构计算环境下的分布式集群,本文提出了一种异构感知的流式图划分算法.该方法既考虑到集群中网络带宽及节点计算能力的不同,同时又考虑到了以InfiniBand为代表的高速网络环境下核之间的共享资源的竞争.实验以图算法BFS、SSSP和PageRank为例,相对于未考虑异构环境的流算法,图计算效率分别平均提高了38％、45.7％、61.8％.同时针对流式图划分过程中邻点缓存查找效率低下问题,本文又设计了一种邻边结构的缓存查找算法,在相同条件下,图划分的效率平均提高了 13.4％.仿真实验结果表明,本文设计的异构感知图划分算法实现了异构集群环境下图计算效率的提升.     

基于图神经网络的多维度池化图分类

目前很多处理图数据的图神经网络方法被提出,然而大多数研究侧重于对特征聚合的卷积层的研究而不是进行下采样的池化层.此外,形成聚类簇的池化方式需要额外计算分配矩阵;节点得分的池化方式排名方式单一.为解决上述问题,提高图分类任务的准确性,本文提出了一种新的基于多维度信息的图池化算子MDPool.该模型使用节点特征信息以及图拓扑结构信息,获取不同维度下的节点得分.使用注意力机制归纳不同维度下的得分权重,生成更为健壮的节点排名,基于节点排名自适应选择节点集合生成诱导子图.提出的MDPool可以集成到多种的图神经网络结构,将MDPool池化算子与图神经网络卷积层堆叠形成编码解码模型EDMDPool.在4个公开数据集的图分类任务中, EDMDPool均高于现有基线模型.     

Graph Embedding Based API Graph Search and Recommendation

Searching application programming interfaces (APIs) is very important for developers to reuse software projects. Existing natural language based API search mainly faces the following challenges. 1) More accurate results are required as software projects evolve to be more heterogeneous and complex. 2) The semantic relationships between APIs (e.g., inheritances between classes, and invocations between methods) need to be illustrated so that developers can better understand their usage scenarios. To deal with these issues, we propose GeAPI, a novel graph embedding based approach for API graph search and recommendation in this paper. First, we build a software project's API graph automatically from its source code and represent each API using graph embedding methods. Second, we search the API graph with a question in natural language, and return the corresponding subgraph that is composed of relevant code elements and their associated relationships, as the best answer of the question. In experiments, we select three well-known open source projects, JodaTime, Apache Lucene and POI, as examples to perform API search tasks. The experimental results show that our approach GeAPI improves F1-score by 10% compared with the existing shortest path based API search approach, while reduces the average response time about 60 times.     

面向模式图变化的增量图模式匹配

在大数据时代,数据图的规模急剧增长,增量图模式匹配算法能够在数据图或模式图发生变化时避免重新在整个数据图上进行匹配、减少响应时间,因此成为了研究的热点.针对实际应用中数据图不变而模式图发生变化的情况,提出了一种面向模式图变化的增量图模式匹配算法PGC_IncGPM,在模式图匹配的过程中记录适当的中间结果作为索引,用于后续的模式匹配.提出了增强的图模式匹配算法GPMS,用于首次整个数据图上的模式匹配.该算法一方面能够建立后续增量匹配所需的索引,另一方面减少了整个数据图匹配的执行时间.设计实现了面向模式图增边和减边的两个核心子算法,通过子算法的组合,能够支持在模式图发生各种变化时进行增量图模式匹配.在真实数据集和合成数据集上进行实验,结果表明:与重新在整个数据图上进行匹配的ReComputing算法相比,当模式图中变化的边的数目不超过不变的边的数目时,PGC_IncGPM算法能够有效减少图模式匹配的执行时间;随着数据图规模的增大,PGC_IncGPM算法相对于ReComputing算法的执行时间的减少程度更加明显,对于大规模数据图具有更好的适用性.     

Graph based Ranked Answers for Keyword Graph Structure

Keyword query processing over graph structured data is beneficial across various real world applications. The basic unit, of search and retrieval, in keyword search over graph, is a structure (interconnection of nodes) that connects all the query keywords. This new answering paradigm, in contrast to single web page results given by search engines, brings forth new challenges for ranking. In this paper, we propose a simple but effective Fuzzy set theory based Ranking measure, called FRank. Fuzzy sets acknowledge the contribution of each individual query keyword, discretely, to enumerate node relevance. A novel aggregation operator is defined, to combine the content relevance based fuzzy sets and, compute query dependent edge weights. The final rank, of an answer, is computed by non-monotonic addition of edge weights, as per their relevance to keyword query. FRank evaluates each answer based on the distribution of query keywords and structural connectivity between those keywords. An extensive empirical analysis shows superior performance by our proposed ranking measure as compared to the ranking measures adopted by current approaches in the literature.     

Graph Farming

The Giant Global Graph is an excellent perspective on how we can consider diverse pieces of Web infrastructure as a conceptual whole. The Web, when augmented with RDF's graph model, provides an interconnected system. The resources on the semantic Web are interconnected through logical predicates, but this layer exists on top of the coupling offered by the link, as built into the Web since day one. Resources can be identified with URIs designed for HTTP, and the primary Web mechanism is that of using HTTP to get representations of those resources.     

The Perception of Graph Properties in Graph Layouts

When looking at drawings of graphs, questions about graph density, community structures, local clustering and other graph properties may be of critical importance for analysis. While graph layout algorithms have focused on minimizing edge crossing, symmetry, and other such layout properties, there is not much known about how these algorithms relate to a user's ability to perceive graph properties for a given graph layout. In this study, we apply previously established methodologies for perceptual analysis to identify which graph drawing layout will help the user best perceive a particular graph property. We conduct a large scale (n = 588) crowdsourced experiment to investigate whether the perception of two graph properties (graph density and average local clustering coefficient) can be modeled using Weber's law. We study three graph layout algorithms from three representative classes (Force Directed ‐ FD, Circular, and Multi‐Dimensional Scaling ‐ MDS), and the results of this experiment establish the precision of judgment for these graph layouts and properties. Our findings demonstrate that the perception of graph density can be modeled with Weber's law. Furthermore, the perception of the average clustering coefficient can be modeled as an inverse of Weber's law, and the MDS layout showed a significantly different precision of judgment than the FD layout.     

Package Graph

<正>~~     

Graph complexity

Summary We develop a complexity theory based on the concept of the graph instead of the Boolean function. We show its relation to the Boolean complexity and prove some lower bounds to the complexity of explicitly given graphs.The paper was written while the first author was visiting Department of Mathematics, Statistics and Computer Science, University of Illinois at Chicago     

Graph clustering

In this survey we overview the definitions and methods for graph clustering, that is, finding sets of “related” vertices in graphs. We review the many definitions for what is a cluster in a graph and measures of cluster quality. Then we present global algorithms for producing a clustering for the entire vertex set of an input graph, after which we discuss the task of identifying a cluster for a specific seed vertex by local computation. Some ideas on the application areas of graph clustering algorithms are given. We also address the problematics of evaluating clusterings and benchmarking cluster algorithms.     

Graph quantization

Vector quantization (VQ) is a lossy data compression technique from signal processing, which is restricted to feature vectors and therefore inapplicable for combinatorial structures. This contribution aims at extending VQ to the quantization of graphs in a theoretically principled way in order to overcome practical limitations known in the context of prototype-based clustering of graphs. For this, we present the following results: (i) A proof of the necessary Lloyd-Max conditions for optimality of a graph quantizer, (ii) consistency statements for optimal graph quantizer design, and (iii) an accelerated version of competitive learning graph quantization. In order to achieve the proposed results, we present graphs as points in some orbifold. The orbifold framework will introduce sufficient mathematical structure to allow an extension of VQ to graph quantization in a theoretically sound way without discarding the relational information of the graphs. In doing so the proposed approach provides a template of how to link structural pattern recognition methods other than graph quantization to statistical pattern recognition.