一种基于多视立体视觉的多视图直线匹配方法

提出了一种基于多视立体视觉（Multiple view stereo,MVS）进行多视图直线匹配的方法. 本文方法首先利用MVS所得到的三维点云及其可见性信息,建立三维点与图像直线的对应关系. 根据此对应关系,为每条图像直线建立由一个三维点集和一个三维单位向量构成的描述子,用以衡量图像直线之间的相似性及一致性. 之后,本文方法以所有图像直线为顶点建立一个图,并引入了图谱分析来获取统一的顶点距离度量. 最后,本方法对DBSCAN聚类算法进行了修改,并用修改后的算法从图谱分析结果中获取可靠的直线匹配. 实验显示,本方法比已有方法更加鲁棒,并且有更高的准确率.     

基于权重学习的图像最大权对集胚配模型

在图匹配模型中权重的设置对匹配性能有很大影响,但直接计算的权重往往不符合匹配图像的实际情况。为此,参照二次分配问题的图匹配学习思想,给出一阶和二阶最大权对集模型的权重学习计算方法。一阶最大权对集模型直接采用图像特征点作为图的顶点,而二阶最大权对集模型则采用某些特征点之间的连接边作为顶点,2个模型都可以通过Kuhn—Munkras算法求解。一阶最大权对集模型在本质上等价于二次分配问题的线性情况。在CMUHouse数据库上的图像匹配实验结果表明,二阶最大权对集模型优于一阶最大权对集模型,且两者在学习计算时的性能也优于直接计算的情况。     

A sparse nonnegative matrix factorization technique for graph matching problems

Graph matching problem that incorporates pairwise constraints can be cast as an Integer Quadratic Programming (IQP). Since it is NP-hard, approximate methods are required. In this paper, a new approximate method based on nonnegative matrix factorization with sparse constraints is presented. Firstly, the graph matching is formulated as an optimization problem with nonnegative and sparse constraints, followed by an efficient algorithm to solve this constrained problem. Then, we show the strong relationship between the sparsity of the relaxation solution and its effectiveness for graph matching based on our model. A key benefit of our method is that the solution is sparse and thus can approximately impose the one-to-one mapping constraints in the optimization process naturally. Therefore, our method can approximate the original IQP problem more closely than other approximate methods. Extensive and comparative experimental results on both synthetic and real-world data demonstrate the effectiveness of our graph matching method.     

A graph-based optimization algorithm for fragmented image reassembly

We propose a graph-based optimization framework for automatic 2D image fragment reassembly. First, we compute the potential matching between each pair of the image fragments based on their geometry and color. After that, a novel multi-piece matching algorithm is proposed to reassemble the overall image fragments. Finally, the reassembly result is refined by applying the graph optimization algorithm. We perform experiments to evaluate our algorithm on multiple torn real-world images, and demonstrate the robustness of this new assembly framework outperforms the existing algorithms in both reassembly accuracy (in handling accumulated pairwise matching error) and robustness (in handling small image fragments).     

基于多维社交关系嵌入的深层图神经网络推荐方法

社会化推荐系统通过用户的社会属性信息能缓解推荐系统中数据稀疏性和冷启动问题，从而提高推荐系统的精度。然而大多数社会化推荐方法主要针对单一的社交网络，或对多个社交网络进行线性叠加，使得用户社会属性难以充分参与计算，因而推荐的精度有限。针对该问题，提出一种多重网络嵌入的图形神经网络模型来实现复杂多维社交网络下的推荐，该模型构建了统一的方法来融合用户-物品、用户-用户等各种关系构成的多维复杂网络，通过注意力机制聚合不同类型的多邻居对节点生成作出贡献，并将多个图神经网络进行组合，从而构建了多维社交关系下的图神经网络推荐框架。这种方法通过拓扑结构直接反映推荐系统中实体及其相互间关系，直接在图上对相关信息进行不断更新计算，具有很强的归纳性，有效避免了传统推荐方法中信息利用不完全的问题。通过与相关的社会推荐算法进行比较，实验结果表明，所提方法在均方根误差（RMSE）和平均绝对误差（MAE）等推荐精度指标上有所改善，甚至在数据稀疏情况下也有良好的精度。     

基于多维社交关系嵌入的深层图神经网络推荐方法

社会化推荐系统通过用户的社会属性信息能缓解推荐系统中数据稀疏性和冷启动问题,从而提高推荐系统的精度。然而大多数社会化推荐方法主要针对单一的社交网络,或对多个社交网络进行线性叠加,使得用户社会属性难以充分参与计算,因而推荐的精度有限。针对该问题,提出一种多重网络嵌入的图形神经网络模型来实现复杂多维社交网络下的推荐,该模型构建了统一的方法来融合用户-物品、用户-用户等各种关系构成的多维复杂网络,通过注意力机制聚合不同类型的多邻居对节点生成作出贡献,并将多个图神经网络进行组合,从而构建了多维社交关系下的图神经网络推荐框架。这种方法通过拓扑结构直接反映推荐系统中实体及其相互间关系,直接在图上对相关信息进行不断更新计算,具有很强的归纳性,有效避免了传统推荐方法中信息利用不完全的问题。通过与相关的社会推荐算法进行比较,实验结果表明,所提方法在均方根误差(RMSE)和平均绝对误差(MAE)等推荐精度指标上有所改善,甚至在数据稀疏情况下也有良好的精度。     

View-Based 3D Model Retrieval via Multi-graph Matching

With the rapid development of 3D technology, 3D model retrieval has attracted a large amount of interest in computer vision field. In this paper, we propose a composition-based multi-graph matching method in this paper. Firstly, compute the pairwise matching affinity one-to-one graph matching. Secondly, seek the optimal intermediate graph by diverse graph matching orders, according to the consistency of global matching. Finally, the classic optimization method is used to get the best matching result for similarity measurement. We validate our approach using ETH, NTU and MV-RED 3D model datasets with convolutional neural network features. Extensive experiments show the superiority of the proposed method.     

计算机视觉中图匹配研究进展: 从二图匹配迈向多图匹配

图匹配试图求解二图或多图之间节点的对应关系.在图像图形领域,图匹配是一个历久弥新的基础性问题.从优化的角度来看,图匹配问题是一个组合优化问题,且在一般情形下具有非确定性多项式复杂程度(non-deter-ministic polynomial, NP)难度的性质.在过去数十年间,出现了大量求解二图匹配的近似算法,并在各个领域得到了较为广泛的应用.然而,受限于优化问题本身的理论困难和实际应用中数据质量的种种限制,各二图匹配算法在匹配精度上的性能日益趋近饱和.相比之下,由于引入了更多信息且往往更符合实际问题的设定,多图的协同匹配则逐渐成为了一个新兴且重要的研究方向.本文首先介绍了经典的二图匹配方法,随后着重介绍近年来多图匹配方法的最新进展和相关工作.最后,本文讨论了图匹配未来的发展.     

基于知识图谱的人岗推荐系统构建

为解决传统人岗推荐系统存在的三个常见问题,即数据稀疏性、数据冷启动和数据利用率低,提出了基于知识图谱的人岗推荐系统构建方法。该方法通过改进传统推荐模型,将知识图谱作为辅助边信息融合到推荐系统中进行人岗推荐,有效解决了数据稀疏性和数据冷启动问题;引入知识图谱补全算法提高了数据利用率。提出的方法在人岗推荐上准确率可达92%,比现有人岗推荐方法准确率提高约1%。实验结果表明该方法是可行的,知识图谱的加入可以提升人岗推荐系统的推荐效果。     

A Path Following Algorithm for the Graph Matching Problem

We propose a convex-concave programming approach for the labeled weighted graph matching problem. The convex-concave programming formulation is obtained by rewriting the weighted graph matching problem as a least-square problem on the set of permutation matrices and relaxing it to two different optimization problems: a quadratic convex and a quadratic concave optimization problem on the set of doubly stochastic matrices. The concave relaxation has the same global minimum as the initial graph matching problem, but the search for its global minimum is also a hard combinatorial problem. We, therefore, construct an approximation of the concave problem solution by following a solution path of a convex-concave problem obtained by linear interpolation of the convex and concave formulations, starting from the convex relaxation. This method allows to easily integrate the information on graph label similarities into the optimization problem, and therefore, perform labeled weighted graph matching. The algorithm is compared with some of the best performing graph matching methods on four data sets: simulated graphs, QAPLib, retina vessel images, and handwritten Chinese characters. In all cases, the results are competitive with the state of the art.     

Robust Higher Order Potentials for Enforcing Label Consistency

This paper proposes a novel framework for labelling problems which is able to combine multiple segmentations in a principled manner. Our method is based on higher order conditional random fields and uses potentials defined on sets of pixels (image segments) generated using unsupervised segmentation algorithms. These potentials enforce label consistency in image regions and can be seen as a generalization of the commonly used pairwise contrast sensitive smoothness potentials. The higher order potential functions used in our framework take the form of the Robust P ⁿ model and are more general than the P ⁿ Potts model recently proposed by Kohli et al. We prove that the optimal swap and expansion moves for energy functions composed of these potentials can be computed by solving a st-mincut problem. This enables the use of powerful graph cut based move making algorithms for performing inference in the framework. We test our method on the problem of multi-class object segmentation by augmenting the conventional crf used for object segmentation with higher order potentials defined on image regions. Experiments on challenging data sets show that integration of higher order potentials quantitatively and qualitatively improves results leading to much better definition of object boundaries. We believe that this method can be used to yield similar improvements for many other labelling problems.     

多源异构数据的实体匹配方法研究

近年来,针对多源异构数据的实体匹配问题,已经有诸多学者提出不同的解决方法。然而,这些方法几乎都集中在RDFS或OWL等语义框架下进行实体匹配,不具有通用性。此外,针对多数据源实体匹配问题,目前主流解决方式是将其转换为多组两两数据源的实体匹配问题,该种方式直接进行两两匹配的计算复杂度过高,且没有从多数据源全局的角度分析问题。从这些问题出发,提出了一种的实体匹配方法,利用了实体中普遍存在的名称、属性和上下文信息,构建多种索引,缩减计算空间同时生成高质量的候选集;还定义了度量实体相似度的计算方法,有效地判别了实体对是否匹配。并根据实体间边的权重以及互斥关系,提出一种基于图划分的优化算法,划分多个等价实体构成的集合。从互联网中抓取商业领域下品牌和人物类别的真实数据进行实验测试,实验结果表明该方法取得了良好的效果。     

A new graph matching method for point-set correspondence using the EM algorithm and Softassign

Finding correspondences between two point-sets is a common step in many vision applications (e.g., image matching or shape retrieval). We present a graph matching method to solve the point-set correspondence problem, which is posed as one of mixture modelling. Our mixture model encompasses a model of structural coherence and a model of affine-invariant geometrical errors. Instead of absolute positions, the geometrical positions are represented as relative positions of the points with respect to each other. We derive the Expectation–Maximization algorithm for our mixture model. In this way, the graph matching problem is approximated, in a principled way, as a succession of assignment problems which are solved using Softassign. Unlike other approaches, we use a true continuous underlying correspondence variable. We develop effective mechanisms to detect outliers. This is a useful technique for improving results in the presence of clutter. We evaluate the ability of our method to locate proper matches as well as to recognize object categories in a series of registration and recognition experiments. Our method compares favourably to other graph matching methods as well as to point-set registration methods and outlier rejectors.     

Recognition of occluded objects by reducing feature interactions

The main difficulty for the recognition of occluded objects lies in the fact that the original feature set is corrupted and no longer reliable to represent the object of interest. This corruption is caused by the interactions between features from different objects, denoted as feature interactions, which is a key issue addressed in our algorithm. In this paper, a local to global strategy is represented for the occlusion recognition problem, which combines the pairwise grouping and graph matching algorithms. Local appearance similarity serves as priors to reduce feature interactions, by which the performance of graph matching algorithms is improved in order to deal with the contaminated data set. With our formulation, a global decision on object recognition can be made based on locally gathered information. Experimental results show that the proposed framework can dramatically reduce incorrect matches and objects under severe occlusions can still be recognized.     

Graph-based deformable matching of 3D line with application in protein fitting

We present an algorithm for matching two sets of line segments in 3D that have undergone non-rigid deformations. This problem is motivated by a biology application that seeks a correspondence between the alpha-helices from two proteins, so that matching helices have similar lengths and these can be aligned by some low-distortion deformation. While matching between two feature sets have been extensively studied, particularly for point features, matching line segments has received little attention so far. As typical in point-matching methods, we formulate a graph matching problem and solve it using continuous relaxation. We make two technical contributions. First, we propose a graph construction for undirected line segments such that the optimal matching between two graphs represents an as-rigid-as-possible deformation between the two sets of segments. Second, we propose a novel heuristic for discretizing the continuous solution in graph matching. Our heuristic can be applied to matching problems (such as ours) that are not amenable to certain heuristics, and it produces better solutions than those applicable heuristics. Our method is compared with a state-of-art method motivated by the same biological application and demonstrates improved accuracy.     

Subgraph similarity maximal all-matching over a large uncertain graph

Recently, uncertain graph data management and mining techniques have attracted significant interests and research efforts due to potential applications such as protein interaction networks and social networks. Specifically, as a fundamental problem, subgraph similarity all-matching is widely applied in exploratory data analysis. The purpose of subgraph similarity all-matching is to find all the similarity occurrences of the query graph in a large data graph. Numerous algorithms and pruning methods have been developed for the subgraph matching problem over a certain graph. However, insufficient efforts are devoted to subgraph similarity all-matching over an uncertain data graph, which is quite challenging due to high computation costs. In this paper, we define the problem of subgraph similarity maximal all-matching over a large uncertain data graph and propose a framework to solve this problem. To further improve the efficiency, several speed-up techniques are proposed such as the partial graph evaluation, the vertex pruning, the calculation model transformation, the incremental evaluation method and the probability upper bound filtering. Finally, comprehensive experiments are conducted on real graph data to test the performance of our framework and optimization methods. The results verify that our solutions can outperform the basic approach by orders of magnitudes in efficiency.     

Hierarchical multi-agent planning for flexible assembly of large-scale lunar facilities

Several national space agencies and commercial aerospace companies plan to set up lunar bases with large-scale facilities that rely on multiple lunar robots’ assembly. Mission planning is necessary to achieve efficient multi-robot cooperation. This paper aims at autonomous multi-robot planning for the flexible assembly of the large-scale lunar facility, considering the harsh lunar environment, mission time optimization, and joint actions. The lunar robots and modules are scattered around the mission area without fixed assembly lines. Thus, the traditional assembly planning methods ignoring the optimal selection of modules are unable to handle this problem. We propose a hierarchical multi-agent planning method based on two-stage two-sided matching (HMAP-TTM) to solve this critical problem. First, the distributed planning framework with multi-replica public agents is introduced, ensuring robot plan knowledge consistency through public agents’ communication. Second, the hierarchical task graph (HTG) divides the mission into task layers based on task dependency knowledge. Third, we develop a novel two-stage two-sided matching algorithm. Time-optimal plans emerge from the matching games among public and private agents in each layer of HTG. Agents make decisions in the game based on action knowledge updated during planning. Finally, an assembly mission is presented to prove the method’s effectiveness. The simulation results show that the HMAP-TTM can generate plans with shorter mission time and require smaller communication costs than the baseline methods.     

基于概率图模型的点集匹配方法研究

在概率图模型框架下提出了一种将回归分析和聚类分析相结合的贝叶斯点集匹配方法,其中,回归分析用来估计两个点集之间的映射函数,而聚类分析用来建立两个点集中点与点之间的对应关系.本文将点集匹配问题表示为一种多层的概率有向图,并提出了一种由粗到精的变分逼近算法来估计点集匹配的不确定性;此外,还利用高斯混合模型估计映射函数回归中的异方差噪声和场景点密度估计中离群点的分布;同时,引入转移变量建立起模型点集与场景点集之间的关系,并与离群点混合模型共同对场景点的分布进行估计.实验结果表明,该方法与其他点集匹配算法相比,在鲁棒性和匹配精度方面均达到了较好的效果.     

基于多视角图编码的选择式阅读理解方法

选择式阅读理解通常采用证据抽取和答案预测的两阶段流水线框架,答案预测的效果非常依赖于证据句抽取的效果.传统的证据抽取多依赖词段匹配或利用噪声标签监督证据抽取的方法,准确率不理想,这极大地影响了答案预测的性能.针对该问题,提出一种联合学习框架下基于多视角图编码的选择式阅读理解方法,从多视角充分挖掘文档句子之间以及文档句子和问句之间的关联关系,实现证据句及其关系的有效建模;同时通过联合训练证据抽取和答案预测任务,利用证据和答案之间强关联关系提升证据抽取与答案预测的性能.具体来说,所提方法首先基于多视角图编码模块对文档、问题和候选答案联合编码,从统计特性、相对距离和深度语义3个视角捕捉文档、问题和候选答案之间的关系,获得问答对感知的文档编码特征;然后,构建证据抽取和答案预测的联合学习模块,通过协同训练强化证据与答案之间的关系,证据抽取子模块实现证据句的选择,并将其结果和文档编码特征进行选择性融合,并用于答案预测子模块完成答案预测.在选择式阅读理解数据集ReCO和RACE上的实验结果表明,所提方法提升了从文档中选择证据句子的能力,进而提高答案预测的准确率.同时,证据抽取与答案预测联合学习很大程...