Graph embedding and extensions: a general framework for dimensionality reduction

A large family of algorithms - supervised or unsupervised; stemming from statistics or geometry theory - has been designed to provide different solutions to the problem of dimensionality reduction. Despite the different motivations of these algorithms, we present in this paper a general formulation known as graph embedding to unify them within a common framework. In graph embedding, each algorithm can be considered as the direct graph embedding or its linear/kernel/tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set, with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided. Furthermore, the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms. By utilizing this framework as a tool, we propose a new supervised dimensionality reduction algorithm called marginal Fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class, while the penalty graph connects the marginal points and characterizes the interclass separability. We show that MFA effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions. Real face recognition experiments show the superiority of our proposed MFA in comparison to LDA, also for corresponding kernel and tensor extensions     

基于判别正则化的局部保留投影

局部保留投影（Locality preserving projections,LPP）是一种常用的线性化流形学习方法,其通过线性嵌入来保留基于图所描述的流形数据本质结构特征,因此LPP对图的依赖性强,且在嵌入过程中缺少对图描述的进一步分析和挖掘。当图对数据本质结构特征描述不恰当时,LPP在嵌入过程中不易实现流形数据本质结构的有效提取。为了解决这个问题,本文在给定流形数据图描述的条件下,通过引入局部相似度阈值进行局部判别分析,并据此建立判别正则化局部保留投影（简称DRLPP）。该方法能够在现有图描述的条件下,有效突出不同流形结构在线性嵌入空间中的可分性。在人造合成数据集和实际标准数据集上对DRLPP以及相关算法进行对比实验,实验结果证明了DRLPP的有效性。     

Kernel-view based discriminant approach for embedded feature extraction in high-dimensional space

Derived from the traditional manifold learning algorithms, local discriminant analysis methods identify the underlying submanifold structures while employing discriminative information for dimensionality reduction. Mathematically, they can all be unified into a graph embedding framework with different construction criteria. However, such learning algorithms are limited by the curse-of-dimensionality if the original data lie on the high-dimensional manifold. Different from the existing algorithms, we consider the discriminant embedding as a kernel analysis approach in the sample space, and a kernel-view based discriminant method is proposed for the embedded feature extraction, where both PCA pre-processing and the pruning of data can be avoided. Extensive experiments on the high-dimensional data sets show the robustness and outstanding performance of our proposed method.     

A novel kernel-based framework for facial-image hallucination

In this paper, we present a kernel-based eigentransformation framework to hallucinate the high-resolution (HR) facial image of a low-resolution (LR) input. The eigentransformation method is a linear subspace approach, which represents an image as a linear combination of training samples. Consequently, those novel facial appearances not included in the training samples cannot be super-resolved properly. To solve this problem, we devise a kernel-based extension of the eigentransformation method, which takes higher-order statistics of the image data into account. To generate HR face images with higher fidelity, the HR face image reconstructed using this kernel-based eigentransformation method is treated as an initial estimation of the target HR face. The corresponding high-frequency components of this estimation are extracted to form a prior in the maximum a posteriori (MAP) formulation of the SR problem so as to derive the final reconstruction result. We have evaluated our proposed method using different kernels and configurations, and have compared these performances with some current SR algorithms. Experimental results show that our kernel-based framework, along with a proper kernel, can produce good HR facial images in terms of both visual quality and reconstruction errors.     

图嵌入算法的分布式优化与实现

随着人工智能时代的到来,图嵌入技术被越来越多地用来挖掘图中的信息.然而,现实生活中的图通常很大,因此,分布式图嵌入技术得到了广泛的关注.分布式图嵌入算法面临着两大难点:(1)图嵌入算法多种多样,没有一个通用的框架能够描述大部分的算法;(2)现在的分布式图嵌入算法扩展性不足,当处理大图时性能较低.针对以上两个挑战,首先提...     

Visual instance mining from the graph perspective

In this paper, we address the problem of visual instance mining, which is to automatically discover frequently appearing visual instances from a large collection of images. We propose a scalable mining method by leveraging the graph structure with images as vertices. Different from most existing approaches that focus on either instance-level similarities or image-level context properties, our method captures both information. In the proposed framework, the instance-level information is integrated during the construction of a sparse instance graph based on the similarity between augmented local features, while the image-level context is explored with a greedy breadth-first search algorithm to discover clusters of visual instances from the graph. This framework can tackle the challenges brought by small visual instances, diverse intra-class variations, as well as noise in large-scale image databases. To further improve the robustness, we integrate two techniques into the basic framework. First, to better cope with the increasing noise of large databases, weak geometric consistency is adopted to efficiently combine the geometric information of local matches into the construction of the instance graph. Second, we propose the layout embedding algorithm, which leverages the algorithm originally designed for graph visualization to fully explore the image database structure. The proposed method was evaluated on four annotated data sets with different characteristics, and experimental results showed the superiority over state-of-the-art algorithms on all data sets. We also applied our framework on a one-million Flickr data set and proved its scalability.     

Commute time guided transformation for feature extraction

This paper presents a random-walk-based feature extraction method called commute time guided transformation (CTG) in the graph embedding framework. The paper contributes to the corresponding field in two aspects. First, it introduces the usage of a robust probability metric, i.e., the commute time (CT), to extract visual features for face recognition via a manifold way. Second, the paper designs the CTG optimization to find linear orthogonal projections that would implicitly preserve the commute time of high dimensional data in a low dimensional subspace. Compared with previous CT embedding algorithms, the proposed CTG is a graph-independent method. Existing CT embedding methods are graph-dependent that could only embed the data on the training graph in the subspace. Differently, CTG paradigm can be used to project the out-of-sample data into the same embedding space as the training graph. Moreover, CTG projections are robust to the graph topology that it can always achieve good recognition performance in spite of different initial graph structures. Owing to these positive properties, when applied to face recognition, the proposed CTG method outperforms other state-of-the-art algorithms on benchmark datasets. Specifically, it is much efficient and effective to recognize faces with noise.     

基于改进多嵌入空间的实时语义数据流推理

将语义数据流处理引擎与知识图谱嵌入表示学习相结合,可以有效提高实时数据流推理查询性能,但是现有的知识表示学习模型更多关注静态知识图谱嵌入,忽略了知识图谱的动态特性,导致难以应用于实时动态语义数据流推理任务。为了使知识表示学习模型适应知识图谱的在线更新并能够应用于语义数据流引擎,建立一种基于改进多嵌入空间的动态知识图谱嵌入模型PUKALE。针对传递闭包等复杂推理场景,提出3种嵌入空间生成算法。为了在进行增量更新时更合理地选择嵌入空间,设计2种嵌入空间选择算法。基于上述算法实现PUKALE模型,并将其嵌入数据流推理引擎CSPARQL-engine中,以实现实时语义数据流推理查询。实验结果表明,与传统的CSPARQL和KALE推理相比,PUKALE模型的推理查询时间分别约降低85%和93%,其在支持动态图谱嵌入的同时能够提升实时语义数据流推理准确率。     

具有统计不相关性的核化图嵌入算法

提出统计不相关的核化图嵌入算法,为求解各种统计不相关的核化降维算法提供了一种统一方法。与已有核化降维算法相比,新的特征提取方法降低甚至消除了最佳鉴别矢量间的统计相关性,提高了识别率。通过在ORL,YALE和FERET人脸库上的实验结果表明,提出的具有统计不相关的核化图嵌入算法在识别率方面好于已有的核算法。另外,揭示了统计不相关的核化图嵌入与已有的核化图嵌入的内在关系。     

基于数据关联感知的无监督深度融合指针网络模型

为了提高组合优化问题可行解集合的收敛性和泛化性,根据不同无监督学习策略的特点,提出一种基于数据关联感知的深度融合指针网络模型(DMAG-PN),模型通过指针网络框架将Mogrifier LSTM、多头注意力机制与图卷积神经网络三者融合.首先,编码器模块中的嵌入层对输入序列进行编码,引入多头注意力机制获取编码矩阵中的特征信息;然后构建数据关联模型探索序列节点间的关联性,采用图卷积神经网络获取其多维度关联特征信息并融合互补,旨在生成多个嵌入有效捕捉序列深层的节点特征和边缘特征;最后,基于多头注意力机制的解码器模块以节点嵌入数据和融合图嵌入数据作为输入,生成选择下一个未访问节点的全局概率分布.采用对称旅行商问题作为测试问题,与当前先进算法进行对比,实验结果表明,所提出DMAG-PN模型在泛化性和求解精确性方面获得较大的改进与提高,预训练好的DMAG-PN模型能够直接对大规模实例进行端到端的求解,避免传统算法迭代搜索的过程,具有较高的求解效率.     

基于低秩稀疏图的结构保持投影算法

在图嵌入理论框架下,能够较好地揭示数据本质特性的图在一些维数约简方法中起到关键性的作用。基于稀疏表示和低秩表示方法,构建了一种低秩稀疏图,能够同时揭示数据的局部结构信息和全局结构信息。然后,利用图嵌入理论方法使这些特性在线性投影的过程中得以保持不变,从而学习出高维数据有效的低维嵌入。在标准的人脸和手写数字数据集(ORL,Yale,PIE,MNIST)上进行实验,同传统的图嵌入方法比较,结果表明了算法的有效性。     

Toward online node classification on streaming networks

The proliferation of networked data in various disciplines motivates a surge of research interests on network or graph mining. Among them, node classification is a typical learning task that focuses on exploiting the node interactions to infer the missing labels of unlabeled nodes in the network. A vast majority of existing node classification algorithms overwhelmingly focus on static networks and they assume the whole network structure is readily available before performing learning algorithms. However, it is not the case in many real-world scenarios where new nodes and new links are continuously being added in the network. Considering the streaming nature of networks, we study how to perform online node classification on this kind of streaming networks (a.k.a. online learning on streaming networks). As the existence of noisy links may negatively affect the node classification performance, we first present an online network embedding algorithm to alleviate this problem by obtaining the embedding representation of new nodes on the fly. Then we feed the learned embedding representation into a novel online soft margin kernel learning algorithm to predict the node labels in a sequential manner. Theoretical analysis is presented to show the superiority of the proposed framework of online learning on streaming networks (OLSN). Extensive experiments on real-world networks further demonstrate the effectiveness and efficiency of the proposed OLSN framework.     

基于节点相似度的无监督属性图嵌入模型

属性图嵌入旨在将属性图中的节点表示为低维向量,并同时保留节点的拓扑信息和属性信息.属性图嵌入已经有一系列相关工作,然而它们大多数提出的是有监督或半监督的算法.在实际应用中,需要标记的节点数量多,导致这些属性图嵌入算法的难度大,且需要消耗巨大的人力物力.针对上述问题以无监督的视角重新分析,提出了一种无监督的属性图嵌入算法...     

Local uncorrelated local discriminant embedding for face recognition

The feature extraction algorithm plays an important role in face recognition. However, the extracted features also have overlapping discriminant information. A property of the statistical uncorrelated criterion is that it eliminates the redundancy among the extracted discriminant features, while many algorithms generally ignore this property. In this paper, we introduce a novel feature extraction method called local uncorrelated local discriminant embedding (LULDE). The proposed approach can be seen as an extension of a local discriminant embedding (LDE) framework in three ways. First, a new local statistical uncorrelated criterion is proposed, which effectively captures the local information of interclass and intraclass. Second, we reconstruct the affinity matrices of an intrinsic graph and a penalty graph, which are mentioned in LDE to enhance the discriminant property. Finally, it overcomes the small-sample-size problem without using principal component analysis to preprocess the original data, which avoids losing some discriminant information. Experimental results on Yale, ORL, Extended Yale B, and FERET databases demonstrate that LULDE outperforms LDE and other representative uncorrelated feature extraction methods.     

基于核化图嵌入的最佳鉴别分析与人脸识别

将压缩映射和同构映射引入核化图嵌入框架(kernel extension of graph embedding,简称KGE),从理论上证明了KGE框架内的各种核算法其实质是KPCA(kernel principal component analysis)+LGE(linear extension of graph embedding,简称LGE)框架内的线性降维算法,并且基于所给出的理论框架提出了一种综合利用零空间和非零空间鉴别信息的组合方法.任何一种可以用核化图嵌入框架描述的核算法,都可以有相应的组合方法.在ORL,Yale,FERET和PIE人脸数据库上验证了所提出的理论和方法的有效性.     

Kernel-based discriminative elastic embedding algorithm

A nonlinear version of discriminative elastic embedding (DEE) algorithm is presented, called kernel discriminative elastic embedding (KDEE). In this paper, we concretely fulfill the following works: (1) class labels and linear projection matrix are integrated into the kernel-based objective function; (2) two different strategies are adopted for optimizing the objective function of KDEE, and accordingly the final algorithms are termed as KDEE1 and KDEE2 respectively; (3) a deliberately selected Laplacian search direction is adopted in KDEE1 for faster convergence. Experimental results on several publicly available databases demonstrate that the proposed algorithm achieves powerful pattern revealing capability for complex manifold data.     

基于Beta分布和半监督学习的非确定性知识图谱嵌入模型

近年来,面向确定性知识图谱的嵌入模型在知识图谱补全等任务中取得了长足的进展,但如何设计和训练面向非确定性知识图谱的嵌入模型仍然是一个重要挑战。不同于确定性知识图谱,非确定性知识图谱的每个事实三元组都有着对应的置信度,因此,非确定性知识图谱嵌入模型需要准确地计算出每个三元组的置信度。现有的非确定性知识图谱嵌入模型结构较为简单,只能处理对称关系,并且无法很好地处理假负(false-negative)样本问题。为了解决上述问题,该文首先提出了一个用于训练非确定性知识图谱嵌入模型的统一框架,该框架使用基于多模型的半监督学习方法训练非确定性知识图谱嵌入模型。为了解决半监督学习中半监督样本噪声过高的问题,我们还使用蒙特卡洛Dropout计算出模型对输出结果的不确定度,并根据该不确定度有效地过滤了半监督样本中的噪声数据。此外,为了更好地表示非确定性知识图谱中实体和关系的不确定性以处理更复杂的关系,该文还提出了基于Beta分布的非确定性知识图谱嵌入模型UBetaE,该模型将实体、关系均表示为一组相互独立的Beta分布。在公开数据集上的实验结果表明,结合该文所提出的半监督学习方法和UBetaE模型,不仅...     

提升图嵌入框架及在表情识别中的应用 *

提出了一种提升图嵌入框架用于特征提取和选择 ,以及一种新的近邻权重计算方法 ,称为分类图。传统图嵌入模型的近邻权重采用欧氏距离 ,不能被提升算法所更新 ;相比较 ,分类图采用的是提升算法中样本的权重,反映的是样本在分类过程中的重要程度 ,有效地提高了图嵌入模型的分类性能。在通用人脸表情库上的识别实验结果验证了提升图嵌入模型的有效性。     

条件变分时序图自编码器

网络表示学习(也被称为图嵌入)是链接预测、节点分类、社区发现、图可视化等图任务的基础.现有大多数的图嵌入算法主要是针对静态图开发的,难以捕捉现实世界的网络随时间进化的动态特征.目前,针对动态网络表示学习方法的研究工作仍相对不足.提出了条件变分时序图自编码器(TS-CVGAE),可以同时学习动态网络的局部结构和随时间的演化模式.该方法首先改进了传统图卷积得到时序图卷积,并在条件变分自编码器的框架下使用时序图卷积对网络节点进行编码.训练结束后,条件变分自编码器的中间层就是最终的网络嵌入结果.实验结果表明,该方法在4个现实动态网络数据集上的链接预测表现均优于相关的静、动态网络表示学习方法.