A spectral approach to learning structural variations in graphs

This paper shows how to construct a linear deformable model for graph structure by performing principal components analysis (PCA) on the vectorised adjacency matrix. We commence by using correspondence information to place the nodes of each of a set of graphs in a standard reference order. Using the correspondences order, we convert the adjacency matrices to long-vectors and compute the long-vector covariance matrix. By projecting the vectorised adjacency matrices onto the leading eigenvectors of the covariance matrix, we embed the graphs in a pattern-space. We illustrate the utility of the resulting method for shape-analysis.     

Coupling measures between modes and state variables in power-system dynamics

A new method for model reduction called Selective Modal Analysis uses the eigenvectors and reciprocal eigenvectors of the system matrix A to calculate participation factors associating modes to certain state variables. This paper investigates the idea in depth. The novel aspect of this paper is the use of MacFarlane's concept that the system matrix A represents an energy transformation map to show that the participation factors are actually modal energies. This interpretation has some advantage because of its direct link to stability. It is shown that the participation factors, or modal energies, can be taken to be coupling measures between modes and state variables. An application to a single-machine infinite-busbar system with and without controllers is given. The properties and limitations of these coupling measures are investigated.     

Spectral clustering with eigenvector selection

The task of discovering natural groupings of input patterns, or clustering, is an important aspect of machine learning and pattern analysis. In this paper, we study the widely used spectral clustering algorithm which clusters data using eigenvectors of a similarity/affinity matrix derived from a data set. In particular, we aim to solve two critical issues in spectral clustering: (1) how to automatically determine the number of clusters, and (2) how to perform effective clustering given noisy and sparse data. An analysis of the characteristics of eigenspace is carried out which shows that (a) not every eigenvectors of a data affinity matrix is informative and relevant for clustering; (b) eigenvector selection is critical because using uninformative/irrelevant eigenvectors could lead to poor clustering results; and (c) the corresponding eigenvalues cannot be used for relevant eigenvector selection given a realistic data set. Motivated by the analysis, a novel spectral clustering algorithm is proposed which differs from previous approaches in that only informative/relevant eigenvectors are employed for determining the number of clusters and performing clustering. The key element of the proposed algorithm is a simple but effective relevance learning method which measures the relevance of an eigenvector according to how well it can separate the data set into different clusters. Our algorithm was evaluated using synthetic data sets as well as real-world data sets generated from two challenging visual learning problems. The results demonstrated that our algorithm is able to estimate the cluster number correctly and reveal natural grouping of the input data/patterns even given sparse and noisy data.     

A semi-supervised regression model for mixed numerical and categorical variables

In this paper, we develop a semi-supervised regression algorithm to analyze data sets which contain both categorical and numerical attributes. This algorithm partitions the data sets into several clusters and at the same time fits a multivariate regression model to each cluster. This framework allows one to incorporate both multivariate regression models for numerical variables (supervised learning methods) and k-mode clustering algorithms for categorical variables (unsupervised learning methods). The estimates of regression models and k-mode parameters can be obtained simultaneously by minimizing a function which is the weighted sum of the least-square errors in the multivariate regression models and the dissimilarity measures among the categorical variables. Both synthetic and real data sets are presented to demonstrate the effectiveness of the proposed method.     

基于图谱分解的人脸特征描述

文章探索用图谱方法嵌入且分析人脸几何特征,以图的邻接矩阵的主要特征向量来定义邻接矩阵的特征模。对每个特征模,计算谱特征向量,包括主分量特征值,模间邻接矩阵。用局部线性嵌入方法(LLE)方法嵌入这些向量到一个模式空间。另外,用人脸特征点来表示邻接图,并以几何平均图和模式特征向量的平均图两种方法对比描述不同嵌入方法的人脸特征。实验结果表明,谱向量特征平均方法能够较好地描述人脸。     

基于矩阵谱分析的文本聚类集成算法

聚类集成技术可有效提高单聚类算法的精度和稳定性,其中的关键问题是如何根据不同的聚类成员组合为更好的聚类结果.文中引入谱聚类算法解决文本聚类集成问题,设计基于正则化拉普拉斯矩阵的谱算法(NLM-SA).该算法基于代数变换,通过求解小规模矩阵的特征值和特征向量间接获得正则化拉普拉斯矩阵的特征向量,并用于后续聚类.进一步研究谱聚类算法的关键思想,设计基于超边转移概率矩阵的谱算法(HTMSA).该算法通过求解超边的低维嵌入间接获得文本的低维嵌入,并用于后续K均值算法.在TREC和Reuters文本集上的实验结果验证NLMSA和HTMSA的有效性,它们都获得比其它基于图划分的集成算法更为优越的结果.HTMSA获得的结果比NLMSA略差,而时间和空间需求则比NLMSA低得多.     

基于局部协方差矩阵的谱聚类算法

针对传统谱聚类算法没有解决簇划分过程中,簇间交叉区域样本点对聚类效果有影响这个问题,提出一种基于局部协方差矩阵的谱聚类算法,主要介绍了一种新的计算样本之间相似度亲和矩阵的方法,即通过计算样本点之间的欧氏距离划分出小子集,计算小子集的协方差,通过设定阈值剔除交叉点,由剩下的点构造相似矩阵,对相似矩阵进行特征值分解,用经典的[k]-means算法对由特征向量组成的矩阵聚类。通过在Control等真实数据集上的实验结果表明,该算法在聚类准确率、标准互信息等指标上比较对比算法获得更优秀的效果。     

Parallel solution of partial symmetric eigenvalue problems from electronic structure calculations

The computation of selected eigenvalues and eigenvectors of a symmetric (Hermitian) matrix is an important subtask in many contexts, for example in electronic structure calculations. If a significant portion of the eigensystem is required then typically direct eigensolvers are used. The central three steps are: reduce the matrix to tridiagonal form, compute the eigenpairs of the tridiagonal matrix, and transform the eigenvectors back. To better utilize memory hierarchies, the reduction may be effected in two stages: full to banded, and banded to tridiagonal. Then the back transformation of the eigenvectors also involves two stages. For large problems, the eigensystem calculations can be the computational bottleneck, in particular with large numbers of processors. In this paper we discuss variants of the tridiagonal-to-banded back transformation, improving the parallel efficiency for large numbers of processors as well as the per-processor utilization. We also modify the divide-and-conquer algorithm for symmetric tridiagonal matrices such that it can compute a subset of the eigenpairs at reduced cost. The effectiveness of our modifications is demonstrated with numerical experiments.     

Spectral Geometry Processing with Manifold Harmonics

We present an explicit method to compute a generalization of the Fourier Transform on a mesh. It is well known that the eigenfunctions of the Laplace Beltrami operator (Manifold Harmonics) define a function basis allowing for such a transform. However, computing even just a few eigenvectors is out of reach for meshes with more than a few thousand vertices, and storing these eigenvectors is prohibitive for large meshes. To overcome these limitations, we propose a band‐by‐band spectrum computation algorithm and an out‐of‐core implementation that can compute thousands of eigenvectors for meshes with up to a million vertices. We also propose a limited‐memory filtering algorithm, that does not need to store the eigenvectors. Using this latter algorithm, specific frequency bands can be filtered, without needing to compute the entire spectrum. Finally, we demonstrate some applications of our method to interactive convolution geometry filtering. These technical achievements are supported by a solid yet simple theoretic framework based on Discrete Exterior Calculus (DEC). In particular, the issues of symmetry and discretization of the operator are considered with great care.     

用神经网络计算矩阵特征值与特征向量

该文研究用神经网格求解一般实对称矩阵的全部特征向量的问题。详细讨论了网络的平均态度合的结构并建立了平衡态集合的构造定理。通过求解简单的一维微分方程求出了网络的解析表达式。这一表达式是由对称矩阵的特征值与特征向量表达的、因而非常清晰利用解的解析表达式分析了网络的解的全局渐近行为。提出了用一些单位向量作为网络初始值计算对称矩阵的全部特征值与特征向量的具体算法。     

Shape Decomposition using Modal Analysis

We introduce a novel algorithm that decomposes a deformable shape into meaningful parts requiring only a single input pose. Using modal analysis, we are able to identify parts of the shape that tend to move rigidly. We define a deformation energy on the shape, enabling modal analysis to find the typical deformations of the shape. We then find a decomposition of the shape such that the typical deformations can be well approximated with deformation fields that are rigid in each part of the decomposition. We optimize for the best decomposition, which captures how the shape deforms. A hierarchical refinement scheme makes it possible to compute more detailed decompositions for some parts of the shape.
Although our algorithm does not require user intervention, it is possible to control the process by directly changing the deformation energy, or interactively refining the decomposition as necessary. Due to the construction of the energy function and the properties of modal analysis, the computed decompositions are robust to changes in pose as well as meshing, noise, and even imperfections such as small holes in the surface.     

基于图谱分解的人脸表情分析

文中探索用人脸的几何结构图谱方法嵌入到模式空间来分析人脸表情,以图的加权邻接矩阵主要特征向量来定义矩阵的特征模。计算谱特征向量-模间邻接矩阵。用两类模式向量在范数下的多维尺度变换方法(MDS)嵌入该向量到一个模式空间,用人脸特征点来表示人脸图,并在模式空间里描述该嵌入方法下的同一人脸的不同表情。     

Spectral clustering based on matrix perturbation theory

This paper exposes some intrinsic characteristics of the spectral clustering method by using the tools from the matrix perturbation theory. We construct a weight ma- trix of a graph and study its eigenvalues and eigenvectors. It shows that the num- ber of clusters is equal to the number of eigenvalues that are larger than 1, and the number of points in each of the clusters can be approximated by the associated eigenvalue. It also shows that the eigenvector of the weight matrix can be used directly to perform clustering; that is, the directional angle between the two-row vectors of the matrix derived from the eigenvectors is a suitable distance measure for clustering. As a result, an unsupervised spectral clustering algorithm based on weight matrix (USCAWM) is developed. The experimental results on a number of artificial and real-world data sets show the correctness of the theoretical analysis.     

A Distributed Algorithm for Content Based Indexing of Images by Projections on Ritz Primary Images

Large collections of images can be indexed by their projections on a few “primary” images. The optimal primary images are the eigenvectors of a large covariance matrix. We address the problem of computing primary images when access to the images is expensive. This is the case when the images cannot be kept locally, but must be accessed through slow communication such as the Internet, or stored in a compressed form. A distributed algorithm that computes optimal approximations to the eigenvectors (known as Ritz vectors) in one pass through the image set is proposed. When iterated, the algorithm can recover the exact eigenvectors. The widely used SVD technique for computing the primary images of a small image set is a special case of the proposed algorithm. In applications to image libraries and learning, it is necessary to compute different primary images for several sub-categories of the image set. The proposed algorithm can compute these additional primary images “offline”, without the image data. Similar computation by other algorithms is impractical even when access to the images is inexpensive.     

Robust M-ary detection filters and smoothers for continuous-time jump Markov systems

In this paper, we consider a dynamic M-ary detection problem when Markov chains are observed through a Wiener process. These systems are fully specified by a candidate set of parameters, whose elements are, a rate matrix for the Markov chain and a parameter for the observation model. Further, we suppose these parameter sets can switch according to the state of an unobserved Markov chain and thereby produce an observation process generated by time varying (jump stochastic) parameter sets. Given such an observation process and a specified collection of models, we estimate the probabilities of each model parameter set explaining the observation. By defining a new augmented state process, then applying the method of reference probability, we compute matrix-valued dynamics, whose solutions estimate joint probabilities for all combinations of candidate model parameter sets and values taken by the indirectly observed state process. These matrix-valued dynamics satisfy a stochastic integral equation with a Wiener process integrator. Using the gauge transformation techniques introduced by Clark and a pointwise matrix product, we compute robust matrix-valued dynamics for the joint probabilities on the augmented state space. In these new dynamics, the observation Wiener process appears as a parameter matrix in a linear ordinary differential equation, rather than an integrator in a stochastic integral equation. It is shown that these robust dynamics, when discretised, enjoy a deterministic upper bound which ensures nonnegative probabilities for any observation sample path. In contrast, no such upper bounds can be computed for Taylor expansion approximations, such as the Euler-Maryauana and Milstein schemes. Finally, by exploiting a duality between causal and anticausal robust detector dynamics, we develop an algorithm to compute smoothed mode probability estimates without stochastic integrations. A computer simulation demonstrating performance is included.     

谱聚类算法研究综述

聚类分析是一种常见的分析方法,谱聚类作为聚类分析的一支,因其不受样本形状约束等特点备受瞩目。为及时掌握当前谱聚类算法研究动态,通过对比分析众多谱聚类优化算法,从半监督学习、二阶段聚类算法选择、算法执行效率优化等三个角度,将谱聚类优化算法分为三类,并对每类算法的优化思想进行综述。介绍经典多路谱聚类与基本理论,并分析相似矩阵及其特征值、特征向量选取原因及影响,旨在明确特征矩阵的重要性与优化的必要性。基于算法改进策略差异,梳理并总结每类算法的改进思想、研究现状及优缺点。在UCI数据集与手写体数据集上,针对谱聚类算法与优化算法进行实验对比,并对谱聚类优化算法的未来研究方向进行展望。     

自动生成量化属性模糊集的算法

提出一种由量化属性数据自动生成模糊集及其隶属函数的算法。该算法首先用聚类方法对每个量化属性进行聚类,求得聚类中心,最后通过聚类中心构造模糊集,定义隶属函数。     

An Entropy Weighting k-Means Algorithm for Subspace Clustering of High-Dimensional Sparse Data

This paper presents a new k-means type algorithm for clustering high-dimensional objects in sub-spaces. In high-dimensional data, clusters of objects often exist in subspaces rather than in the entire space. For example, in text clustering, clusters of documents of different topics are categorized by different subsets of terms or keywords. The keywords for one cluster may not occur in the documents of other clusters. This is a data sparsity problem faced in clustering high-dimensional data. In the new algorithm, we extend the k-means clustering process to calculate a weight for each dimension in each cluster and use the weight values to identify the subsets of important dimensions that categorize different clusters. This is achieved by including the weight entropy in the objective function that is minimized in the k-means clustering process. An additional step is added to the k-means clustering process to automatically compute the weights of all dimensions in each cluster. The experiments on both synthetic and real data have shown that the new algorithm can generate better clustering results than other subspace clustering algorithms. The new algorithm is also scalable to large data sets.