An approach to facial expression recognition integrating radial basis function kernel and multidimensional scaling analysis

To better deal with high dimensions and extract the essential feature of facial expression images in facial expression recognition task, a novel approach integrating radial basis function kernel and multidimensional scaling analysis is proposed in this paper. Firstly, the radial basis function kernel is invoked to map facial expression images to the Hilbert space. Then, Hilbert distance is substituted for the Euclidean distance and a neighbor graph is constructed to express the relationship between data points by employing k nearest neighbor method. Finally, we apply the modified MDS algorithm to reduce the dimension and extract features of facial expression images. Experiments results on the JAFFE database show that this proposed algorithm performs better than Isomap algorithm and supervised Isomap algorithm, and it is more feasible and effective.     

Learning eigenfunctions links spectral embedding and kernel PCA

In this letter, we show a direct relation between spectral embedding methods and kernel principal components analysis and how both are special cases of a more general learning problem: learning the principal eigenfunctions of an operator defined from a kernel and the unknown data-generating density. Whereas spectral embedding methods provided only coordinates for the training points, the analysis justifies a simple extension to out-of-sample examples (the Nystr?m formula) for multidimensional scaling (MDS), spectral clustering, Laplacian eigenmaps, locally linear embedding (LLE), and Isomap. The analysis provides, for all such spectral embedding methods, the definition of a loss function, whose empirical average is minimized by the traditional algorithms. The asymptotic expected value of that loss defines a generalization performance and clarifies what these algorithms are trying to learn. Experiments with LLE, Isomap, spectral clustering, and MDS show that this out-of-sample embedding formula generalizes well, with a level of error comparable to the effect of small perturbations of the training set on the embedding.     

An effective double-bounded tree-connected Isomap algorithm for microarray data classification

Isometric mapping (Isomap) is a popular nonlinear dimensionality reduction technique which has shown high potential in visualization and classification. However, it appears sensitive to noise or scarcity of observations. This inadequacy may hinder its application for the classification of microarray data, in which the expression levels of thousands of genes in a few normal and tumor sample tissues are measured. In this paper we propose a double-bounded tree-connected variant of Isomap, aimed at being more robust to noise and outliers when used for classification and also computationally more efficient. It differs from the original Isomap in the way the neighborhood graph is generated: in the first stage we apply a double-bounding rule that confines the search to at most k nearest neighbors contained within an ε-radius hypersphere; the resulting subgraphs are then joined by computing a minimum spanning tree among the connected components. We therefore achieve a connected graph without unnaturally inflating the values of k and ε. The computational experiences show that the new method performs significantly better in terms of accuracy with respect to Isomap, k-edge-connected Isomap and the direct application of support vector machines to data in the input space, consistently across seven microarray datasets considered in our tests.     

Nonlinear Dimensionality Reduction and Data Visualization： A Review

Dimensionality reduction and data visualization are useful and important processes in pattern recognition.Many techniques have been developed in the recent years.The self-organizing map (SOM) can be an efficient method for this purpose.This paper reviews recent advances in this area and related approaches such as multidimensional scaling (MDS),nonlinear PCA,principal manifolds,as well as the connections of the SOM and its recent variant,the visualization induced SOM (ViSOM),with these approaches. The SOM is shown to produce a quantized,qualitative scaling and while the ViSOM a quantitative or metric scaling and approximates principal curve/surface.The SOM can also be regarded as a generalized MDS to relate two metric spaces by forming a topological mapping between them.The relationships among various recently proposed techniques such as ViSOM,Isomap,LLE,and eigenmap are discussed and compared.     

基于空间句法和最短路径的图核

针对图模式识别领域中现有图核方法对反映图本身拓扑结构的节点特征挖掘不够充分的问题,提出了基于空间句法和最短路径的图核。借鉴建筑学与城市规划学科中的空间句法理论构造分布于图节点上的拓扑特征的量化描述,基于此提出了可表示、计算,正定、适用范围较广的空间句法核和基于最短路径的空间句法核,进而借助支持向量机实现了非精确图匹配。不同于其他图核方法,该方法对图的拓扑特征表达能力强,通用性较好。实验结果表明,所设计的图核在分类精度方面相较于最短路径核有较显著的改善。     

A Generalized Kernel Consensus-Based Robust Estimator

In this paper, we present a new Adaptive-Scale Kernel Consensus (ASKC) robust estimator as a generalization of the popular and state-of-the-art robust estimators such as RANdom SAmple Consensus (RANSAC), Adaptive Scale Sample Consensus (ASSC), and Maximum Kernel Density Estimator (MKDE). The ASKC framework is grounded on and unifies these robust estimators using nonparametric kernel density estimation theory. In particular, we show that each of these methods is a special case of ASKC using a specific kernel. Like these methods, ASKC can tolerate more than 50 percent outliers, but it can also automatically estimate the scale of inliers. We apply ASKC to two important areas in computer vision, robust motion estimation and pose estimation, and show comparative results on both synthetic and real data.     

Robust Performance Controller Synthesis Based on Discrete‐Time Noncausal Linear Periodically Time‐Varying Scaling

This paper is concerned with the technique called discrete‐time noncausal linear periodically time‐varying (LPTV) scaling for robust stability analysis and synthesis. It is defined through the lifting treatment of discrete‐time systems, and naturally leads to a sort of noncausal operation of signals. In the robust stability analysis of linear time‐invariant (LTI) systems, it has been shown that even static noncausal LPTV scaling induces some frequency‐dependent scaling when it is interpreted in the context of lifting‐free treatment. This paper first discusses in detail different aspects of the effectiveness of noncausal LPTV scaling, with the aim of showing its effectiveness in controller synthesis. More precisely, we study the robust performance controller synthesis problem, where we allow the controllers to be LPTV. As in the LTI robust performance controller synthesis problem, we tackle our problem with an iterative method without guaranteed convergence to a globally optimal controller. Despite such a design procedure, the closed‐loop H_∞ performance is expected to improve as the period of the controller is increased, and we discuss how the frequency‐domain properties of noncausal LPTV scaling could contribute to such improvement. We demonstrate with a numerical example that an effective LPTV controller can be designed for a class of uncertainties for which the well‐known μ‐synthesis fails to derive even a robust stabilization controller.     

Categorical structural optimization using discrete manifold learning approach and custom-built evolutionary operators

In present work, we address the non-ordinal categorical design variables, such as different beam/bar cross-section types, various materials or components available within a catalog. We interpret the admissible values of categorical variables as discrete points in multi-dimensional space of physical attributes, which allows computing distances but has no ordering property. Then we propose to use the Isomap manifold learning approach to eliminate the possibly redundant dimensionality and obtain a reduced-order design space in which the geodesic distances are preserved in a low-dimensional graph. Then, taking advantage of the shortest path and the neighbors provided by Dijkstra algorithm, we propose graph-based crossover and mutation operators to be used in evolutionary optimization. The method is applied to the optimal design of truss and frame structures.     

Robust stability analysis with cycling-based LPTV scaling. Part II: its properties under the use of FIR separators

This paper is concerned with robust stability analysis of discrete-time linear periodically time-varying (LPTV) systems using the cycling-based LPTV scaling approach. It consists of applying the separator-type robust stability theorem through the cycling-based treatment of such systems, where this paper aims at revealing fundamental properties of this approach when we confine ourselves to what we call finite impulse response (FIR) separators as a theoretically and numerically very tractable class of separators. Specifically, we clarify such properties of the cycling-based LPTV scaling approach using FIR separators that cannot readily be seen under the treatment of general class of separators. This is accomplished by comparing it with another approach, called lifting-based LPTV scaling using FIR separators, through the framework of representing the associated robust stability conditions with infinite matrices. More precisely, this leads us to clarifying the fundamental relationships between the cycling-based and lifting-based approaches under the use of FIR separators. We also provide a numerical example demonstrating the fundamental relationships clarified in this paper.     

Graph-optimized locality preserving projections

Locality preserving projections (LPP) is a typical graph-based dimensionality reduction (DR) method, and has been successfully applied in many practical problems such as face recognition. However, LPP depends mainly on its underlying neighborhood graph whose construction suffers from the following issues: (1) such neighborhood graph is artificially defined in advance, and thus does not necessary benefit subsequent DR task; (2) such graph is constructed using the nearest neighbor criterion which tends to work poorly due to the high-dimensionality of original space; (3) it is generally uneasy to assign appropriate values for the neighborhood size and heat kernel parameter involved in graph construction. To address these problems, we develop a novel DR algorithm called Graph-optimized Locality Preserving Projections (GoLPP). The idea is to integrate graph construction with specific DR process into a unified framework, which results in an optimized graph rather than predefined one. Moreover, an entropy regularization term is incorporated into the objective function for controlling the uniformity level of the edge weights in graph, so that a principled graph updating formula naturally corresponding to conventional heat kernel weights can be obtained. Finally, the experiments on several publicly available UCI and face data sets show the feasibility and effectiveness of the proposed method with encouraging results.     

Combining Isomap ordination and imaging spectroscopy to map continuous floristic gradients in a heterogeneous landscape

Mapping requires a meaningful generalization of information. For vegetation maps, classification is frequently used to generalize the species composition of (semi-)natural plant assemblages. As an alternative to classification, ordination methods aim to extract major floristic gradients describing the prevailing compositional variation in a floristic data set as metric variables. This ability has been used previously to derive gradient maps of homogeneous landscapes that show plant species composition in continuous fields. In the present study, gradient mapping was used in a more heterogeneous landscape with intricate environmental gradients and higher variation in vegetation physiognomy. Since established ordination methods may have difficulties to cope with the highly variable plant species composition, we tested the novel method Isometric Feature Mapping (Isomap) against conventional methods (Detrended Correspondence Analysis and Nonmetric Multidimensional Scaling). The resulting floristic gradients were related to hyperspectral imagery (HyMap) using partial least squares regression (PLSR) and subsequently mapped. Prediction uncertainties are provided as additional map. Isomap was able to preserve 74% of the original variation inherent to the floristic data set in a three-dimensional solution. This was considerably more than the established techniques achieved. The PLSR models for the floristic gradients extracted with Isomap showed model fits ranging from R² = 0.59 to R² = 0.73 in calibration and from R² = 0.55 to R² = 0.69 in tenfold cross-validation. The resulting gradient map provides detailed information on compositional vegetation patterns.     

Robust Positive semidefinite L-Isomap Ensemble

In this paper, we derive an ensemble method inspired by boosting, a novel Robust Positive semidefinite L-Isomap Ensemble (RPL-IsomapE) approach. Specifically, we first apply a constant-shifting method to yield a symmetric positive semidefinite (SPSD) matrix. For topological stability, we also employ a method for eliminating critical outlier points using the confusion rate of all the data points. Then we align individual Robust Positive semidefinite L-Isomap (RPL-Isomap) solutions in common coordinate system through high dimensional affine transformations. Finally, we combine multiple RPL-Isomap solutions by the weighted averaging procedure according to residual variance to improve the noise-robustness of our method. Our RPL-IsomapE maintains the scalability and the speed of L-Isomap. Experiments on two images data sets and a video data set confirm the promising performance of the proposed RPL-IsomapE.     

A Minimal Line Property Preserving Representation of Line Images

In line image understanding a minimal line property preserving (MLPP) graph of the image compliments the structural information in geometric graph representations like the run graph. With such a graph and its dual it is possible to efficiently detect topological features like loops and holes and to make use of relations like containment. We present a new rule based method on dual graph contraction for transforming the run graph and its dual into MLPP graphs. A parallel O(log(longest curve)) algorithm is presented and results given. Received: May 28, 1998; revised November 17, 1998     

Heat Walk: Robust Salient Segmentation of Non‐rigid Shapes

Segmenting three dimensional objects using properties of heat diffusion on meshes aim to produce salient results. The few existing algorithms based on heat diffusion do not use the full knowledge that can be gained from heat diffusion and are sensitive to varying kinds of perturbations. Our simple algorithm, Heat Walk, converts the implicit information in the heat kernel to explicit knowledge about the pathways for maximum heat flow capacity. We develop a two stage strategy for segmentation. In the first stage we quickly identify regions which are dominated by heat accumulators by employing a greedy algorithm. The second stage partitions out dissipative regions from the previously discovered accumulative regions by using a KL‐divergence based criterion. The resulting algorithm is both independent of human intervention and fast because of the globally aware directed walk along the maximal heat flow capacity. Extensive experimental evidence shows the method is robust to a variety of noise factors including topological short circuits, surface holes, pose variations, variations in tessellation, missing features, scaling, as well as normal and shot noise. Comparison with the Princeton Segmentation Benchmark (PSB) shows that our method is comparable with state of the art segmentation methods and has additional advantages of being robust and self contained. Based upon theoretical insight the convergence and stability of the Heat Walk is shown.     

Kernel ridge regression for out-of-sample mapping in supervised manifold learning

Manifold learning methods for unsupervised nonlinear dimensionality reduction have proven effective in the visualization of high dimensional data sets. When dealing with classification tasks, supervised extensions of manifold learning techniques, in which class labels are used to improve the embedding of the training points, require an appropriate method for out-of-sample mapping.In this paper we propose multi-output kernel ridge regression (KRR) for out-of-sample mapping in supervised manifold learning, in place of general regression neural networks (GRNN) that have been adopted by previous studies on the subject. Specifically, we consider a supervised agglomerative variant of Isomap and compare the performance of classification methods when the out-of-sample embedding is based on KRR and GRNN, respectively. Extensive computational experiments, using support vector machines and k-nearest neighbors as base classifiers, provide statistical evidence that out-of-sample mapping based on KRR consistently dominates its GRNN counterpart, and that supervised agglomerative Isomap with KRR achieves a higher accuracy than direct classification methods on most data sets.     

<Emphasis Type="Italic">υ</Emphasis>-Support vector machine based on discriminant sparse neighborhood preserving embedding

In this paper, we mainly focus on two issues (1) SVM is very sensitive to noise. (2) The solution of SVM does not take into consideration of the intrinsic structure and the discriminant information of the data. To address these two problems, we first propose an integration model to integrate both the local manifold structure and the local discriminant information into ?₁ graph embedding. Then we add the integration model into the objection function of υ-support vector machine. Therefore, a discriminant sparse neighborhood preserving embedding υ-support vector machine (υ-DSNPESVM) method is proposed. The theoretical analysis demonstrates that υ-DSNPESVM is a reasonable maximum margin classifier and can obtain a very lower generalization error upper bound by minimizing the integration model and the upper bound of margin error. Moreover, in the nonlinear case, we construct the kernel sparse representation-based ?₁ graph for υ-DSNPESVM, which is more conducive to improve the classification accuracy than ?₁ graph constructed in the original space. Experimental results on real datasets show the effectiveness of the proposed υ-DSNPESVM method.     

Geometric characterization and clustering of graphs using heat kernel embeddings

In this paper, we investigate the use of heat kernels as a means of embedding the individual nodes of a graph in a vector space. The reason for turning to the heat kernel is that it encapsulates information concerning the distribution of path lengths and hence node affinities on the graph. The heat kernel of the graph is found by exponentiating the Laplacian eigensystem over time. In this paper, we explore how graphs can be characterized in a geometric manner using embeddings into a vector space obtained from the heat kernel. We explore two different embedding strategies. The first of these is a direct method in which the matrix of embedding co-ordinates is obtained by performing a Young–Householder decomposition on the heat kernel. The second method is indirect and involves performing a low-distortion embedding by applying multidimensional scaling to the geodesic distances between nodes. We show how the required geodesic distances can be computed using parametrix expansion of the heat kernel. Once the nodes of the graph are embedded using one of the two alternative methods, we can characterize them in a geometric manner using the distribution of the node co-ordinates. We investigate several alternative methods of characterization, including spatial moments for the embedded points, the Laplacian spectrum for the Euclidean distance matrix and scalar curvatures computed from the difference in geodesic and Euclidean distances. We experiment with the resulting algorithms on the COIL database.     

Choosing parameters of kernel subspace LDA for recognition of face images under pose and illumination variations.

This paper addresses the problem of automatically tuning multiple kernel parameters for the kernel-based linear discriminant analysis (LDA) method. The kernel approach has been proposed to solve face recognition problems under complex distribution by mapping the input space to a high-dimensional feature space. Some recognition algorithms such as the kernel principal components analysis, kernel Fisher discriminant, generalized discriminant analysis, and kernel direct LDA have been developed in the last five years. The experimental results show that the kernel-based method is a good and feasible approach to tackle the pose and illumination variations. One of the crucial factors in the kernel approach is the selection of kernel parameters, which highly affects the generalization capability and stability of the kernel-based learning methods. In view of this, we propose an eigenvalue-stability-bounded margin maximization (ESBMM) algorithm to automatically tune the multiple parameters of the Gaussian radial basis function kernel for the kernel subspace LDA (KSLDA) method, which is developed based on our previously developed subspace LDA method. The ESBMM algorithm improves the generalization capability of the kernel-based LDA method by maximizing the margin maximization criterion while maintaining the eigenvalue stability of the kernel-based LDA method. An in-depth investigation on the generalization performance on pose and illumination dimensions is performed using the YaleB and CMU PIE databases. The FERET database is also used for benchmark evaluation. Compared with the existing PCA-based and LDA-based methods, our proposed KSLDA method, with the ESBMM kernel parameter estimation algorithm, gives superior performance.     

Learning to rank on graphs

Graph representations of data are increasingly common. Such representations arise in a variety of applications, including computational biology, social network analysis, web applications, and many others. There has been much work in recent years on developing learning algorithms for such graph data; in particular, graph learning algorithms have been developed for both classification and regression on graphs. Here we consider graph learning problems in which the goal is not to predict labels of objects in a graph, but rather to rank the objects relative to one another; for example, one may want to rank genes in a biological network by relevance to a disease, or customers in a social network by their likelihood of being interested in a certain product. We develop algorithms for such problems of learning to rank on graphs. Our algorithms build on the graph regularization ideas developed in the context of other graph learning problems, and learn a ranking function in a reproducing kernel Hilbert space (RKHS) derived from the graph. This allows us to show attractive stability and generalization properties. Experiments on several graph ranking tasks in computational biology and in cheminformatics demonstrate the benefits of our framework.     

基于改进的等距离映射算法的人脸识别

针对等距离映射(Isomap)算法在处理扰动图像时拓扑结构不稳定的缺点,提出了一种改进算法。改进算法将图像欧氏距离(IMED)嵌入到等距离映射算法之中。首先引入坐标度量系数计算图像的坐标度量矩阵,通过线性变换将原始图像从欧氏距离(ED)空间转换到图像欧氏距离空间;然后计算变换空间中样本的欧氏距离矩阵,并在此基础上构建样本邻域图,得到近似测地距离矩阵;最后采用多维标度(MDS)分析算法构造样本的低维表示。对ORL和Yale人脸数据库降维并结合最近邻分类器进行实验,基于改进算法的识别率平均分别提高了5.57%和3.95%,表明与原算法相比,改进算法在人脸识别中对图像扰动具有较好的鲁棒性。