一种半监督判别邻域嵌入算法

邻域保持嵌入（Neighborhood Preserving Embedding,NPE）,作为局部线性嵌入（Locally Linear Embedding,LLE）的线性化版本,由于在映射前后保持了数据的局部几何结构并得到了原始数据的子空间描述,在模式识别领域具有较强的应用价值。但作为非监督处理算法,在具体的模式分类中有一定局限性,提出一种NPE的改进算法——半监督判别邻域嵌入（SSDNE）算法,引入标记后样本点的类别信息,并在正则项中引入样本的流形结构,最大化标记样本点的类间信息和类内信息。既增加了算法的辨别能力又减少了监督算法中对样本点进行全标记的工作量。在ORL和YaleB人脸库上的实验结果表明,改进的算法较PCA、LDA、LPP以及原保持近邻判别嵌入算法的识别性能有了较明显的改善。     

Metric embedding of view-graphs

Most recent robotic systems, capable of exploring unknown environments, use topological structures (graphs) as a spatial representation. Localization can be done by deriving an estimate of the global pose from landmark information. In this case navigation is tightly coupled to metric knowledge, and hence the derived control method is mainly pose-based. Alternative to continuous metric localization, it is also possible to base localization methods on weaker constraints, e.g. the similarity between images capturing the appearance of places or landmarks. In this case navigation can be controlled by a homing algorithm. Similarity based localization can be scaled to continuous metric localization by adding additional constraints, such as alignment of depth estimates. We present a method to scale a similarity based navigation system (the view-graph-model) to continuous metric localization. Instead of changing the landmark model, we embed the graph into the three dimensional pose space. Therefore, recalibration of the path integrator is only possible at discrete locations in the environment. The navigation behavior of the robot is controlled by a homing algorithm which combines three local navigation capabilities, obstacle avoidance, path integration, and scene based homing. This homing scheme allows automated adaptation to the environment. It is further used to compensate for path integration errors, and therefore allows to derive globally consistent pose estimates based on “weak” metric knowledge, i.e. knowledge solely derived from odometry. The system performance is tested with a robotic setup and with a simulated agent which explores a large, open, and cluttered environment. This work is part of the GNOSYS (FP6-003835-GNOSYS) project, supported by the European Commission.     

Two-dimensional complete neighborhood preserving embedding

Complete neighborhood preserving embedding (CNPE) is an improvement to the neighborhood preserving embedding (NPE) algorithm, which can address the singularity and stability problems of NPE and at the same time preserve useful discriminative information. However, CNPE works with vectorized representations of data, and thus, the original 2D face image matrices should be previously transformed into the same dimensional vectors. Such a matrix-to-vector transform usually leads to a high-dimensional image vector space, which makes the eigenanalysis quite difficult and time-consuming. Beyond computational issues, some spatial structural information between nearby pixels may be lost after vectorization. In this paper, we develop a new scheme for image feature extraction, namely, two-dimensional complete neighborhood preserving embedding (2D-CNPE). 2D-CNPE builds the eigenmatrix and the weight matrix which characterize local neighborhood properties of data directly based on the original face images, and then, the optimal embedding axes are obtained by performing an eigen-decomposition. Experimental results on three face databases show that the proposed 2D-CNPE achieves better performance than other feature extraction methods, such as Eigenfaces, Fisherfaces, and 2D-PCA.     

Discriminant neighborhood embedding for classification

In this paper a novel subspace learning method called discriminant neighborhood embedding (DNE) is proposed for pattern classification. We suppose that multi-class data points in high-dimensional space tend to move due to local intra-class attraction or inter-class repulsion and the optimal embedding from the point of view of classification is discovered consequently. After being embedded into a low-dimensional subspace, data points in the same class form compact submanifod whereas the gaps between submanifolds corresponding to different classes become wider than before. Experiments on the UMIST and MNIST databases demonstrate the effectiveness of our method.     

High-dimensional image data feature extraction by double discriminant embedding

We propose a supervised feature extraction method in this paper that uses two successive transformations to produce the extracted features. The first projection maximizes the difference between spectral features. Thus, produced features have minimum overlap in the new feature space. The second projection maximizes the discrimination between classes. The proposed method, which is called double discriminant embedding (DDE), uses just the first statistics of data. Thus, DDE has good efficiency using limited training samples. The experimental results on four popular hyperspectral images show the better efficiency of DDE in comparison with LDA, GDA, NWFE, and supervised LPP methods in small sample size situation.     

Distance metric learning by knowledge embedding

This paper presents an algorithm which learns a distance metric from a data set by knowledge embedding and uses the new distance metric to solve nonlinear pattern recognition problems such a clustering.     

Graph embedding discriminant analysis for face recognition

This paper develops a supervised discriminant technique, called graph embedding discriminant analysis (GEDA), for dimensionality reduction of high-dimensional data in small sample size problems. GEDA can be seen as a linear approximation of a multimanifold-based learning framework in which nonlocal property is taken into account besides the marginal property and local property. GEDA seeks to find a set of perfect projections that not only can impact the samples of intraclass and maximize the margin of interclass, but also can maximize the nonlocal scatter at the same time. This characteristic makes GEDA more intuitive and more powerful than linear discriminant analysis (LDA) and marginal fisher analysis (MFA). The proposed method is applied to face recognition and is examined on the Yale, ORL and AR face image databases. The experimental results show that GEDA consistently outperforms LDA and MFA when the training sample size per class is small.     

Metric propositional neighborhood logics on natural numbers

Interval logics formalize temporal reasoning on interval structures over linearly (or partially) ordered domains, where time intervals are the primitive ontological entities and truth of formulae is defined relative to time intervals, rather than time points. In this paper, we introduce and study Metric Propositional Neighborhood Logic (MPNL) over natural numbers. MPNL features two modalities referring, respectively, to an interval that is “met by” the current one and to an interval that “meets” the current one, plus an infinite set of length constraints, regarded as atomic propositions, to constrain the length of intervals. We argue that MPNL can be successfully used in different areas of computer science to combine qualitative and quantitative interval temporal reasoning, thus providing a viable alternative to well-established logical frameworks such as Duration Calculus. We show that MPNL is decidable in double exponential time and expressively complete with respect to a well-defined sub-fragment of the two-variable fragment ${{\rm FO}^2[\mathbb{N},=,<,s]}$ of first-order logic for linear orders with successor function, interpreted over natural numbers. Moreover, we show that MPNL can be extended in a natural way to cover full ${{\rm FO}^2[\mathbb{N},=,<,s]}$ , but, unexpectedly, the latter (and hence the former) turns out to be undecidable.     

Complete neighborhood preserving embedding for face recognition

Neighborhood preserving embedding (NPE) is a linear approximation to the locally linear embedding algorithm which can preserve the local neighborhood structure on the data manifold. However, in typical face recognition where the number of data samples is smaller than the dimension of data space, it is difficult to directly apply NPE to high dimensional matrices because of computational complexity. Moreover, in such case, NPE often suffers from the singularity problem of eigenmatrix, which makes the direct implementation of the NPE algorithm almost impossible. In practice, principal component analysis or singular value decomposition is applied as a preprocessing step to attack these problems. Nevertheless, this strategy may discard dimensions that contain important discriminative information and the eigensystem computation of NPE could be unstable. Towards a practical dimensionality reduction method for face data, we develop a new scheme in this paper, namely, the complete neighborhood preserving embedding (CNPE). CNPE transforms the singular generalized eigensystem computation of NPE into two eigenvalue decomposition problems. Moreover, a feasible and effective procedure is proposed to alleviate the computational burden of high dimensional matrix for typical face image data. Experimental results on the ORL face database and the Yale face database show that the proposed CNPE algorithm achieves better performance than other feature extraction methods, such as Eigenfaces, Fisherfaces and NPE, etc.     

Process monitoring via enhanced neighborhood preserving embedding

A novel process monitoring scheme named enhanced neighborhood preserving embedding (ENPE) is proposed. Neighborhood preserving embedding (NPE) only considers the reconstruction error on the basis of each local neighborhood is linear. For the purpose of addressing both the reconstruction error and the distance, the dual weight matrix and the enhanced objective function are constructed in the ENPE method. Finally, under a numerical example and the Tennessee Eastman (TE) benchmark, the superiority of the proposed ENPE method is evaluated through comparing with principal component analysis (PCA) and NPE.     

Model-based automatic neighborhood design by unsupervised learning

The definition of a suitable neighborhood structure on the solution space is a key step when designing a heuristic for Mixed Integer Programming (MIP). In this paper, we move on from a MIP compact formulation and show how to take advantage of its features to automatically design efficient neighborhoods, without any human analysis. In particular, we use unsupervised learning to automatically identify “good” regions of the search space “around” a given feasible solution. Computational results on compact formulations of three well-known combinatorial optimization problems show that, on large instances, the neighborhoods constructed by our procedure outperform state-of-the-art domain-independent neighborhoods.     

Linear discriminant projection embedding based on patches alignment

Dimensionality reduction is often required as a preliminary stage in many data analysis applications. In this paper, we propose a novel supervised dimensionality reduction method, called linear discriminant projection embedding (LDPE), for pattern recognition. LDPE first chooses a set of overlapping patches which cover all data points using a minimum set cover algorithm with geodesic distance constraint. Then, principal component analysis (PCA) is applied on each patch to obtain the data's local representations. Finally, patches alignment technique combined with modified maximum margin criterion (MMC) is used to yield the discriminant global embedding. LDPE takes both label information and structure of manifold into account, thus it can maximize the dissimilarities between different classes and preserve data's intrinsic structures simultaneously. The efficiency of the proposed algorithm is demonstrated by extensive experiments using three standard face databases (ORL, YALE and CMU PIE). Experimental results show that LDPE outperforms other classical and state of art algorithms.     

Incremental manifold learning by spectral embedding methods

Recent years have witnessed great success of manifold learning methods in understanding the structure of multidimensional patterns. However, most of these methods operate in a batch mode and cannot be effectively applied when data are collected sequentially. In this paper, we propose a general incremental learning framework, capable of dealing with one or more new samples each time, for the so-called spectral embedding methods. In the proposed framework, the incremental dimensionality reduction problem reduces to an incremental eigen-problem of matrices. Furthermore, we present, using this framework as a tool, an incremental version of Hessian eigenmaps, the IHLLE method. Finally, we show several experimental results on both synthetic and real world datasets, demonstrating the efficiency and accuracy of the proposed algorithm.     

Local uncorrelated local discriminant embedding for face recognition

The feature extraction algorithm plays an important role in face recognition. However, the extracted features also have overlapping discriminant information. A property of the statistical uncorrelated criterion is that it eliminates the redundancy among the extracted discriminant features, while many algorithms generally ignore this property. In this paper, we introduce a novel feature extraction method called local uncorrelated local discriminant embedding (LULDE). The proposed approach can be seen as an extension of a local discriminant embedding (LDE) framework in three ways. First, a new local statistical uncorrelated criterion is proposed, which effectively captures the local information of interclass and intraclass. Second, we reconstruct the affinity matrices of an intrinsic graph and a penalty graph, which are mentioned in LDE to enhance the discriminant property. Finally, it overcomes the small-sample-size problem without using principal component analysis to preprocess the original data, which avoids losing some discriminant information. Experimental results on Yale, ORL, Extended Yale B, and FERET databases demonstrate that LULDE outperforms LDE and other representative uncorrelated feature extraction methods.     

Regularized locality preserving discriminant embedding for face recognition

For face recognition, graph embedding techniques attempt to produce a high data locality projection for better recognition performance. However, estimation of population data locality could be severely biased due to small number of training samples. The biased estimation triggers overfitting problem and hence poor generalization. In this paper, we propose a new linear graph embedding technique based upon an adaptive locality preserving regulation model (ALPRM), known as Regularized Locality Preserving Discriminant Embedding (RLPDE). In RLPDE, the projection features are regulated based on ALPRM to approach population data locality, which can directly enhance the locality preserving capability of the projection features. This paper also presents the relation between locality preserving capability and class discrimination. Specifically, we show that the optimization of the locality preserving function minimizes the within-class variability. Experiments on three face datasets such as PIE, FRGC and FERET show the promising performance of the proposed technique.     

Orthogonal neighborhood preserving discriminant analysis for face recognition

In this paper, we propose a new linear subspace analysis algorithm, called orthogonal neighborhood preserving discriminant analysis (ONPDA). Given a set of data points in the ambient space, a weight matrix is firstly built which describes the relationship between the data points. Then optimal between-class scatter matrix and within-class scatter matrix are defined such that the neighborhood structure can be preserved. In order to improve the discriminating power, a new method is presented for orthogonalizing the basis eigenvectors. We evaluate the performance of the proposed algorithm for face recognition with the use of different databases. Consistent and promising results demonstrate the effectiveness of our algorithm.     

Building k-connected neighborhood graphs for isometric data embedding

Isometric data embedding using geodesic distance requires the construction of a connected neighborhood graph so that the geodesic distance between every pair of data points can be estimated. This paper proposes an approach for constructing k-connected neighborhood graphs. The approach works by applying a greedy algorithm to add each edge, in a nondecreasing order of edge length, to a neighborhood graph if end vertices of the edge are not yet k-connected on the graph. The k-connectedness between vertices is tested using a network flow technique by assigning every vertex a unit flow capacity. This approach is applicable to a wide range of data. Experiments show that it gives better estimation of geodesic distances than other approaches, especially when the data are undersampled or nonuniformly distributed.     

Discriminant sparse neighborhood preserving embedding for face recognition

Sparse subspace learning has drawn more and more attentions recently. However, most of the sparse subspace learning methods are unsupervised and unsuitable for classification tasks. In this paper, a new sparse subspace learning algorithm called discriminant sparse neighborhood preserving embedding (DSNPE) is proposed by adding the discriminant information into sparse neighborhood preserving embedding (SNPE). DSNPE not only preserves the sparse reconstructive relationship of SNPE, but also sufficiently utilizes the global discriminant structures from the following two aspects: (1) maximum margin criterion (MMC) is added into the objective function of DSNPE; (2) only the training samples with the same label as the current sample are used to compute the sparse reconstructive relationship. Extensive experiments on three face image datasets (Yale, Extended Yale B and AR) demonstrate the effectiveness of the proposed DSNPE method.