Relative manifold based semi-supervised dimensionality reduction

A well-designed graph plays a fundamental role in graph-based semi-supervised learning; however, the topological structure of a constructed neighborhood is unstable in most current approaches, since they are very sensitive to the high dimensional, sparse and noisy data. This generally leads to dramatic performance degradation. To deal with this issue, we developed a relative manifold based semisupervised dimensionality reduction （RMSSDR） approach by utilizing the relative manifold to construct a better neighborhood graph with fewer short-circuit edges. Based on the relative cognitive law and manifold distance, a relative transformation is used to construct the relative space and the relative manifold. A relative transformation can improve the ability to distinguish between data points and reduce the impact of noise such that it may be more intuitive, and the relative manifold can more truly reflect the manifold structure since data sets commonly exist in a nonlinear structure. Specifically, RMSSDR makes full use of pairwise constraints that can define the edge weights of the neighborhood graph by minimizing the local reconstruction error and can preserve the global and local geometric structures of the data set. The experimental results on face data sets demonstrate that RMSSDR is better than the current state of the art comparing methods in both performance of classification and robustness.     

Relative manifold based semi-supervised dimensionality reduction

Figure 8 of this article shows YaleB and CMU PIE with incorrect legend titles:YaleB(Tr=1900,Te=514,NOC=100)should be YaleB(Tr=1900,Te=514,d=100)(Fig.8(a));TIE(Tr=1200,Te=2880,d=100)should be PIE(Tr=1200,Te=2880,d=100)(Fig.8(b)).In Fig.9,the legend keys and the legend texts are mismatched.The correct figure is ilustrated as follows.     

Mixture graph based semi-supervised dimensionality reduction

Graph structure is crucial to graph based dimensionality reduction. A mixture graph based semi-supervised dimensionality reduction (MGSSDR) method with pairwise constraints is proposed. MGSSDR first constructs multiple diverse graphs on different random subspaces of dataset, then it combines these graphs into a mixture graph and does dimensionality reduction on this mixture graph. MGSSDR can preserve the pairwise constraints and local structure of samples in the reduced subspace. Meanwhile, it is robust to noise and neighborhood size. Experimental results on facial images feature extraction demonstrate its effectiveness.     

基于权值的局部保持半监督降维算法*

针对现有的半监督降维算法没有考虑存在于数据集中的大量未标记信息,不能得到最好的降维效果的问题。本文提出了一种改进的基于权值的局部保持半监督降维算法。该算法在保持正、负约束信息的同时,还利用距离权值来保持数据集所在的局部结构,从而提高降维效果。在UCI数据集上的实验表明,该算法能够提高降维的效果,尤其是在数据分布特性不满足流形结构时,仍能得到较好的聚类结果。     

加权成对约束半监督局部维数约减算法

考虑到已有的半监督维数约减方法在利用边信息时将所有边信息等同,不能充分挖掘边所含信息,提出加权成对约束半监督局部维数约减算法(WSLDR).通过构建近邻图对边信息进行扩充,使边信息数量有所增加.另外,根据边所含信息量的不同构建边的权系数矩阵.将边信息融入近邻图对其进行修正,对修正后的近邻图和加权的成对约束寻找最优投影.算法不仅保持了数据的内在局部几何结构,而且使得类内数据分布更加紧密,类间数据分布更加分散.在UCI数据集上的实验结果验证了该算法的有效性.     

Effective multiplicative updates for non-negative discriminative learning in multimodal dimensionality reduction

Fisher discriminant analysis gives the unsatisfactory results if points in the same class have within-class multimodality and fails to produce the non-negativity of projection vectors. In this paper, we focus on the newly formulated within and between-class scatters based supervised locality preserving dimensionality reduction problem and propose an effective dimensionality reduction algorithm, namely, Multiplicative Updates based non-negative Discriminative Learning (MUNDL), which optimally seeks to obtain two non-negative embedding transformations with high preservation and discrimination powers for two data sets in different classes such that nearby sample pairs in the original space compact in the learned embedding space, under which the projections of the original data in different classes can be appropriately separated from each other. We also show that MUNDL can be easily extended to nonlinear dimensionality reduction scenarios by employing the standard kernel trick. We verify the feasibility and effectiveness of MUNDL by conducting extensive data visualization and classification experiments. Numerical results on some benchmark UCI and real-world datasets show the MUNDL method tends to capture the intrinsic local and multimodal structure characteristics of the given data and outperforms some established dimensionality reduction methods, while being much more efficient.     

Multiple view semi-supervised dimensionality reduction

Multiple view data, together with some domain knowledge in the form of pairwise constraints, arise in various data mining applications. How to learn a hidden consensus pattern in the low dimensional space is a challenging problem. In this paper, we propose a new method for multiple view semi-supervised dimensionality reduction. The pairwise constraints are used to derive embedding in each view and simultaneously, the linear transformation is introduced to make different embeddings from different pattern spaces comparable. Hence, the consensus pattern can be learned from multiple embeddings of multiple representations. We derive an iterating algorithm to solve the above problem. Some theoretical analyses and out-of-sample extensions are also provided. Promising experiments on various data sets, together with some important discussions, are also presented to demonstrate the effectiveness of the proposed algorithm.     

A unified framework for semi-supervised dimensionality reduction

In practice, many applications require a dimensionality reduction method to deal with the partially labeled problem. In this paper, we propose a semi-supervised dimensionality reduction framework, which can efficiently handle the unlabeled data. Under the framework, several classical methods, such as principal component analysis (PCA), linear discriminant analysis (LDA), maximum margin criterion (MMC), locality preserving projections (LPP) and their corresponding kernel versions can be seen as special cases. For high-dimensional data, we can give a low-dimensional embedding result for both discriminating multi-class sub-manifolds and preserving local manifold structure. Experiments show that our algorithms can significantly improve the accuracy rates of the corresponding supervised and unsupervised approaches.     

Perceptual relativity-based semi-supervised dimensionality reduction algorithm

As we all know, a well-designed graph tends to result in good performance for graph-based semi-supervised learning. Although most graph-based semi-supervised dimensionality reduction approaches perform very well on clean data sets, they usually cannot construct a faithful graph which plays an important role in getting a good performance, when performing on the high dimensional, sparse or noisy data. So this will generally lead to a dramatic performance degradation. To deal with these issues, this paper proposes a feasible strategy called relative semi-supervised dimensionality reduction (RSSDR) by utilizing the perceptual relativity to semi-supervised dimensionality reduction. In RSSDR, firstly, relative transformation will be performed over the training samples to build the relative space. It should be indicated that relative transformation improves the distinguishing ability among data points and diminishes the impact of noise on semi-supervised dimensionality reduction. Secondly, the edge weights of neighborhood graph will be determined through minimizing the local reconstruction error in the relative space such that it can preserve the global geometric structure as well as the local one of the data. Extensive experiments on face, UCI, gene expression, artificial and noisy data sets have been provided to validate the feasibility and effectiveness of the proposed algorithm with the promising results both in classification accuracy and robustness.     

Learning a tensor subspace for semi-supervised dimensionality reduction

The high-dimensional data is frequently encountered and processed in real-world applications and unlabeled samples are readily available, but labeled or pairwise constrained ones are fairly expensive to capture. Traditionally, when a pattern itself is an n ₁?×?n ₂ image, the image first has to be vectorized to the vector pattern in $ \Re^{{n_{1} \times n_{2} }} $ by concatenating its pixels. However, such a vector representation fails to take into account the spatial locality of pixels in the images, which are intrinsically matrices. In this paper, we propose a tensor subspace learning-based semi-supervised dimensionality reduction algorithm (TS²DR), in which an image is naturally represented as a second-order tensor in $ \Re^{{n_{1} }} \otimes \Re^{{n_{2} }} $ and domain knowledge in the forms of pairwise similarity and dissimilarity constraints is used to specify whether pairs of instances belong to the same class or different classes. TS²DR has an analytic form of the global structure preserving embedding transformation, which can be easily computed based on eigen-decomposition. We also verify the efficiency of TS²DR by conducting unbalanced data classification experiments based on the benchmark real-word databases. Numerical results show that TS²DR tends to capture the intrinsic structure characteristics of the given data and achieves better classification accuracy, while being much more efficient.     

Stable and orthogonal local discriminant embedding using trace ratio criterion for dimensionality reduction

Stable orthogonal local discriminant embedding (SOLDE) is a recently proposed dimensionality reduction method, in which the similarity, diversity and interclass separability of the data samples are well utilized to obtain a set of orthogonal projection vectors. By combining multiple features of data, it outperforms many prevalent dimensionality reduction methods. However, the orthogonal projection vectors are obtained by a step-by-step procedure, which makes it computationally expensive. By generalizing the objective function of the SOLDE to a trace ratio problem, we propose a stable and orthogonal local discriminant embedding using trace ratio criterion (SOLDE-TR) for dimensionality reduction. An iterative procedure is provided to solve the trace ratio problem, due to which the SOLDE-TR method is always faster than the SOLDE. The projection vectors of the SOLDE-TR will always converge to a global solution, and the performances are always better than that of the SOLDE. Experimental results on two public image databases demonstrate the effectiveness and advantages of the proposed method.     

Adaptive sparse graph learning based dimensionality reduction for classification

To preserve the sparsity structure in dimensionality reduction, sparsity preserving projection (SPP) is widely used in many fields of classification, which has the advantages of noise robustness and data adaptivity compared with other graph based method. However, the sparsity parameter of SPP is fixed for all samples without any adjustment. In this paper, an improved SPP method is proposed, which has an adaptive parameter adjustment strategy during sparse graph construction. With this adjustment strategy, the sparsity parameter of each sample is adjusted adaptively according to the relationship of those samples with nonzero sparse representation coefficients, by which the discriminant information of graph is enhanced. With the same expectation, similarity information both in original space and projection space is applied for sparse representation as guidance information. Besides, a new measurement is introduced to control the influence of each sample’s local structure on projection learning, by which more correct discriminant information should be preserved in the projection space. With the contributions of above strategies, the low-dimensional space with high discriminant ability is found, which is more beneficial for classification. Experimental results on three datasets demonstrate that the proposed approach can achieve better classification performance over some available state-of-the-art approaches.     

Integrating local and global topological structures for semi-supervised dimensionality reduction

Dimensionality reduction plays an important role in many machine learning tasks. This paper studies semi-supervised dimensionality reduction using pairwise constraints. In this setting, domain knowledge is given in the form of pairwise constraint, which specifies whether a pair of instances belongs to the same class (must-link constraint) or different classes (cannot-link constraint). In this paper, a novel semi-supervised dimensionality reduction method called LGS3DR is proposed, which can integrate both local and global topological structures of the data as well as pairwise constraints. The LGS3DR method is effective and has a closed form solution. Experiments on data visualization and face recognition show that LGS3DR is superior to many existing dimensionality reduction methods.     

Sentiment visualization and classification via semi-supervised nonlinear dimensionality reduction

Sentiment analysis, which detects the subjectivity or polarity of documents, is one of the fundamental tasks in text data analytics. Recently, the number of documents available online and offline is increasing dramatically, and preprocessed text data have more features. This development makes analysis more complex to be analyzed effectively. This paper proposes a novel semi-supervised Laplacian eigenmap (SS-LE). The SS-LE removes redundant features effectively by decreasing detection errors of sentiments. Moreover, it enables visualization of documents in perceptible low dimensional embedded space to provide a useful tool for text analytics. The proposed method is evaluated using multi-domain review data set in sentiment visualization and classification by comparing other dimensionality reduction methods. SS-LE provides a better similarity measure in the visualization result by separating positive and negative documents properly. Sentiment classification models trained over reduced data by SS-LE show higher accuracy. Overall, experimental results suggest that SS-LE has the potential to be used to visualize documents for the ease of analysis and to train a predictive model in sentiment analysis. SS-LE can also be applied to any other partially annotated text data sets.     

A unified dimensionality reduction framework for semi-paired and semi-supervised multi-view data

Canonical correlation analysis (CCA) is a popular and powerful dimensionality reduction method to analyze paired multi-view data. However, when facing semi-paired and semi-supervised multi-view data which widely exist in real-world problems, CCA usually performs poorly due to its requirement of data pairing between different views and un-supervision in nature. Recently, several extensions of CCA have been proposed, however, they just handle the semi-paired scenario by utilizing structure information in each view or just deal with semi-supervised scenario by incorporating the discriminant information. In this paper, we present a general dimensionality reduction framework for semi-paired and semi-supervised multi-view data which naturally generalizes existing related works by using different kinds of prior information. Based on the framework, we develop a novel dimensionality reduction method, termed as semi-paired and semi-supervised generalized correlation analysis (S²GCA). S²GCA exploits a small amount of paired data to perform CCA and at the same time, utilizes both the global structural information captured from the unlabeled data and the local discriminative information captured from the limited labeled data to compensate the limited pairedness. Consequently, S²GCA can find the directions which make not only maximal correlation between the paired data but also maximal separability of the labeled data. Experimental results on artificial and four real-world datasets show its effectiveness compared to the existing related dimensionality reduction methods.     

A dimensionality reduction technique based on a least squared error criterion

A method of achieving dimensionality reduction is presented. The reduced dimensionality is achieved by utilizing a least squared error technique under the assumption that the goodness criterion is the maximum separation of classes. The criterion is met by first maximizing the spread of the cluster centers, and then minimizing the within class scatter. The derivation of the desired transformation from an arbitrary p-space to a space of lower dimension, say l, is completed with the assumption that the cluster centers are known. The criterion for the cluster center location is the minimization of the variance of the distance between the cluster center and the transformed pattern. It is demonstrated that the resulting cluster center set is similar to the simplex signal set in communication theory, which is a minimum energy signal set.     

A new quality assessment criterion for nonlinear dimensionality reduction

A new quality assessment criterion for evaluating the performance of the nonlinear dimensionality reduction (NLDR) methods is proposed in this paper. Differing from the current quality assessment criteria focusing on the local-neighborhood-preserving performance of the NLDR methods, the proposed criterion capitalizes on a new aspect, the global-structure-holding performance, of the NLDR methods. By taking both properties into consideration, the intrinsic capability of the NLDR methods can be more faithfully reflected, and hence more rational measurement for the proper selection of NLDR methods in real-life applications can be offered. The theoretical argument is supported by experiment results implemented on a series of benchmark data sets.     

Trace ratio criterion for multi-view discriminant analysis

Applied Intelligence - Learning from heterogeneous views, termed multi-view learning (MvL), is a significant yet challenging problem in computer vision. Many existing MvL methods apply the two-view...