Kernel based nonlinear dimensionality reduction for microarray gene expression data analysis

Accurate recognition of cancers based on microarray gene expressions is very important for doctors to choose a proper treatment. Genomic microarrays are powerful research tools in bioinformatics and modern medicinal research. However, a simple microarray experiment often leads to very high-dimensional data and a huge amount of information, the vast amount of data challenges researchers into extracting the important features and reducing the high dimensionality. This paper proposed the kernel method based locally linear embedding to selecting the optimal number of nearest neighbors, constructing uniform distribution manifold. In this paper, a nonlinear dimensionality reduction kernel method based locally linear embedding is proposed to select the optimal number of nearest neighbors, constructing uniform distribution manifold. In addition, support vector machine which has given rise to the development of a new class of theoretically elegant learning machines will be used to classify and recognise genomic microarray. We demonstrate the application of the techniques to two published DNA microarray data sets. The experimental results and comparisons demonstrate that the proposed method is effective approach.     

Multiple view semi-supervised dimensionality reduction

Multiple view data, together with some domain knowledge in the form of pairwise constraints, arise in various data mining applications. How to learn a hidden consensus pattern in the low dimensional space is a challenging problem. In this paper, we propose a new method for multiple view semi-supervised dimensionality reduction. The pairwise constraints are used to derive embedding in each view and simultaneously, the linear transformation is introduced to make different embeddings from different pattern spaces comparable. Hence, the consensus pattern can be learned from multiple embeddings of multiple representations. We derive an iterating algorithm to solve the above problem. Some theoretical analyses and out-of-sample extensions are also provided. Promising experiments on various data sets, together with some important discussions, are also presented to demonstrate the effectiveness of the proposed algorithm.     

半监督复杂结构数据降维方法

现有的一些典型半监督降维算法,往往在利用标记信息的同时却忽略了样本数据本身的流形特征,或者是对流形特征使用不当,导致算法性能表现不佳并且应用领域狭窄。针对上述问题提出了半监督复杂结构数据降维方法,同时保持样本数据的全局与局部的流形特征。通过设置适当的目标函数,使算法结果能有更广泛的应用场合,实验证明了算法的有效性。     

An empirical evaluation on dimensionality reduction schemes for dissimilarity-based classifications

This paper presents an empirical evaluation on the methods of reducing the dimensionality of dissimilarity spaces for optimizing dissimilarity-based classifications (DBCs). One problem of DBCs is the high dimensionality of the dissimilarity spaces. To address this problem, two kinds of solutions have been proposed in the literature: prototype selection (PS) based methods and dimension reduction (DR) based methods. Although PS-based and DR-based methods have been explored separately by many researchers, not much analysis has been done on the study of comparing the two. Therefore, this paper aims to find a suitable method for optimizing DBCs by a comparative study. Our empirical evaluation, obtained with the two approaches for an artificial and three real-life benchmark databases, demonstrates that DR-based methods, such as principal component analysis (PCA) and linear discriminant analysis (LDA) based methods, generally improve the classification accuracies more than PS-based methods. Especially, the experimental results demonstrate that PCA is more useful for the well-represented data sets, while LDA is more helpful for the small sample size problems.     

An adaptive and dynamic dimensionality reduction method for high-dimensional indexing

The notorious “dimensionality curse” is a well-known phenomenon for any multi-dimensional indexes attempting to scale up to high dimensions. One well-known approach to overcome degradation in performance with respect to increasing dimensions is to reduce the dimensionality of the original dataset before constructing the index. However, identifying the correlation among the dimensions and effectively reducing them are challenging tasks. In this paper, we present an adaptive Multi-level Mahalanobis-based Dimensionality Reduction (MMDR) technique for high-dimensional indexing. Our MMDR technique has four notable features compared to existing methods. First, it discovers elliptical clusters for more effective dimensionality reduction by using only the low-dimensional subspaces. Second, data points in the different axis systems are indexed using a single B ⁺-tree. Third, our technique is highly scalable in terms of data size and dimension. Finally, it is also dynamic and adaptive to insertions. An extensive performance study was conducted using both real and synthetic datasets, and the results show that our technique not only achieves higher precision, but also enables queries to be processed efficiently.     

Discriminative concept factorization for data representation

Non-negative matrix factorization (NMF) has become a popular technique for finding low-dimensional representations of data. While the standard NMF can only be performed in the original feature space, one variant of NMF, named concept factorization, can be naturally kernelized and inherits all the strengths of NMF. To make use of label information, we propose a semi-supervised concept factorization technique called discriminative concept factorization (DCF) for data representation in this paper. DCF adopts a unified objective to combine the task of data reconstruction with the task of classification. These two tasks have mutual impacts on each other, which results in a concept factorization adapted to the classification task and a classifier built on the low-dimensional representations. Furthermore, we develop an iterative algorithm to solve the optimization problem through alternative convex programming. Experimental results on three real-word classification tasks demonstrate the effectiveness of DCF.     

基于局部重构与全局保持的半监督维数约减算法

针对基于局部与全局保持的半监督维数约减算法(LGSSDR)对部域参数选择比较敏感以及对部域图边权值设定不够准确的问题,提出一种基于局部重构与全局保持的半监督维数约减算法(工RGPSSDR)。该算法通过最小化局部重构误差来确定部域图的边权值,在保持数据集局部结构的同时能够保持其全局结构。在Extended YaleB和 CMU PIE标准人脸库上的实验结果表明LRGPSSDR算法的分类性能要优于其它半监督维数约减算法。     

Supervised dimensionality reduction via sequential semidefinite programming

Many dimensionality reduction problems end up with a trace quotient formulation. Since it is difficult to directly solve the trace quotient problem, traditionally the trace quotient cost function is replaced by an approximation such that the generalized eigenvalue decomposition can be applied. In contrast, we directly optimize the trace quotient in this work. It is reformulated as a quasi-linear semidefinite optimization problem, which can be solved globally and efficiently using standard off-the-shelf semidefinite programming solvers. Also this optimization strategy allows one to enforce additional constraints (for example, sparseness constraints) on the projection matrix. We apply this optimization framework to a novel dimensionality reduction algorithm. The performance of the proposed algorithm is demonstrated in experiments by several UCI machine learning benchmark examples, USPS handwritten digits as well as ORL and Yale face data.     

Semi-supervised orthogonal discriminant analysis via label propagation

Trace ratio is a natural criterion in discriminant analysis as it directly connects to the Euclidean distances between training data points. This criterion is re-analyzed in this paper and a fast algorithm is developed to find the global optimum for the orthogonal constrained trace ratio problem. Based on this problem, we propose a novel semi-supervised orthogonal discriminant analysis via label propagation. Differing from the existing semi-supervised dimensionality reduction algorithms, our algorithm propagates the label information from the labeled data to the unlabeled data through a specially designed label propagation, and thus the distribution of the unlabeled data can be explored more effectively to learn a better subspace. Extensive experiments on toy examples and real-world applications verify the effectiveness of our algorithm, and demonstrate much improvement over the state-of-the-art algorithms.     

Multi-period portfolio selection using kernel-based control policy with dimensionality reduction

This paper studies a nonlinear control policy for multi-period investment. The nonlinear strategy we implement is categorized as a kernel method, but solving large-scale instances of the resulting optimization problem in a direct manner is computationally intractable in the literature. In order to overcome this difficulty, we employ a dimensionality reduction technique which is often used in principal component analysis. Numerical experiments show that our strategy works not only to reduce the computation time, but also to improve out-of-sample investment performance.     

一种用于处理高维稀疏数据的半监督聚类算法

半监督聚类是近年来研究的热点,传统的方法是在无监督算法的基础上加入有限的背景知识来提高聚类性能.然而大多数半监督聚类技术都基于邻近或密度,难以处理高维数据,因此必须将约减的特征加入到半监督聚类过程中.为解决此问题,提出了一种新的半监督聚类算法框架.该算法利用样本约束传递性进行预处理,然后将特征投影到低维空间实现降维,最终用半监督算法对约减后的样本进行聚类.通过实验同现行主要降维方法进行了比较,说明此方法能有效地处理高维数据,聚类效果良好.     

A visualization metric for dimensionality reduction

Data visualization of high-dimensional data is possible through the use of dimensionality reduction techniques. However, in deciding which dimensionality reduction techniques to use in practice, quantitative metrics are necessary for evaluating the results of the transformation and visualization of the lower dimensional embedding. In this paper, we propose a manifold visualization metric based on the pairwise correlation of the geodesic distance in a data manifold. This metric is compared with other metrics based on the Euclidean distance, Mahalanobis distance, City Block metric, Minkowski metric, cosine distance, Chebychev distance, and Spearman distance. The results of applying different dimensionality reduction techniques on various types of nonlinear manifolds are compared and discussed. Our experiments show that our proposed metric is suitable for quantitatively evaluating the results of the dimensionality reduction techniques if the data lies on an open planar nonlinear manifold. This has practical significance in the implementation of knowledge-based visualization systems and the application of knowledge-based dimensionality reduction methods.     

Enhanced algorithm for high-dimensional data classification

Minimum class variance support vector machine (MCVSVM) and large margin linear projection (LMLP) classifier, in contrast with traditional support vector machine (SVM), take the distribution information of the data into consideration and can obtain better performance. However, in the case of the singularity of the within-class scatter matrix, both MCVSVM and LMLP only exploit the discriminant information in a single subspace of the within-class scatter matrix and discard the discriminant information in the other subspace. In this paper, a so-called twin-space support vector machine (TSSVM) algorithm is proposed to deal with the high-dimensional data classification task where the within-class scatter matrix is singular. TSSVM is rooted in both the non-null space and the null space of the within-class scatter matrix, takes full advantage of the discriminant information in the two subspaces, and so can achieve better classification accuracy. In the paper, we first discuss the linear case of TSSVM, and then develop the nonlinear TSSVM. Experimental results on real datasets validate the effectiveness of TSSVM and indicate its superior performance over MCVSVM and LMLP.     

Adaptive sparse graph learning based dimensionality reduction for classification

To preserve the sparsity structure in dimensionality reduction, sparsity preserving projection (SPP) is widely used in many fields of classification, which has the advantages of noise robustness and data adaptivity compared with other graph based method. However, the sparsity parameter of SPP is fixed for all samples without any adjustment. In this paper, an improved SPP method is proposed, which has an adaptive parameter adjustment strategy during sparse graph construction. With this adjustment strategy, the sparsity parameter of each sample is adjusted adaptively according to the relationship of those samples with nonzero sparse representation coefficients, by which the discriminant information of graph is enhanced. With the same expectation, similarity information both in original space and projection space is applied for sparse representation as guidance information. Besides, a new measurement is introduced to control the influence of each sample’s local structure on projection learning, by which more correct discriminant information should be preserved in the projection space. With the contributions of above strategies, the low-dimensional space with high discriminant ability is found, which is more beneficial for classification. Experimental results on three datasets demonstrate that the proposed approach can achieve better classification performance over some available state-of-the-art approaches.     

Kernel logistic PLS: A tool for supervised nonlinear dimensionality reduction and binary classification

“Kernel logistic PLS” (KL-PLS) is a new tool for supervised nonlinear dimensionality reduction and binary classification. The principles of KL-PLS are based on both PLS latent variables construction and learning with kernels. The KL-PLS algorithm can be seen as a supervised dimensionality reduction (complexity control step) followed by a classification based on logistic regression. The algorithm is applied to 11 benchmark data sets for binary classification and to three medical problems. In all cases, KL-PLS proved its competitiveness with other state-of-the-art classification methods such as support vector machines. Moreover, due to successions of regressions and logistic regressions carried out on only a small number of uncorrelated variables, KL-PLS allows handling high-dimensional data. The proposed approach is simple and easy to implement. It provides an efficient complexity control by dimensionality reduction and allows the visual inspection of data segmentation.     

A scalable supervised algorithm for dimensionality reduction on streaming data

Algorithms on streaming data have attracted increasing attention in the past decade. Among them, dimensionality reduction algorithms are greatly interesting due to the desirability of real tasks. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are two of the most widely used dimensionality reduction approaches. However, PCA is not optimal for general classification problems because it is unsupervised and ignores valuable label information for classification. On the other hand, the performance of LDA is degraded when encountering limited available low-dimensional spaces and singularity problem. Recently, Maximum Margin Criterion (MMC) was proposed to overcome the shortcomings of PCA and LDA. Nevertheless, the original MMC algorithm could not satisfy the streaming data model to handle large-scale high-dimensional data set. Thus an effective, efficient and scalable approach is needed. In this paper, we propose a supervised incremental dimensionality reduction algorithm and its extension to infer adaptive low-dimensional spaces by optimizing the maximum margin criterion. Experimental results on a synthetic dataset and real datasets demonstrate the superior performance of our proposed algorithm on streaming data.     

Integrating nonlinear graph based dimensionality reduction schemes with SVMs for credit rating forecasting

By integrating graph based nonlinear dimensionality reduction with support vector machines (SVMs), this study develops a novel prediction model for credit ratings forecasting. SVMs have been successfully applied in numerous areas, and have demonstrated excellent performance. However, due to the high dimensionality and nonlinear distribution of the input data, this study employed a kernel graph embedding (KGE) scheme to reduce the dimensionality of input data, and enhance the performance of SVM classifiers. Empirical results indicated that one-vs-one SVM with KGE outperforms other multi-class SVMs and traditional classifiers. Compared with other dimensionality reduction methods the performance improvement owing to KGE is significant.     

基于半监督流形学习的人脸识别方法

如何有效地将流形学习(Manifold learning,ML)和半监督学习(Semi-supervised learning,SSL)方法进行结合是近年来模式识别和机器学习领域研究的热点问题.提出一种基于半监督流形学习(Semi-supervised manifold learning,SSML)的人脸识别方法,它在部分有标签信息的人脸数据的情况下,通过利用人脸数据本身的非线性流形结构信息和部分标签信息来调整点与点之间的距离形成距离矩阵,而后基于被调整的距离矩阵进行线性近邻重建来实现维数约简,提取低维鉴别特征用于人脸识别.基于公开的人脸数据库上的实验结果表明,该方法能有效地提高人脸识别的性能.     

Perceptual relativity-based semi-supervised dimensionality reduction algorithm

As we all know, a well-designed graph tends to result in good performance for graph-based semi-supervised learning. Although most graph-based semi-supervised dimensionality reduction approaches perform very well on clean data sets, they usually cannot construct a faithful graph which plays an important role in getting a good performance, when performing on the high dimensional, sparse or noisy data. So this will generally lead to a dramatic performance degradation. To deal with these issues, this paper proposes a feasible strategy called relative semi-supervised dimensionality reduction (RSSDR) by utilizing the perceptual relativity to semi-supervised dimensionality reduction. In RSSDR, firstly, relative transformation will be performed over the training samples to build the relative space. It should be indicated that relative transformation improves the distinguishing ability among data points and diminishes the impact of noise on semi-supervised dimensionality reduction. Secondly, the edge weights of neighborhood graph will be determined through minimizing the local reconstruction error in the relative space such that it can preserve the global geometric structure as well as the local one of the data. Extensive experiments on face, UCI, gene expression, artificial and noisy data sets have been provided to validate the feasibility and effectiveness of the proposed algorithm with the promising results both in classification accuracy and robustness.     

一种在源数据稀疏情况下的数据降维算法

流形学习方法是根据流形的定义提出的一种非线性数据降维方法,主要思想是发现嵌入在高维数据空间的低维光滑流形。从分析基于流形学习理论的局部线性嵌入算法入手,针对传统的局部线性嵌入算法在源数据稀疏时会失效的缺点,提出了基于局部线性逼近思想的流形学习算法,并在S-曲线上采样测试取得良好降维效果。