加权成对约束半监督局部维数约减算法

考虑到已有的半监督维数约减方法在利用边信息时将所有边信息等同,不能充分挖掘边所含信息,提出加权成对约束半监督局部维数约减算法(WSLDR).通过构建近邻图对边信息进行扩充,使边信息数量有所增加.另外,根据边所含信息量的不同构建边的权系数矩阵.将边信息融入近邻图对其进行修正,对修正后的近邻图和加权的成对约束寻找最优投影.算法不仅保持了数据的内在局部几何结构,而且使得类内数据分布更加紧密,类间数据分布更加分散.在UCI数据集上的实验结果验证了该算法的有效性.     

Multi-class pairwise linear dimensionality reduction using heteroscedastic schemes

Linear dimensionality reduction (LDR) techniques have been increasingly important in pattern recognition (PR) due to the fact that they permit a relatively simple mapping of the problem onto a lower-dimensional subspace, leading to simple and computationally efficient classification strategies. Although the field has been well developed for the two-class problem, the corresponding issues encountered when dealing with multiple classes are far from trivial. In this paper, we argue that, as opposed to the traditional LDR multi-class schemes, if we are dealing with multiple classes, it is not expedient to treat it as a multi-class problem per se. Rather, we shall show that it is better to treat it as an ensemble of Chernoff-based two-class reductions onto different subspaces, whence the overall solution is achieved by resorting to either Voting, Weighting, or to a Decision Tree strategy. The experimental results obtained on benchmark datasets demonstrate that the proposed methods are not only efficient, but that they also yield accuracies comparable to that obtained by the optimal Bayes classifier.     

基于局域主方向重构的适应性非线性维数约减

现有的主要非线性维数约减算法,如SIE和Isomap等,其邻域参数的设定是全局性的。仿真表明,对于局域流形结构差异较大的数据集,全局一致的邻域参数可能无法获得合理的嵌入结果。为此给出基于局域主方向重构的适应性邻域选择算法。算法首先为每个参考点选择一个邻域集,使各邻域集近似处于局域主线性子空间,并计算各邻域集的基向量集;再由基向量集对各邻域点的线性拟合误差判定该邻域点与主线性子空间的偏离程度,删除偏离较大的点。仿真表明,基于局域主方向重构的适应性邻域选择可有效处理局域流形结构差异较大的数据集;且相对于已有的适应性邻域选择算法,可以更好屏蔽靠近参考点的孤立噪声点及较大的空间曲率导致的虚假连通性。     

基于图嵌入的自适应多视降维方法

随着监控摄像头的普及和数据采集技术的快速发展,多视数据呈现出规模大、维度高和多源异构的特点,使得数据存储空间大、传输慢、算法复杂度高,造成“有数据、难利用”的困境。到目前为止,国内外在多视降维方面的研究还比较少。针对这一问题,本文提出一种基于图嵌入的自适应多视降维方法。该方法在考虑视角内降维后数据重构原始高维数据的基础上,提出自适应学习相似矩阵来探索不同视角之间降维后数据的关联关系,学习各视数据的正交投影矩阵实现多视降维任务。本文在多个数据集上对降维后的多视数据进行了聚类/识别实验验证,实验结果表明基于图嵌入的自适应多视降维方法优于其他降维方法。     

An adaptive and dynamic dimensionality reduction method for high-dimensional indexing

The notorious “dimensionality curse” is a well-known phenomenon for any multi-dimensional indexes attempting to scale up to high dimensions. One well-known approach to overcome degradation in performance with respect to increasing dimensions is to reduce the dimensionality of the original dataset before constructing the index. However, identifying the correlation among the dimensions and effectively reducing them are challenging tasks. In this paper, we present an adaptive Multi-level Mahalanobis-based Dimensionality Reduction (MMDR) technique for high-dimensional indexing. Our MMDR technique has four notable features compared to existing methods. First, it discovers elliptical clusters for more effective dimensionality reduction by using only the low-dimensional subspaces. Second, data points in the different axis systems are indexed using a single B ⁺-tree. Third, our technique is highly scalable in terms of data size and dimension. Finally, it is also dynamic and adaptive to insertions. An extensive performance study was conducted using both real and synthetic datasets, and the results show that our technique not only achieves higher precision, but also enables queries to be processed efficiently.     

Perceptual relativity-based semi-supervised dimensionality reduction algorithm

As we all know, a well-designed graph tends to result in good performance for graph-based semi-supervised learning. Although most graph-based semi-supervised dimensionality reduction approaches perform very well on clean data sets, they usually cannot construct a faithful graph which plays an important role in getting a good performance, when performing on the high dimensional, sparse or noisy data. So this will generally lead to a dramatic performance degradation. To deal with these issues, this paper proposes a feasible strategy called relative semi-supervised dimensionality reduction (RSSDR) by utilizing the perceptual relativity to semi-supervised dimensionality reduction. In RSSDR, firstly, relative transformation will be performed over the training samples to build the relative space. It should be indicated that relative transformation improves the distinguishing ability among data points and diminishes the impact of noise on semi-supervised dimensionality reduction. Secondly, the edge weights of neighborhood graph will be determined through minimizing the local reconstruction error in the relative space such that it can preserve the global geometric structure as well as the local one of the data. Extensive experiments on face, UCI, gene expression, artificial and noisy data sets have been provided to validate the feasibility and effectiveness of the proposed algorithm with the promising results both in classification accuracy and robustness.     

一种邻域线性竞争的排列降维方法

局部线性嵌入算法以及局部切空间排列算法是目前对降维研究有着重要影响的算法, 但对于稀疏数据及噪声数据, 在使用这些经典算法降维时效果欠佳。一个重要问题就是这些算法在处理局部邻域时存在信息涵盖量不足。对经典算法中全局信息和局部信息的提取机制进行分析后, 提出一种邻域线性竞争的排列方法（neighborhood linear rival alignment algorithm, NLRA）。通过对数据点的近邻作局部结构提取, 有效挖掘稀疏数据内部信息, 使得数据整体降维效果更加稳定。通过手工流形和真实数据集的实验, 验证了算法的有效性和稳定性。     

An adaptive global reduction algorithm for wormhole-routed 2D meshes

This paper presents a global reduction algorithm for wormhole-routed 2D meshes. Well-known reduction algorithms that are optimized for short vectors have complexity O(M log N), where N = n × n is the number of nodes, and M the vector length. Algorithms suitable for long vectors have complexity O(√N + M). Previously known asymptotically optimal algorithms with complexity O(log N + M) incur inherent network contention among constituent messages. The proposed algorithm adapts to the given vector length, resulting in complexities O(M log N) for short vectors, O(log N + M) for medium-sized vectors, and O(√N + M) for sufficiently long vectors. The O(√N + M) version is preferred to the O(log N + M) version for long vectors, due to its small coefficient associated with M, the dominating factor for such vectors. The algorithm is contention-free in a synchronous environment. Under asynchronous execution models, depth contention (contention among message-passing steps) may occur. However, simulation studies show that the effect of depth contention on the actual performance is negligible.     

基于最优密度方向的等距映射降维算法

等距映射算法(ISOMAP)是一种典型的非线性流形降维算法,该算法可在尽量保持高维数据测地距离与低维数据空间距离对等关系的基础上实现降维.但ISOMAP容易受噪声的影响,导致数据降维后不能保持高维拓扑结构.针对这一问题,提出了一种基于最优密度方向的等距映射(ODD-ISOMAP)算法.该算法通过筛选数据的自然邻居确定每...     

一种在源数据稀疏情况下的数据降维算法

流形学习方法是根据流形的定义提出的一种非线性数据降维方法,主要思想是发现嵌入在高维数据空间的低维光滑流形。从分析基于流形学习理论的局部线性嵌入算法入手,针对传统的局部线性嵌入算法在源数据稀疏时会失效的缺点,提出了基于局部线性逼近思想的流形学习算法,并在S-曲线上采样测试取得良好降维效果。     

基于自变量简约的大规模稀疏多目标优化

现有的大多数进化算法在求解大规模优化问题时性能会随决策变量维数的增长而下降。通常,多目标优化的Pareto有效解集是自变量空间的一个低维流形,该流形的维度远小于自变量空间的维度。鉴于此,提出一种基于自变量简约的多目标进化算法求解大规模稀疏多目标优化问题。该算法通过引入局部保持投影降维,保留原始自变量空间中的局部近邻关系,并设计一个归档集,将寻找到的非劣解存入其中进行训练,以提高投影的准确性。将该算法与四种流行的多目标进化算法在一系列测试问题和实际应用问题上进行了比较。实验结果表明,所提算法在解决稀疏多目标问题上具有较好的效果。因此,通过自变量简约能降低问题的求解难度,提高算法的搜索效率,在解决大规模稀疏多目标问题方面具有显著的优势。     

基于非线性降维的人脸识别新算法

提出了一种有监督的非线性核子空间人脸识别新方法。在核邻域保持投影方法的局部邻域构建过程中引入监督机制,更好地利用了人脸训练样本的类别信息,提高人脸识别的效率;在获取最佳重建权矩阵的过程中引入一个正则项约束 ,降低了其对噪声的敏感性。实验阶段,采用了AT&T和Yale人脸库和最近邻分类器测试该方法。结果表明,这种方法是有效的,且较无监督的KNPP方法及传统的经典人脸识别法具有更好的识别率和鲁棒性。     

基于遗传算法的无监督分形属性规约技术

属性规约是应对“维数灾难”的有效技术,分形属性规约FDR（Fractal Dimensionality Reduction）是近年来出现的一种无监督属性选择技术,令人遗憾的是其需要多遍扫描数据集,因而难于应对高维数据集情况;基于遗传算法的属性规约技术对于高维数据而言优越于传统属性选择技术,但其无法应用于无监督学习领域。为此,结合遗传算法内在随机并行寻优机制及分形属性选择的无监督特点,设计并实现了基于遗传算法的无监督分形属性子集选择算法GABUFSS（Genetic Algorithm Based Unsupervised Feature Subset Selection）。基于合成与实际数据集的实验对比分析了GABUFSS算法与FDR算法的性能,结果表明GABUFSS相对优于FDR算法,并具有发现等价结果属性子集的特点。     

非线性降维算法及其在医院绩效考核上的应用

流形学习算法中的等距嵌入算法（ISOMAP）具有对离群点敏感的瑕疵,针对此问题,提出利用基于共享近邻的距离度量方式,并充分利用了流形上对象的局部密度信息,有效改善了算法的性能,提高了算法的健壮性。同时,首次尝试将该改进的流形学习算法应用于医院绩效考核。人工数据与真实数据上的实验表明,改进的算法健壮且有效,在绩效考核上应用成功。     

A scalable supervised algorithm for dimensionality reduction on streaming data

Algorithms on streaming data have attracted increasing attention in the past decade. Among them, dimensionality reduction algorithms are greatly interesting due to the desirability of real tasks. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are two of the most widely used dimensionality reduction approaches. However, PCA is not optimal for general classification problems because it is unsupervised and ignores valuable label information for classification. On the other hand, the performance of LDA is degraded when encountering limited available low-dimensional spaces and singularity problem. Recently, Maximum Margin Criterion (MMC) was proposed to overcome the shortcomings of PCA and LDA. Nevertheless, the original MMC algorithm could not satisfy the streaming data model to handle large-scale high-dimensional data set. Thus an effective, efficient and scalable approach is needed. In this paper, we propose a supervised incremental dimensionality reduction algorithm and its extension to infer adaptive low-dimensional spaces by optimizing the maximum margin criterion. Experimental results on a synthetic dataset and real datasets demonstrate the superior performance of our proposed algorithm on streaming data.     

A dimensionality reduction algorithm and its application for interactive visualization

Visualization is one of the most effective methods for analyzing how high-dimensional data are distributed. Dimensionality reduction techniques, such as PCA, can be used to map high dimensional data to a two- or three-dimensional space. In this paper, we propose an algorithm called HyperMap that can be effectively applied to visualization. Our algorithm can be seen as a generalization of FastMap. It preserves its linear computation complexity, and overcomes several main shortcomings, especially in visualization. Since there are more than two pivot objects in each axis of a target space, more distance information needs to be preserved in each dimension. Then in visualization, the number of pivot objects can go beyond the limitation of six (2-pivot objects × 3-dimensions). Our HyperMap algorithm also gives more flexibility to the target space, such that the data distribution can be observed from various viewpoints. Its effectiveness is confirmed by empirical evaluations on both real and synthetic datasets.     

A novel supervised dimensionality reduction algorithm: Graph-based Fisher analysis

In this paper, a novel supervised dimensionality reduction (DR) algorithm called graph- based Fisher analysis (GbFA) is proposed. More specifically, we redefine the intrinsic and penalty graph and trade off the importance degrees of the same-class points to the intrinsic graph and the importance degrees of the not-same-class points to the penalty graph by a strictly monotone decreasing function; then the novel feature extraction criterion based on the intrinsic and penalty graph is applied. For the non-linearly separable problems, we study the kernel extensions of GbFA with respect to positive definite kernels and indefinite kernels, respectively. In addition, experiments are provided for analyzing and illustrating our results.     

Fractional-step dimensionality reduction

Linear projections for dimensionality reduction, computed using linear discriminant analysis (LDA), are commonly based on optimization of certain separability criteria in the output space. The resulting optimization problem is linear, but these separability criteria are not directly related to the classification accuracy in the output space. Consequently, a trial and error procedure has to be invoked, experimenting with different separability criteria that differ in the weighting function used and selecting the one that performed best on the training set. Often, even the best weighting function among the trial choices results in poor classification of data in the subspace. In this short paper, we introduce the concept of fractional dimensionality and develop an incremental procedure, called the fractional-step LDA (F-LDA) to reduce the dimensionality in fractional steps. The F-LDA algorithm is more robust to the selection of weighting function and for any given weighting function, it finds a subspace in which the classification accuracy is higher than that obtained using LDA     

An HOG-LGBPHS human detector with dimensionality reduction by PLS

By combining Histogram of Oriented Gradient (HOG), which is based on evaluating well-normalized local histograms of image gradient orientations in a dense grid, with Local Gabor Binary Pattern Histogram Sequence (LGBPHS), which concatenate the histograms of all the local regions of all the local Gabor magnitude binary pattern maps, as a feature set, we proposed a novel human detection feature. We employ Partial Least Squares (PLS) analysis, an efficient dimensionality reduction technique, to project the feature onto a much lower dimensional subspace (9 dimensions, reduced from the original over 12000). We test the new feature in INRIA person dataset by using a linear SVM, and it yields an error rate of 1.35% with a false negatives (FN) rate of 0.46% and a false positive (FP) rate of 0.89%, while the error rate of HOG is 7.11% with a FN rate of 4.09% and a FP rate of 3.02%, and the error rate of LGBPHS is 13.55% with a FN rate of 4.94% and a FP rate of 8.61%.     

Robust linear dimensionality reduction

We present a novel family of data-driven linear transformations, aimed at finding low-dimensional embeddings of multivariate data, in a way that optimally preserves the structure of the data. The well-studied PCA and Fisher's LDA are shown to be special members in this family of transformations, and we demonstrate how to generalize these two methods such as to enhance their performance. Furthermore, our technique is the only one, to the best of our knowledge, that reflects in the resulting embedding both the data coordinates and pairwise relationships between the data elements. Even more so, when information on the clustering (labeling) decomposition of the data is known, this information can also be integrated in the linear transformation, resulting in embeddings that clearly show the separation between the clusters, as well as their internal structure. All of this makes our technique very flexible and powerful, and lets us cope with kinds of data that other techniques fail to describe properly.