基于自适应邻域嵌入的无监督特征选择算法

无监督特征选择算法可以对高维无标记数据进行有效的降维,从而减少数据处理的时间和空间复杂度,避免算法模型出现过拟合现象.然而,现有的无监督特征选择方法大都运用k近邻法捕捉数据样本的局部几何结构,忽略了数据分布不均的问题.为了解决这个问题,提出了一种基于自适应邻域嵌入的无监督特征选择(adaptive neighborhood embedding based unsupervised feature selection, ANEFS)算法,该算法根据数据集自身的分布特点确定每个样本的近邻数,进而构造样本相似矩阵,同时引入从高维空间映射到低维空间的中间矩阵,利用拉普拉斯乘子法优化目标函数进行求解.6个UCI数据集的实验结果表明：所提出的算法能够选出具有更高聚类精度和互信息的特征子集.     

基于有效距离的谱聚类算法

在现有多种距离度量和传统谱聚类算法的基础上,提出了一种新的基于有效距离的谱聚类算法(spectral clustering based on effective distance,SCED)。SCED算法通过稀疏重构系数来构建样本与样本之间的有效距离,从而代替传统谱聚类算法中的欧氏距离,进行样本之间的相似度评估。与传统距离度量相比,有效距离不仅利用了样本对之间的距离信息,同时考虑了目标样本与其他所有相关样本之间的距离信息,因而该距离度量具有全局特性。在UCI标准数据集上的实验结果表明,SCED算法能有效提高聚类效果。     

域间F-范数正则化迁移谱聚类方法

传统聚类算法在目标数据集被噪声或异常数据大量污染的场景下聚类效果不佳。针对此问题,在经典谱聚类算法(spectral clustering,SC)基础上加入迁移学习知识,提出了新的域间F-范数正则化迁移谱聚类算法(transfer spectral clustering based on inter-domain F-norm regularization,TSC-IDFR)。该算法通过第K最近邻原则为目标域数据从源域(历史数据)获取等量的可参照数据样本,然后基于域间F范数正则化机制,迁移这些源域可参照数据样本的谱聚类特征矩阵,以辅助目标域数据集上的谱聚类过程,从而解决实际问题中由于目标域数据污染带来的聚类难题,最终提高谱聚类效果。通过在模拟数据集和真实数据集上的仿真实验,证明了该算法的有效性。     

一种用于处理高维稀疏数据的半监督聚类算法

半监督聚类是近年来研究的热点,传统的方法是在无监督算法的基础上加入有限的背景知识来提高聚类性能.然而大多数半监督聚类技术都基于邻近或密度,难以处理高维数据,因此必须将约减的特征加入到半监督聚类过程中.为解决此问题,提出了一种新的半监督聚类算法框架.该算法利用样本约束传递性进行预处理,然后将特征投影到低维空间实现降维,最终用半监督算法对约减后的样本进行聚类.通过实验同现行主要降维方法进行了比较,说明此方法能有效地处理高维数据,聚类效果良好.     

基于局部重构的无监督特征选择方法

利用局部线性嵌入(LLE)算法中获得局部邻域之间的重构关系与使用最小角回归方法解决L1归一化问题都使用回归方法,针对在通过映射获得低维嵌入空间与通过特征选择获得低维空间上有着一致的思想,提出一种能保持局部重构关系的无监督谱特征选择方法.该方法利用最小二乘法计算样本的邻域重构系数,并用这些系数表示样本之间的关系,通过解决稀疏特征值问题获得能够保持样本间关系的低维嵌入空间,最后通过解决L1归一化问题实现自动特征选择.通过在四个不同数据集上的聚类实验结果证明,该方法能更准确地评价每个特征的重要性,能自动适应不同的数据集,受参数影响更小,可以明显提升聚类效果.     

基于强类别特征近邻传播的半监督文本聚类

为处理高维稀疏的大规模文档数据,提出一种基于强类别特征近邻传播(SCFAP)的半监督文本聚类算法.聚类过程中,利用少量带类别标签的监督数据,提取具有强类别区分能力的特征项以构建更有效的样本间相似性测度.并在每轮迭代完成后将类别确定性程度最高的未标记样本转移到已标注集,使算法执行效率提高.实验结果表明,这种改进对于近邻传播算法的性能和准确度的提升有较大帮助,在Reuter-21578和20Newsgroups两个相异数据集上,SCFAP算法表现较好的适用性.综合考察聚类微平均Fμ指标和类簇纯度Pt指标,该算法在少量监督信息辅助下能快速获得较好的聚类结果.     

一种基于投影聚类的无监督表型区分算法

高通量微阵列技术与手工表型标定方式间的矛盾导致了基因表达数据的获取与表型确定间的不平衡.然而,已有的表型区分方法大多是有监督的,并且通常忽略了基因间广泛存在的相互作用,根据单个基因的独立区分能力划分样本表型,选择相关基因.从新的视角,基于投影聚类的思想,提出一种考虑基因间相互关系的无监督表型区分算法USPD.通过将基因表达数据转化为具有负间隔约束的序列数据,强化了基因间的相互关系.利用设计的质量函数,采用深度优先方式遍历样本穷举树,无监督地生成样本表型划分.同时采用高效的削减策略,大大提高了算法的效率.通过将算法与现有的以基因表达数据分析为目的的代表性投影聚类算法HARP相比较,证明提出算法的高效性和有效性.     

基于两级权重的多视角聚类

在聚类过程中,多视角数据的高维性和稀疏性使视角内描述样本的不同特征对聚类结果的影响不同,同一样本在不同的视角中对聚类的贡献也不同.层次化区分视角内不同特征的权重和相同样本在不同视角内的权重是提高多视角聚类性能的重要因素.提出了具有特征级和样本级两级权重的多视角聚类(multi-view clustering based on two-level weights, MVC2W)算法.该算法引入了特征级和样本级注意力机制学习每个视角内不同特征的权重和每个样本在不同视角内的权重.两级注意力机制使算法在训练过程中能够更加关注重要的特征和重要的样本,更加合理地融合不同视角的信息,从而有效克服数据高维性和稀疏性对聚类结果的影响.此外,MVC2W将表征学习和聚类过程融为一体,协同训练、相互促进,进一步提升聚类性能.在5个稀疏程度不同的数据集上的实验结果表明：MVC2W算法的聚类性能比11个基线算法均有提升,尤其是在稀疏程度高的数据集上,MVC2W的提升更加显著.     

基于三元组排序局部性的SOCFS改进算法

特征选择是一种常用的机器学习降维方法,然而传统非监督特征选择算法在保持数据样本维度的局部结构时,却忽略了排序局部性对特征选择的影响。利用数据的三元组局部结构,构建数据之间的排序关系并在特征选择过程中进行局部性保持,提出基于三元组排序局部性的同时正交基聚类特征选择(SOCFS)改进算法,选择具有局部结构保持性且判别区分度高的特征。实验结果表明,与传统非监督特征选择算法相比,SOCFS改进算法聚类效果更好、收敛速度更快。     

一种基于深度谱空网络的高光谱图像聚类方法

通常无监督算法在对高光谱数据进行聚类时仅使用光谱信息,忽略了空间信息,使得聚类准确率较低.针对上述问题提出一种基于深度谱空网络和无监督判别极限学习的高光谱图像聚类算法.利用深度谱空网络对高光谱数据进行光谱特征和空间特征的分层交叉学习,通过反复学习获得深度空谱特征,为后续无监督聚类提供方便.在三种高光谱图像上进行实验,结果表明,该算法获得的聚类效果优于其他基于极限学习机的方法和其他无监督方法.     

Unsupervised spectral feature selection algorithms for high dimensional data

It is a significant and challenging task to detect the informative features to carry out explainable analysis for high dimensional data, especially for those with very small number of samples. Feature selection especially the unsupervised ones are the right way to deal with this challenge and realize the task. Therefore, two unsupervised spectral feature selection algorithms are proposed in this paper. They group features using advanced Self-Tuning spectral clustering algorithm based on local standard deviation, so as to detect the global optimal feature clusters as far as possible. Then two feature ranking techniques, including cosine-similarity-based feature ranking and entropy-based feature ranking, are proposed, so that the representative feature of each cluster can be detected to comprise the feature subset on which the explainable classification system will be built. The effectiveness of the proposed algorithms is tested on high dimensional benchmark omics datasets and compared to peer methods, and the statistical test are conducted to determine whether or not the proposed spectral feature selection algorithms are significantly different from those of the peer methods. The extensive experiments demonstrate the proposed unsupervised spectral feature selection algorithms outperform the peer ones in comparison, especially the one based on cosine similarity feature ranking technique. The statistical test results show that the entropy feature ranking based spectral feature selection algorithm performs best. The detected features demonstrate strong discriminative capabilities in downstream classifiers for omics data, such that the AI system built on them would be reliable and explainable. It is especially significant in building transparent and trustworthy medical diagnostic systems from an interpretable AI perspective.     

A new unsupervised feature selection algorithm using similarity‐based feature clustering

Unsupervised feature selection is an important problem, especially for high‐dimensional data. However, until now, it has been scarcely studied and the existing algorithms cannot provide satisfying performance. Thus, in this paper, we propose a new unsupervised feature selection algorithm using similarity‐based feature clustering, Feature Selection‐based Feature Clustering (FSFC). FSFC removes redundant features according to the results of feature clustering based on feature similarity. First, it clusters the features according to their similarity. A new feature clustering algorithm is proposed, which overcomes the shortcomings of K‐means. Second, it selects a representative feature from each cluster, which contains most interesting information of features in the cluster. The efficiency and effectiveness of FSFC are tested upon real‐world data sets and compared with two representative unsupervised feature selection algorithms, Feature Selection Using Similarity (FSUS) and Multi‐Cluster‐based Feature Selection (MCFS) in terms of runtime, feature compression ratio, and the clustering results of K‐means. The results show that FSFC can not only reduce the feature space in less time, but also significantly improve the clustering performance of K‐means.     

一种改进的特征子集区分度评价准则

针对特征子集区分度准则(Discernibility of feature subsets, DFS)没有考虑特征测量量纲对特征子集区分能力影响的缺陷, 引入离散系数, 提出GDFS (Generalized discernibility of feature subsets)特征子集区分度准则. 结合顺序前向、顺序后向、顺序前向浮动和顺序后向浮动4种搜索策略, 以极限学习机为分类器, 得到4种混合特征选择算法. UCI数据集与基因数据集的实验测试, 以及与DFS、Relief、DRJMIM、mRMR、LLE Score、AVC、SVM-RFE、VMInaive、AMID、AMID-DWSFS、CFR和FSSC-SD的实验比较和统计重要度检测表明: 提出的GDFS优于DFS, 能选择到分类能力更好的特征子集.     

Document Clustering With Dual Supervision Through Feature Reweighting

Traditional semi‐supervised clustering uses only limited user supervision in the form of instance seeds for clusters and pairwise instance constraints to aid unsupervised clustering. However, user supervision can also be provided in alternative forms for document clustering, such as labeling a feature by indicating whether it discriminates among clusters. This article thus fills this void by enhancing traditional semi‐supervised clustering with feature supervision, which asks the user to label discriminating features during defining (labeling) the instance seeds or pairwise instance constraints. Various types of semi‐supervised clustering algorithms were explored with feature supervision. Our experimental results on several real‐world data sets demonstrate that augmenting the instance‐level supervision with feature‐level supervision can significantly improve document clustering performance.     

Simultaneous feature selection and clustering using mixture models

Clustering is a common unsupervised learning technique used to discover group structure in a set of data. While there exist many algorithms for clustering, the important issue of feature selection, that is, what attributes of the data should be used by the clustering algorithms, is rarely touched upon. Feature selection for clustering is difficult because, unlike in supervised learning, there are no class labels for the data and, thus, no obvious criteria to guide the search. Another important problem in clustering is the determination of the number of clusters, which clearly impacts and is influenced by the feature selection issue. In this paper, we propose the concept of feature saliency and introduce an expectation-maximization (EM) algorithm to estimate it, in the context of mixture-based clustering. Due to the introduction of a minimum message length model selection criterion, the saliency of irrelevant features is driven toward zero, which corresponds to performing feature selection. The criterion and algorithm are then extended to simultaneously estimate the feature saliencies and the number of clusters.     

一种新的无监督前景目标检测方法

针对基于无监督特征提取的目标检测方法效率不高的问题,提出一种在无标记数据集中准确检测前景目标的方法.其基本出发点是:正确的特征聚类结果可以指导目标特征提取,同时准确提取的目标特征可以提高特征聚类的精度.该方法首先对无标记样本图像进行局部特征提取,然后根据最小化特征距离进行无监督特征聚类.将同一个聚类内的图像两两匹配,将特征匹配的重现程度作为特征权重,最后根据更新后的特征权重指导下一次迭代的特征聚类.多次迭代后同时得到聚类结果和前景目标.实验结果表明,该方法有效地提高Caltech 256数据集和Google车辆图像的检测精度.此外,针对目前绝大部分无监督目标检测方法不具备增量学习能力这一缺点,提出了增量学习方法实现,实验结果表明,增量学习方法有效地提高了计算速度.     

Spectral clustering with fuzzy similarity measure

Spectral clustering algorithms have been successfully used in the field of pattern recognition and computer vision. The widely used similarity measure for spectral clustering is Gaussian kernel function which measures the similarity between data points. However, it is difficult for spectral clustering to choose the suitable scaling parameter in Gaussian kernel similarity measure. In this paper, utilizing the prototypes and partition matrix obtained by fuzzy c-means clustering algorithm, we develop a fuzzy similarity measure for spectral clustering (FSSC). Furthermore, we introduce the K-nearest neighbor sparse strategy into FSSC and apply the sparse FSSC to texture image segmentation. In our experiments, we firstly perform some experiments on artificial data to verify the efficiency of the proposed fuzzy similarity measure. Then we analyze the parameters sensitivity of our method. Finally, we take self-tuning spectral clustering and Nyström methods for baseline comparisons, and apply these three methods to the synthetic texture and remote sensing image segmentation. The experimental results show that the proposed method is significantly effective and stable.     

面向大规模特征选择的自监督数据驱动粒子群优化算法

大规模特征选择问题的求解通常面临两大挑战:一是真实标签不足,难以引导算法进行特征选择;二是搜索空间规模大,难以搜索到满意的高质量解。为此,提出了新型的面向大规模特征选择的自监督数据驱动粒子群优化算法。第一,提出了自监督数据驱动特征选择的新型算法框架,可不依赖于真实标签进行特征选择。第二,提出了基于离散区域编码的搜索策略,帮助算法在大规模搜索空间中找到更优解。第三,基于上述的框架和方法,提出了自监督数据驱动粒子群优化算法,实现对问题的求解。在大规模特征数据集上的实验结果显示,提出的算法与主流有监督算法表现相当,并比前沿无监督算法具有更高的特征选择效率。     

Soft spectral clustering ensemble applied to image segmentation

An unsupervised learning algorithm, named soft spectral clustering ensemble (SSCE), is proposed in this paper. Until now many proposed ensemble algorithms cannot be used on image data, even images of a mere 256 × 256 pixels are too expensive in computational cost and storage. The proposed method is suitable for performing image segmentation and can, to some degree, solve some open problems of spectral clustering (SC). In this paper, a random scaling parameter and Nystr?m approximation are applied to generate the individual spectral clusters for ensemble learning. We slightly modify the standard SC algorithm to aquire a soft partition and then map it via a centralized logcontrast transform to relax the constraint of probability data, the sum of which is one. All mapped data are concatenated to form the new features for each instance. Principal component analysis (PCA) is used to reduce the dimension of the new features. The final aggregated result can be achieved by clustering dimension-reduced data. Experimental results, on UCI data and different image types, show that the proposed algorithm is more efficient compared with some existing consensus functions.     

基于可辨识矩阵的完全自适应2D特征选择算法

针对基于信息增益与皮尔森相关系数的特征选择算法FSIP(feature selection based on information gain and Pearson correlation coefficient)存在的特征子集选取需要人工参与的问题,提出基于可辨识矩阵的完全自适应2D特征选择算法DFSIP(disc...