基于概率密度距离的监督特征选择

为了降低特征维数,提高分类效率,提出了一种新的基于概率密度距离的有监督的特征排序方法.首先依次对所有样本的某一维特征进行加权变换,然后对变换后的各类别样本进行概率密度估计,计算由该特征加权变换后所引起的各类别样本的类间概率密度距离,距离越大,则该特征对于区分各类别样本的作用越大,以此来对特征进行排序.实验结果表明,该方法是有效的,而且表现出了比经典的Relief-F特征排序方法更好的性能.     

基于最小分类错误率和Parzen窗的降维方法

提出了一种基于最小分类错误率和Parzen窗的降维方法,利用Parzen窗估计数据的概率密度分布;通过计算各特征维度下的分类错误率,判断该特征维度对目标分类的贡献度;依据贡献度大小进行特征维度选择从而达到降维的目的。     

基于概率密度逼近的无监督特征排序*

依据概率密度逼近提出了一种新的无监督特征排序,应用于特征选择降维.实验证明,这种方法与一些现有的方法相比,更为有效.     

特征降维技术的研究与进展

数据特征的质量会直接影响模型的准确度。在模式识别领域,特征降维技术一直受到研究者们的关注。随着大数据时代的到来,数据量巨增,数据维度不断升高。在处理高维数据时,传统的数据挖掘方法的性能降低甚至失效。实践表明,在数据分析前先对其特征进行降维是避免“维数灾难”的有效手段。降维技术在各领域被广泛应用,文中详细介绍了特征提取和特征选择两类不同的降维方法,并对其特点进行了比较。通过子集搜索策略和评价准则两个关键过程对特征选择中最具代表性的算法进行了总结和分析。最后从实际应用出发,探讨了特征降维技术值得关注的研究方向。     

质谱数据的特征降维新方法

由于质谱数据的维数较高,处理时运算量也特别大,同时还降低分类精度.研究者提出一些降维方法,产生了较好的效果.一般降维方法分两类:特征提取和特征选择.两类方法各有优劣,本文提出对特征提取的主成分进行特征选择,提出了将几种特征提取方法与特征选择方法结合的框架,来对数据集进行维数约简.在三个质谱数据集上的实验结果证明新提出的框架对于质谱数据有好的效果,加入特征选择后,建模精度得到了提高.     

高维数据特征降维研究综述

特征降维能够有效地提高机器学习的效率,特征子集的搜索过程以及特征评价标准是特征降维的两个核心问题。综述国际上关于特征降维的研究成果,总结并提出了较完备的特征降维模型定义;通过列举解决特征降维上重要问题的各种方案来比较各种算法的特点以及优劣,并讨论了该方向上尚未解决的问题和发展趋势。     

基于覆盖算法的垃圾邮件过滤

电子邮件系统分类的正确性与风险性是评价邮件系统好坏的关键因素,邮件过滤是文本分类问题的一种特殊应用.将神经网络中的覆盖算法引入到邮件过滤中,结合多种特征降维方法进行邮件分类实验,并与SVM方法进行了比较.给出一个结合覆盖算法、合适的特征选择与降维方法的分类器,可以实现较好的效果.另外,根据垃圾邮件过滤在实际使用中的最小风险性的要求,从风险角度分析了覆盖算法对测试样本进行分类时的过程.根据分析结果提出对其拒识样本的处理过程进行改进,通过改变非垃圾邮件所属覆盖的影响范围降低了垃圾邮件过滤时的风险.     

基于类别选择的改进KNN文本分类

特征高维性以及算法的泛化能力影响了KNN分类器的分类性能.提出了一种降维条件下基于类别的KNN改进模型,解决了k近邻选择时大类别、高密度样本占优问题.首先使用一种改进的优势率方法进行特征选择,随后使用类别向量对文本类别进行初步判定,最后在压缩后的样本集上使用KNN分类器进行分类.试验结果表明,提出的改进分类模型提高了分类效率.     

统计模式识别中的维数削减与低损降维

较为全面地回顾了统计模式识别中常用的一些特征选择、特征提取等主流特征降维方法,介绍了它们各自的特点及其适用范围,在此基础上,提出了一种新的基于最优分类器——贝叶斯分类器的可用于自动文本分类及其它大样本模式分类的特征选择方法——低损降维.在标准数据集Reuters-21578上进行的仿真实验结果表明,与互信息、χ^2统计量以及文档频率这三种主流文本特征选择方法相比,低损降维的降维效果与互信息、χ^2统计量相当,而优于文档频率.     

文本挖掘中特征降维方法比较研究

研究了文本挖掘中的高维特征选取问题,对常见的降维主要方法:特征选择和特征抽取等算法进行了理论分析与性能比较,评价了它们的优缺点和适用范围。     

Clustering biomedical and gene expression datasets with kernel density and unique neighborhood set based vein detection

It is a crucial need for a clustering technique to produce high-quality clusters from biomedical and gene expression datasets without requiring any user inputs. Therefore, in this paper we present a clustering technique called KUVClust that produces high-quality clusters when applied on biomedical and gene expression datasets without requiring any user inputs. The KUVClust algorithm uses three concepts namely multivariate kernel density estimation, unique closest neighborhood set and vein-based clustering. Although these concepts are known in the literature, KUVClust combines the concepts in a novel manner to achieve high-quality clustering results. The performance of KUVClust is compared with established clustering techniques on real-world biomedical and gene expression datasets. The comparisons were evaluated in terms of three criteria (purity, entropy, and sum of squared error (SSE)). Experimental results demonstrated the superiority of the proposed technique over the existing techniques for clustering both the low dimensional biomedical and high dimensional gene expressions datasets used in the experiments.     

基于多源的跨领域数据分类快速新算法

研究跨领域学习与分类是为了将对多源域的有监督学习结果有效地迁移至目标域,实现对目标域的无标记分 类. 当前的跨领域学习一般侧重于对单一源域到目标域的学习,且样本规模普遍较小,此类方法领域自适应性较差,面对 大样本数据更显得无能为力,从而直接影响跨域学习的分类精度与效率. 为了尽可能多地利用相关领域的有用数据,本文 提出了一种多源跨领域分类算法（Multiple sources cross-domain classification,MSCC）,该算法依据被众多实验证明有效的罗杰斯特回归模型与一致性方法构建多个源域分类器并综合指导目标域的数据分类. 为了充分高效利用大样本的 源域数据,满足大样本的快速运算,在MSCC的基础上,本文结合最新的CDdual （Dual coordinate descent method）算 法,提出了算法MSCC的快速算法MSCC-CDdual,并进行了相关的理论分析. 人工数据集、文本数据集与图像数据集的实 验运行结果表明,该算法对于大样本数据集有着较高的分类精度、快速的运行速度和较高的领域自适应性. 本文的主要贡 献体现在三个方面:1）针对多源跨领域分类提出了一种新的一致性方法,该方法有利于将MSCC算法发展为MSCC-CDdual快速算法;2）提出了MSCC-CDdual快速算法,该算法既适用于样本较少的数据集又适用于大样本数据集;3） MSCC-CDdual 算法在高维数据集上相比其他算法展现了其独特的优势.     

高维Turnstile型数据流聚类算法

现有数据流聚类算法只能处理Time Series和Cash Register型数据流，并且应用于高维数据流时其精度不甚理想。提出针对高维Turnstile型数据流的子空间聚类算法HT-Stream，算法对数据空间进行网格划分，在线动态维护网格单元信息，采用倾斜时间窗口存储统计信息，根据用户指定时间跨度离线输出聚类结果。基于真实数据集与仿真数据集的实验表明，算法具有良好的适用性和有效性。     

Visualizing High‐Dimensional Structures by Dimension Ordering and Filtering using Subspace Analysis

High‐dimensional data visualization is receiving increasing interest because of the growing abundance of high‐dimensional datasets. To understand such datasets, visualization of the structures present in the data, such as clusters, can be an invaluable tool. Structures may be present in the full high‐dimensional space, as well as in its subspaces. Two widely used methods to visualize high‐dimensional data are the scatter plot matrix (SPM) and the parallel coordinate plot (PCP). SPM allows a quick overview of the structures present in pairwise combinations of dimensions. On the other hand, PCP has the potential to visualize not only bi‐dimensional structures but also higher dimensional ones. A problem with SPM is that it suffers from crowding and clutter which makes interpretation hard. Approaches to reduce clutter are available in the literature, based on changing the order of the dimensions. However, usually this reordering has a high computational complexity. For effective visualization of high‐dimensional structures, also PCP requires a proper ordering of the dimensions. In this paper, we propose methods for reordering dimensions in PCP in such a way that high‐dimensional structures (if present) become easier to perceive. We also present a method for dimension reordering in SPM which yields results that are comparable to those of existing approaches, but at a much lower computational cost. Our approach is based on finding relevant subspaces for clustering using a quality criterion and cluster information. The quality computation and cluster detection are done in image space, using connected morphological operators. We demonstrate the potential of our approach for synthetic and astronomical datasets, and show that our method compares favorably with a number of existing approaches.     

Feature subset selection Filter–Wrapper based on low quality data

Today, feature selection is an active research in machine learning. The main idea of feature selection is to choose a subset of available features, by eliminating features with little or no predictive information, as well as redundant features that are strongly correlated. There are a lot of approaches for feature selection, but most of them can only work with crisp data. Until now there have not been many different approaches which can directly work with both crisp and low quality (imprecise and uncertain) data. That is why, we propose a new method of feature selection which can handle both crisp and low quality data. The proposed approach is based on a Fuzzy Random Forest and it integrates filter and wrapper methods into a sequential search procedure with improved classification accuracy of the features selected. This approach consists of the following main steps: (1) scaling and discretization process of the feature set; and feature pre-selection using the discretization process (filter); (2) ranking process of the feature pre-selection using the Fuzzy Decision Trees of a Fuzzy Random Forest ensemble; and (3) wrapper feature selection using a Fuzzy Random Forest ensemble based on cross-validation. The efficiency and effectiveness of this approach is proved through several experiments using both high dimensional and low quality datasets. The approach shows a good performance (not only classification accuracy, but also with respect to the number of features selected) and good behavior both with high dimensional datasets (microarray datasets) and with low quality datasets.     

高维数据流聚类及其演化分析研究

基于数据流数据的聚类分析算法已成为研究的热点．提出一种基于子空间的高维数据流聚类及演化分析算法CAStream，该算法对数据空间进行网格化，采用近似的方法记录网格单元的统计信息，并将潜在密集网格单元快照以改进的金字塔时间结构进行存储，最后采用深度优先搜索方法进行聚类及其演化分析．CAStream能够有效处理高雏数据流，并能发现任意形状分布的聚类．基于真实数据集与仿真数据集的实验表明，算法具有良好的适用性和有效性．     

高维数据流子空间聚类发现及维护算法

近年来由于数据流应用的大量涌现,基于数据流模型的数据挖掘算法研究已成为重要的应用前沿课题.提出一种基于Hoeffding界的高维数据流的子空间聚类发现及维护算法--SHStream.算法将数据流分段（分段长度由Hoeffding界确定）,在数据分段上进行子空间聚类,通过迭代逐步得到满足聚类精度要求的聚类结果,同时针对数据流的动态性,算法对聚类结果进行调整和维护.算法可以有效地处理高雏数据流和对任意形状分布数据的聚类问题.基于真实数据集与仿真数据集的实验表明,算法具有良好的适用性和有效性.     

一种基于几何概率的聚类有效性函数

聚类有效性是聚类分析中尚未解决的基本问题,最佳聚类数的确定是聚类有效性问题中的主要研究内容。以几何概率为理论依据,针对2维数据集提出了一种新的聚类有效性函数,用于确定最佳聚类数。该函数利用2维数据集与2维离散点集之间存在的对应关系,以2维离散点集在特征空间中的分布特征为依据,测度对应数据集的聚类结构,思路直观、容易理解。测度过程中,将点集中的点两两相连生成一个线段集合保存点集的结构信息,通过比较线段集合中线段方向取值与完全随机条件下线段方向取值的相对大小,构造聚类有效性函数。实验结果表明,针对给定的样本数据集,生成该函数的曲线,再根据曲线的形态能够有效地确定2维数据集的最佳聚类数,指导聚类算法设计。     

基于网格和最近邻居的聚类算法

针对目前已有的聚类算法不能很好地处理包含不同密度的簇数据,或者不能很好地区分相邻的密度相差不大的簇的问题,提出1种新的基于严格最近邻居和共享最近邻居的聚类算法．通过构造共享严格最近邻图,使样本点在密度一致的区域保持连接,而在密度不同的相邻区域断开连接,并尽可能去除噪声点和孤立点．该算法可以处理包含有不同密度的簇数据,而且在处理高维数据时具有较低的时间复杂度、实验结果证明,该算法能有效找出不同大小、形状和密度的聚类．     

Perturbation scheme for online learning of features: Incremental principal component analysis

A new incremental online feature extraction approach is proposed based on principal component analysis in conjunction with perturbation theory which for its validity requires the perturbation parameter to be small. Our approach is found to be computationally more efficient in comparison to batch method which for its applicability requires simultaneous availability of all observations for computation of features. It is found on the basis of numerical experiments that the results based on our approach besides being in good agreement with the batch method and other incremental methods are also computationally more efficient. To demonstrate the efficacy of the proposed scheme, experiments have been performed on randomly generated datasets as well as on low and high dimensional datasets, i.e. UCI and face datasets which are available in public domain.