A user parameter-free approach for mining robust sequential classification rules

Recent years have witnessed extensive studies of graph classification due to the rapid increase in applications involving structural data and complex relationships. To support graph classification, all existing methods require that training graphs should be relevant (or belong) to the target class, but cannot integrate graphs irrelevant to the class of interest into the learning process. In this paper, we study a new universum graph classification framework which leverages additional “non-example” graphs to help improve the graph classification accuracy. We argue that although universum graphs do not belong to the target class, they may contain meaningful structure patterns to help enrich the feature space for graph representation and classification. To support universum graph classification, we propose a mathematical programming algorithm, ugBoost, which integrates discriminative subgraph selection and margin maximization into a unified framework to fully exploit the universum. Because informative subgraph exploration in a universum setting requires the search of a large space, we derive an upper bound discriminative score for each subgraph and employ a branch-and-bound scheme to prune the search space. By using the explored subgraphs, our graph classification model intends to maximize the margin between positive and negative graphs and minimize the loss on the universum graph examples simultaneously. The subgraph exploration and the learning are integrated and performed iteratively so that each can be beneficial to the other. Experimental results and comparisons on real-world dataset demonstrate the performance of our algorithm.     

Stacking model of multi-label classification based on pruning strategies

Exploiting dependencies between the labels is the key of improving the performance of multi-label classification. In this paper, we divide the utilizing methods of label dependence into two groups from the perspective of different ways of problem transformation: label grouping method and feature space extending method. As to the feature space extending method, we find that the common problem is how to measure the dependencies between labels and to select proper labels to add to the original feature space. Therefore, we propose a ReliefF-based pruning model for multi-label classification (ReliefF-based stacking, RFS). RFS measures the dependencies between labels in a feature selection perspective and then selects the more relative labels into the original feature space. Experimental results of 9 multi-label benchmark datasets shows that RFS is more effective compared to other advanced multi-label classification algorithms.

     

融合BERT和图注意力网络的多标签文本分类

多标签文本分类问题是多标签分类的重要分支之一, 现有的方法往往忽视了标签之间的关系, 难以有效利用标签之间存在着的相关性, 从而影响分类效果. 基于此, 本文提出一种融合BERT和图注意力网络的模型HBGA (hybrid BERT and graph attention): 首先, 利用BERT获得输入文本的上下文向量表示, 然后用Bi-LSTM和胶囊网络分别提取文本全局特征和局部特征, 通过特征融合方法构建文本特征向量, 同时, 通过图来建模标签之间的相关性, 用图中的节点表示标签的词嵌入, 通过图注意力网络将这些标签向量映射到一组相互依赖的分类器中, 最后, 将分类器应用到特征提取模块获得的文本特征进行端到端的训练, 综合分类器和特征信息得到最终的预测结果. 在Reuters-21578和AAPD两个数据集上面进行了对比实验, 实验结果表明, 本文模型在多标签文本分类任务上得到了有效的提升.     

gBoost: a mathematical programming approach to graph classification and regression

Graph mining methods enumerate frequently appearing subgraph patterns, which can be used as features for subsequent classification or regression. However, frequent patterns are not necessarily informative for the given learning problem. We propose a mathematical programming boosting method (gBoost) that progressively collects informative patterns. Compared to AdaBoost, gBoost can build the prediction rule with fewer iterations. To apply the boosting method to graph data, a branch-and-bound pattern search algorithm is developed based on the DFS code tree. The constructed search space is reused in later iterations to minimize the computation time. Our method can learn more efficiently than the simpler method based on frequent substructure mining, because the output labels are used as an extra information source for pruning the search space. Furthermore, by engineering the mathematical program, a wide range of machine learning problems can be solved without modifying the pattern search algorithm.     

融合标记独有属性特征的k近邻多标记分类新算法

在多标记学习系统中,每个样本同时与多个类别标记相关,却均由一个属性特征向量描述。大部分已有的多标记分类算法采用的共同策略是使用相同的属性特征集合预测所有的类别标记,但它并非最佳选择,原因在于每个标记可能与其自身独有的属性特征相关性最大。针对这一问题,提出了融合标记独有属性特征的k近邻多标记分类算法—IML-kNN。首先对多标记数据的特征向量进行预处理,分别为每类标记构造对该类标记最具有判别能力的属性特征;然后基于得到的属性特征使用改进后的ML-kNN算法进行分类。实验结果表明,IML-kNN算法在yeast和image数据集上的性能明显优于ML-kNN算法以及其他3种常用的多标记分类算法。     

基于局部子空间的多标记特征选择算法*

在已有的特征选择算法中,常用策略是通过相关准则选择与标记集合相关性较强的特征,然而该策略不一定是最优选择,因为与标记集合相关性较弱的特征可能是决定某些类别标记的关键特征.基于这一假设,文中提出基于局部子空间的多标记特征选择算法.该算法首先利用特征与标记集合之间的互信息得到一个重要度由高到低的特征序列,然后将新的特征排序空间划分为几个局部子空间,并在每个子空间设置采样比例以选择冗余性较小的特征,最后融合各子空间的特征子集,得到一组合理的特征子集.在6个数据集和4个评价指标上的实验表明,文中算法优于一些通用的多标记特征选择算法.     

多源适应多标签分类框架

多源适应学习是一种旨在提升目标学习性能的有效机器学习方法。针对多标签视觉分类问题,基于现有的研究进展,研究提出一种新颖的联合特征选择和共享特征子空间学习的多源适应多标签分类框架,在现有的图Laplacian正则化半监督学习范式中充分考虑目标视觉特征的优化处理,多标签相关信息在共享特征子空间的嵌入,以及多个相关领域的判别信息桥接利用等多个方面,并将其融为一个统一的学习模型,理论证明了其局部最优解只需通过求解一个广义特征分解问题便可分别获得,并给出了算法实现及其收敛性定理。在两个实际的多标签视觉数据分类上分别进行深入实验分析,证实了所提框架的鲁棒有效性和优于现有相关方法的分类性能。     

基于标记权重的多标记特征选择算法

在多标记学习中,特征选择是解决多标记数据高维性的有效手段。每个标记对样本的可分性程度不同,这可能会为多标记学习提供一定的信息。基于这一假设,提出了一种基于标记权重的多标记特征选择算法。该算法首先利用样本在整个特征空间的分类间隔对标记进行加权,然后将特征在整个标记集合下对样本的可区分性作为特征权重,以此衡量特征对标记集合的重要性。最后,根据特征权重对特征进行降序排列,从而得到一组新的特征排序。在6个多标记数据集和4个评价指标上的实验结果表明,所提算法优于一些当前流行的多标记特征选择算法。     

Multi-label classification via learning a unified object-label graph with sparse representation

Automatic annotation is an essential technique for effectively handling and organizing Web objects (e.g., Web pages), which have experienced an unprecedented growth over the last few years. Automatic annotation is usually formulated as a multi-label classification problem. Unfortunately, labeled data are often time-consuming and expensive to obtain. Web data also accommodate much richer feature space. This calls for new semi-supervised approaches that are less demanding on labeled data to be effective in classification. In this paper, we propose a graph-based semi-supervised learning approach that leverages random walks and ? ₁ sparse reconstruction on a mixed object-label graph with both attribute and structure information for effective multi-label classification. The mixed graph contains an object-affinity subgraph, a label-correlation subgraph, and object-label edges with adaptive weight assignments indicating the assignment relationships. The object-affinity subgraph is constructed using ? ₁ sparse graph reconstruction with extracted structural meta-text, while the label-correlation subgraph captures pairwise correlations among labels via linear combination of their co-occurrence similarity and kernel-based similarity. A random walk with adaptive weight assignment is then performed on the constructed mixed graph to infer probabilistic assignment relationships between labels and objects. Extensive experiments on real Yahoo! Web datasets demonstrate the effectiveness of our approach.     

一种基于标签相关性的类属属性多标签分类算法

多标签学习中一个样本可同时属于多个类别标签,每个标签都可能拥有反映该标签特定特点的特征,即类属属性,目前已经出现了基于类属属性的多标签分类算法LIFT。针对LIFT算法中未考虑标签之间相互关系的问题,提出一种基于标签相关性的类属属性多标签分类算法CLLIFT。该算法使用标签距离度量标签之间的相关性,通过在类属属性空间附加相关标签的方式完成标签相关性的引入,以达到提升分类性能的目的。在四个多标签数据集上的实验结果表明,所提算法与LIFT算法相比在多个多标签评价指标上平均提升21.1%。     

基于文本分类的Fisher Score快速多标记特征选择算法

Fisher Score （FS）是一种快速高效的评价特征分类能力的指标,但传统的FS指标既无法直接应用于多标记学习,也不能有效处理样本极值导致的类中心与实际类中心的误差。提出一种结合中心偏移和多标记集合关联性的FS多标记特征选择算法,找出不同标记下每类样本的极值点,以极值点到该类样本的中心距离乘以半径系数筛选新的样本,从而获得分布更为密集的样本集合,以此计算特征的FS得分,通过整体遍历全体样本的标记集合中的每个标记,并在遍历过程中针对具有更多标记数量的样本自适应地赋以标记权值,得到整体特征的平均FS得分,以特征的FS得分进行排序过滤出目标子集实现特征选择目标。在8个公开的多标记文本数据集上进行参数分析及5种指标性能比较,结果表明,该算法具有一定的有效性和鲁棒性,在多数指标上优于MLNB、MLRF、PMU、MLACO等多标记特征选择算法。     

Information-theoretic selection of high-dimensional spectral features for structural recognition

Pattern recognition methods often deal with samples consisting of thousands of features. Therefore, the reduction of their dimensionality becomes crucial to make the data sets tractable. Feature selection techniques remove the irrelevant and noisy features and select a subset of features which describe better the samples and produce a better classification performance. In this paper, we propose a novel feature selection method for supervised classification within an information-theoretic framework. Mutual information is exploited for measuring the statistical relation between a subset of features and the class labels of the samples. Traditionally it has been measured for ranking single features; however, in most data sets the features are not independent and their combination provides much more information about the class than the sum of their individual prediction power. We analyze the use of different estimation methods which bypass the density estimation and estimate entropy and mutual information directly from the set of samples. These methods allow us to efficiently evaluate multivariate sets of thousands of features.Within this framework we experiment with spectral graph features extracted from 3D shapes. Most of the existing graph classification techniques rely on the graph attributes. We use unattributed graphs to show what is the contribution of each spectral feature to graph classification. Apart from succeeding to classify graphs from shapes relying only on their structure, we test to what extent the set of selected spectral features are robust to perturbations of the dataset.     

多标签特征选择研究进展

特征选择一直是机器学习和数据挖掘中的一个重要问题。在多标签学习任务中,数据集中的每个样本都与多个标签相关联,标签与标签之间通常也是相关的。在多标签高维数据分析中,为降低特征维数和提高分类性能,研究者们提出了多标签特征选择方法。系统综述了多标签特征选择的研究进展。在介绍多标签分类以及评价准则之后,详细分析了多标签特征选择的三类方法,即过滤式算法、包裹式算法和嵌入式算法,对多标签特征选择未来的研究提出展望。     

基于指数损失间隔的多标记特征选择算法

在多标记学习的任务中,多标记学习的每个样本可被多个标签标记,比单标记学习的应用空间更广关注度更高,多标记学习可以利用关联性提高算法的性能。在多标记学习中,传统特征选择算法已不再适用,一方面,传统的特征选择算法可被用于单标记的评估标准。多标记学习使得多个标记被同时优化;而且在多标记学习中关联信息存在于不同标记间。因此,可设计一种能够处理多标记问题的特征选择算法,使标记之间的关联信息能够被提取和利用。通过设计最优的目标损失函数,提出了基于指数损失间隔的多标记特征选择算法。该算法可以通过样本相似性的方法,将特征空间和标记空间的信息融合在一起,独立于特定的分类算法或转换策略。优于其他特征选择算法的分类性能。在现实世界的数据集上验证了所提算法的正确性以及较好的性能。     

A novel classifier for multivariate instance using graph class signatures

Applications like identifying different customers from their unique buying behaviours, determining ratingsof a product given by users based on different sets of features, etc. require classification using class-specific subsets of features. Most of the existing state-of-the-art classifiers for multivariate data use complete feature set for classification regardless of the different class labels. Decision tree classifier can produce class-wise subsets of features. However, none of these classifiers model the relationship between features which may enhance classification accuracy. We call the class-specific subsets of features and the features’ interrelationships as class signatures. In this work, we propose to map the original input space of multivariate data to the feature space characterized by connected graphs as graphs can easily model entities, their attributes, and relationships among attributes. Mostly, entities are modeled using graphs, where graphs occur naturally, for example, chemical compounds. However, graphs do not occur naturally in multivariate data. Thus, extracting class signatures from multivariate data is a challenging task. We propose some feature selection heuristics to obtain class-specific prominent subgraph signatures. We also propose two variants of class signatures based classifier namely: 1) maximum matching signature (gMM), and 2) score and size of matched signatures (gSM). The effectiveness of the proposed approach on real-world and synthetic datasets has been studied and compared with other established classifiers. Experimental results confirm the ascendancy of the proposed class signatures based classifier on most of the datasets.     

缺失标记下基于类属属性的多标记特征选择

多标记特征选择已在图像分类、疾病诊断等领域得到广泛应用;然而,现实中数据的标记空间往往存在部分标记缺失的问题,这破坏了标记间的结构性和关联性,使得学习算法难以准确地选择重要特征。针对此问题,提出一种缺失标记下基于类属属性的多标记特征选择（MFSLML）算法。首先,通过利用稀疏学习方法获取每个类标记的类属属性;同时基于线性回归模型构建类属属性与标记的映射关系,以用于恢复缺失标记;最后,选取7组数据集以及4个评价指标进行实验。实验结果表明：相比基于最大依赖度和最小冗余度的多标记特征选择算法（MDMR）和基于特征交互的多标记特征选择算法（MFML）等一些先进的多标记特征选择算法,MFSLML在平均查准率指标上能够提升4.61~5.5个百分点,由此可见MFSLML具有更优的分类性能。     

结合标记相关性的特征选择算法

在多标记分类问题中,每个样本可以同时与多个标记类别相关,其中一些标记之间可能具有相关性,充分利用这些标记相关性,可优化分类性能.因此,文中利用标记的频繁项集对标记相关性进行挖掘,提出针对基于邻域粗糙集的多标记属性约简算法进行改进的特征选择算法,并进一步将训练样本根据特征之间的相似性进行聚类,结合局部样本上的标记相关性,进行属性约简及分类.在5个多标记分类数据集上的实验验证文中算法的有效性.     

基于自编码器及超图学习的多标签特征提取

在实际应用场景中越来越多的数据具有多标签的特性,且特征维度较高,包含大量冗余信息.为提高多标签数据挖掘的效率,多标签特征提取已经成为当前研究的热点.本文采用去噪自编码器获取多标签数据特征空间的鲁棒表达,在此基础上结合超图学习理论,融合多个标签对样本间几何关系的影响以提升特征提取的性能,构建多标签数据样本间几何关系所对应超图的Laplacian矩阵,并通过Laplacian矩阵的特征值分解得到低维投影空间.实验结果证明了本文所提出的算法在分类性能上是有效可行的.     

一种面向图包容搜索的图索引模型

由于图模型的广泛采用,图数据的快速包容搜索在许多不同领域广泛应用。给定一个模型图集D和一个查询图集q,传统的图搜索旨在检索所有包含q的图(qg),与此不同,包容搜索有其自身的索引特性,针对这些特性进行系统地研究,并提出一种基于对比子图的索引模型（csgIndex）：使用一个冗余感知特征选择过程,csgIndex能挑选出一个鲜明的具有区分力的对比子图集,并最大化其索引能力。对实时测试数据的实验结果显示,csgIndex对不同的包容搜索荷载能达到近优化修剪力,相较于传统图搜索方法表现出明显的索引性能优势。     

Efficient subgraph matching using topological node feature constraints

This paper presents techniques designed to minimise the number of states which are explored during subgraph isomorphism detection. A set of advanced topological node features, calculated from n-neighbourhood graphs, is presented and shown to outperform existing features. Further, the pruning effectiveness of both the new and existing topological node features is significantly improved through the introduction of strengthening techniques. In addition to topological node features, these strengthening techniques can also be used to enhance application-specific node labels using a proposed novel extension to existing pruning algorithms. Through the combination of these techniques, the number of explored search states can be reduced to near-optimal levels.