基于概率矩阵分解的不完整数据集特征选择方法

在机器学习理论与应用中,特征选择是降低高维数据特征维度的常用方法之一。传统的特征选择方法多数基于完整数据集,对实际应用中普遍存在缺失数据的情形研究较少。针对不完整数据中含有未被观察信息和存在异常值的特点,提出一种基于概率矩阵分解技术的鲁棒特征选择方法。使用基于分簇的概率矩阵分解模型对数据集中的缺失值进行近似估计,以有效测量相邻簇之间数据的相似性,缩小问题规模,同时降低填充误差。依据缺失数据值存在少量异常值的情形,利用基于l_2,1损失函数的方法进行特征选择,在此基础上给出不完整数据集的特征选择方法流程,并对其收敛性进行理论分析。该方法利用不完整数据集中的所有信息,有效应对不完整数据集中异常值带来的影响。实验结果表明,相比传统特征选择方法,该方法在合成数据集上选择更少的无关特征,可降低异常值带来的影响,在真实数据集上获得了较高的分类准确率,能够选择出更为准确的特征。     

特征选择方法综述

特征选择作为一个数据预处理过程,在数据挖掘、模式识别和机器学习中有着重要地位。通过特征选择,可以降低问题的复杂度,提高学习算法的预测精度、鲁棒性和可解释性。介绍特征选择方法框架,重点描述生成特征子集、评价准则两个过程;根据特征选择和学习算法的不同结合方式对特征选择算法分类,并分析各种方法的优缺点;讨论现有特征选择算法存在的问题,提出一些研究难点和研究方向。     

基于多任务学习的输电线路小金具缺失推理加速算法

针对输电线路小金具缺失的检测问题,对小金具缺失算法的推理加速进行了研究,采用了多任务学习的方法,将小金具缺失检测任务使用一个Swin Transformer _[2^6]主干网络连接和多个MLP任务头的方式进行多任务学习和多任务推理,并进行了单任务学习和多任务学习的精度和性能对比实验,最后还验证了在多任务学习中无缝增加扩展任务,实验结果表明多任务学习的输电线路小金具缺失推理在比单任务学习的推理速度提升了2倍以上,在推理显存占用上降低了22%以上。通过扩展任务实验结果验证了扩展任务的有效性,提高了任务配置的灵活性。     

基于衣物共现信息与多任务学习的衣物识别*

多任务学习(MTL)未考虑先验概率对学习的影响.针对这一问题,文中提出基于衣物共现信息与多任务学习的衣物识别方法(CA-MLT),通过在MTL模型中加入先验约束项整合衣物共现信息,并对传统的扩展梯度算法进行相应的修改,从而提高衣物类别分类器的性能.实验表明,CA-MLT的平均性能优于单一任务学习、神经网络及传统的多任务学习等方法,训练结果便于可视化,可用于特征选择.     

缺失标记下基于类属属性的多标记特征选择

多标记特征选择已在图像分类、疾病诊断等领域得到广泛应用;然而,现实中数据的标记空间往往存在部分标记缺失的问题,这破坏了标记间的结构性和关联性,使得学习算法难以准确地选择重要特征。针对此问题,提出一种缺失标记下基于类属属性的多标记特征选择（MFSLML）算法。首先,通过利用稀疏学习方法获取每个类标记的类属属性;同时基于线性回归模型构建类属属性与标记的映射关系,以用于恢复缺失标记;最后,选取7组数据集以及4个评价指标进行实验。实验结果表明：相比基于最大依赖度和最小冗余度的多标记特征选择算法（MDMR）和基于特征交互的多标记特征选择算法（MFML）等一些先进的多标记特征选择算法,MFSLML在平均查准率指标上能够提升4.61~5.5个百分点,由此可见MFSLML具有更优的分类性能。     

多任务学习的车辆结构化信息提取方法

目前, 大部分的车辆结构化信息需要通过多个步骤进行提取, 存在模型训练繁琐、各步骤模型训练数据有限和过程误差累加等问题. 为此, 采用多任务学习将车辆结构化信息提取整合在统一的神经网络之中, 通过共享特征提取结构, 减少过程误差累加, 并构建了一个多任务损失函数用于端到端训练神经网络; 针对训练样本有限的问题, 提出了新的数据整合和增广方法. 在KITTI数据集上实验结果表明, VSENet可以达到93.82%的mAP(均值平均精度), 且能达到实时的处理速度; 与多阶段的车辆结构化特征提取方法对比, 平均运行时间缩减了60%, 其精度能达到相似或者更好的效果; 实验结果表明, 该方法具有一定的先进性和有效性.     

基于邻域互信息的三支特征选择

特征选择是机器学习非常重要的预处理步骤,而邻域互信息是一种能直接处理连续型或离散型特征的有效方法。然而基于邻域互信息的特征选择方法一般采用启发式贪婪策略,其特征子集质量难以得到有效保证。基于三支决策的思想,提出了三支邻域互信息特征选择方法（NMI-TWD）。通过扩展三个潜在的候选特征子集,并保持各子集之间的差异性,以获得更高质量的特征子集。对三个差异性的特征子集进行集成学习,构建三支协同决策模型,以进一步提高分类学习性能。UCI实验数据表明,新方法的特征选择结果和分类性能较其他方法更优,说明了其有效性。     

基于组合特征选择的随机森林信用评估

构建个人信用风险评估模型的过程中, 特征工程很大程度上决定了评估器的性能, 传统的特征选择方法无法全面的考虑高维度指标对评估结果的影响, 且大多数研究在构建模型的过程中人为决定特征集大小, 导致随机性强、可信度低; 基于此, 提出基于传统风控指标优化XGBoost的随机森林模型(IV-XGBoostRF), 将传统风控指标IV与XGBoost相结合对原始特征集进行筛选, 建立较为完善的信用评估模型. 通过对比实验的结果显示改进后的随机森林模型准确度提高了0.90%, 且其他各项评估指标均优于传统信用评估模型, 证明了该组合特征选择方法的可行性, 有一定的应用价值.     

面向特征选择问题的协同演化方法

特征选择技术是机器学习和数据挖掘任务的关键预处理技术。传统贪婪式特征选择方法仅考虑本轮最佳特征,从而导致获取的特征子集仅为局部最优,无法获得最优或者近似最优的特征集合。进化搜索方式则有效地对特征空间进行搜索,然而不同的进化算法在搜索过程中存在自身的局限。本文吸取遗传算法（GA）和粒子群优化算法（PSO）的进化优势,以信息熵度量为评价,通过协同演化的方式获取最终特征子集。并提出适用于特征选择问题特有的比特率交叉算子和信息交换策略。实验结果显示,遗传算法和粒子群协同进化（GA-PSO）在进化搜索特征子集的能力和具体分类学习任务上都优于单独的演化搜索方式。进化搜索提供的组合判断能力优于贪婪式特征选择方法。     

面向高维数据的个人信贷风险评估方法

随着电商平台分期付款方式和P2P信贷平台的不断推广,如何从海量的用户信贷数据中挖掘出潜在的用户模型并对未知用户进行信贷风险评估,以降低信贷业务的风险,已经成为研究的主流。针对现有方法无法高效处理高维度信贷数据的问题,使用一系列的数据预处理方法和基于Embedded思想的特征选择方法XGBFS（XGBoost Feature Selection）,以降低用户信贷数据维度并训练出XGBoost评估模型,最终实现用户信贷风险评估。实验表明,与现有的方法相比,该方法能够从高维的数据中选择出重要属性,并且分类器在精确率、召回率等方面具有较为突出的性能。     

A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring

Credit scoring is an effective tool for banks to properly guide decision profitably on granting loans. Ensemble methods, which according to their structures can be divided into parallel and sequential ensembles, have been recently developed in the credit scoring domain. These methods have proven their superiority in discriminating borrowers accurately. However, among the ensemble models, little consideration has been provided to the following: (1) highlighting the hyper-parameter tuning of base learner despite being critical to well-performed ensemble models; (2) building sequential models (i.e., boosting, as most have focused on developing the same or different algorithms in parallel); and (3) focusing on the comprehensibility of models. This paper aims to propose a sequential ensemble credit scoring model based on a variant of gradient boosting machine (i.e., extreme gradient boosting (XGBoost)). The model mainly comprises three steps. First, data pre-processing is employed to scale the data and handle missing values. Second, a model-based feature selection system based on the relative feature importance scores is utilized to remove redundant variables. Third, the hyper-parameters of XGBoost are adaptively tuned with Bayesian hyper-parameter optimization and used to train the model with selected feature subset. Several hyper-parameter optimization methods and baseline classifiers are considered as reference points in the experiment. Results demonstrate that Bayesian hyper-parameter optimization performs better than random search, grid search, and manual search. Moreover, the proposed model outperforms baseline models on average over four evaluation measures: accuracy, error rate, the area under the curve (AUC) H measure (AUC-H measure), and Brier score. The proposed model also provides feature importance scores and decision chart, which enhance the interpretability of credit scoring model.     

Unsupervised feature selection via joint local learning and group sparse regression

Feature selection has attracted a great deal of interest over the past decades. By selecting meaningful feature subsets, the performance of learning algorithms can be effectively improved. Because label information is expensive to obtain, unsupervised feature selection methods are more widely used than the supervised ones. The key to unsupervised feature selection is to find features that effectively reflect the underlying data distribution. However, due to the inevitable redundancies and noise in a dataset, the intrinsic data distribution is not best revealed when using all features. To address this issue, we propose a novel unsupervised feature selection algorithm via joint local learning and group sparse regression (JLLGSR). JLLGSR incorporates local learning based clustering with group sparsity regularized regression in a single formulation, and seeks features that respect both the manifold structure and group sparse structure in the data space. An iterative optimization method is developed in which the weights finally converge on the important features and the selected features are able to improve the clustering results. Experiments on multiple real-world datasets (images, voices, and web pages) demonstrate the effectiveness of JLLGSR.

     

A genetic algorithm-based method for feature subset selection

As a commonly used technique in data preprocessing, feature selection selects a subset of informative attributes or variables to build models describing data. By removing redundant and irrelevant or noise features, feature selection can improve the predictive accuracy and the comprehensibility of the predictors or classifiers. Many feature selection algorithms with different selection criteria has been introduced by researchers. However, it is discovered that no single criterion is best for all applications. In this paper, we propose a framework based on a genetic algorithm (GA) for feature subset selection that combines various existing feature selection methods. The advantages of this approach include the ability to accommodate multiple feature selection criteria and find small subsets of features that perform well for a particular inductive learning algorithm of interest to build the classifier. We conducted experiments using three data sets and three existing feature selection methods. The experimental results demonstrate that our approach is a robust and effective approach to find subsets of features with higher classification accuracy and/or smaller size compared to each individual feature selection algorithm.     

多任务实时声音事件检测卷积模型与复合数据扩增

现有的声音事件检测研究多为对离线音频进行分析,且模型参数量较多、计算效率低,不适用于实时检测。提出一种面向多任务实时声音事件检测的轻量化卷积神经网络模型,它将唤醒与检测任务整合成多任务学习框架,此外模型的卷积结构联合了稠密连接、Ghost模组与SE注意力机制;另外还提出了一种复合数据扩增方法,将音频变换、随机裁剪与频谱掩蔽相结合。实验结果显示,该模型在ESC-10和Urbansound8K数据集上的平均预测准确率高于当前新型的基线模型2%以上,同时模型的参数和内存更少。研究表明,多任务学习的方式节省了计算量,又因为卷积结构复用了中间层特征,模型可以快速地反馈检测结果。另外,复合数据方法相比传统方法使模型获得了更好的性能和鲁棒性。     

基于标记与特征依赖最大化的弱标记集成分类

弱标记学习是多标记学习的一个重要分支,近几年已被广泛研究并被应用于多标记样本的缺失标记补全和预测等问题.然而,针对特征集合较大、更容易拥有多个语义标记和出现标记缺失的高维数据问题,现有弱标记学习方法普遍易受这类数据包含的噪声和冗余特征的干扰.为了对高维多标记数据进行准确的分类,提出了一种基于标记与特征依赖最大化的弱标记集成分类方法EnWL.EnWL首先在高维数据的特征空间多次利用近邻传播聚类方法,每次选择聚类中心构成具有代表性的特征子集,降低噪声和冗余特征的干扰;再在每个特征子集上训练一个基于标记与特征依赖最大化的半监督多标记分类器;最后,通过投票集成这些分类器实现多标记分类.在多种高维数据集上的实验结果表明,EnWL在多种评价度量上的预测性能均优于已有相关方法.     

基于粗糙集和改进鲸鱼优化算法的特征选择方法

随着互联网和物联网技术的发展,数据的收集变得越发容易。但是,高维数据中包含了很多冗余和不相关的特征,直接使用会徒增模型的计算量,甚至会降低模型的表现性能,故很有必要对高维数据进行降维处理。特征选择可以通过减少特征维度来降低计算开销和去除冗余特征,以提高机器学习模型的性能,并保留了数据的原始特征,具有良好的可解释性。特征选择已经成为机器学习领域中重要的数据预处理步骤之一。粗糙集理论是一种可用于特征选择的有效方法,它可以通过去除冗余信息来保留原始特征的特性。然而,由于计算所有的特征子集组合的开销较大,传统的基于粗糙集的特征选择方法很难找到全局最优的特征子集。针对上述问题,文中提出了一种基于粗糙集和改进鲸鱼优化算法的特征选择方法。为避免鲸鱼算法陷入局部优化,文中提出了种群优化和扰动策略的改进鲸鱼算法。该算法首先随机初始化一系列特征子集,然后用基于粗糙集属性依赖度的目标函数来评价各子集的优劣,最后使用改进鲸鱼优化算法,通过不断迭代找到可接受的近似最优特征子集。在UCI数据集上的实验结果表明,当以支持向量机为评价所用的分类器时,文中提出的算法能找到具有较少信息损失的特征子集,且具有较高的分类精度。因此,所提算法在特征选择方面具有一定的优势。     

Using Rough Sets with Heuristics for Feature Selection

Practical machine learning algorithms are known to degrade in performance (prediction accuracy) when faced with many features (sometimes attribute is used instead of feature) that are not necessary for rule discovery. To cope with this problem, many methods for selecting a subset of features have been proposed. Among such methods, the filter approach that selects a feature subset using a preprocessing step, and the wrapper approach that selects an optimal feature subset from the space of possible subsets of features using the induction algorithm itself as a part of the evaluation function, are two typical ones. Although the filter approach is a faster one, it has some blindness and the performance of induction is not considered. On the other hand, the optimal feature subsets can be obtained by using the wrapper approach, but it is not easy to use because of the complexity of time and space. In this paper, we propose an algorithm which is using rough set theory with greedy heuristics for feature selection. Selecting features is similar to the filter approach, but the evaluation criterion is related to the performance of induction. That is, we select the features that do not damage the performance of induction.     

基于多目标优化多任务学习的端到端车牌识别方法

本文针对多个车牌识别任务之间存在竞争和冲突,导致难以同时提高多个车牌的识别率的问题,提出基于多目标优化多任务学习的端到端车牌识别方法.首先,通过分析某些车牌识别任务容易占主导地位,而其他任务无法得到充分优化的问题,建立基于多任务学习的车牌识别模型.接着,针对字符分割造成车牌识别准确率较低、鲁棒性较差的问题,提出基于多任务学习的端到端车牌识别方法.最后,针对多个车牌识别任务间难以权衡的问题,提出一种基于多目标优化的多任务学习方法,以提高多个车牌识别的准确率.将本文所提方法在标准车牌数据集上进行测试,实验结果验证了该方法的有效性和优越性,其他代表性方法相比可以提高车牌识别的准确率、快速性和鲁棒性.     

Learning with tensors: a framework based on convex optimization and spectral regularization

We present a framework based on convex optimization and spectral regularization to perform learning when feature observations are multidimensional arrays (tensors). We give a mathematical characterization of spectral penalties for tensors and analyze a unifying class of convex optimization problems for which we present a provably convergent and scalable template algorithm. We then specialize this class of problems to perform learning both in a transductive as well as in an inductive setting. In the transductive case one has an input data tensor with missing features and, possibly, a partially observed matrix of labels. The goal is to both infer the missing input features as well as predict the missing labels. For induction, the goal is to determine a model for each learning task to be used for out of sample prediction. Each training pair consists of a multidimensional array and a set of labels each of which corresponding to related but distinct tasks. In either case the proposed technique exploits precise low multilinear rank assumptions over unknown multidimensional arrays; regularization is based on composite spectral penalties and connects to the concept of Multilinear Singular Value Decomposition. As a by-product of using a tensor-based formalism, our approach allows one to tackle the multi-task case in a natural way. Empirical studies demonstrate the merits of the proposed methods.     

混合式的K-匿名特征选择算法

K-匿名算法通过对数据的泛化、隐藏等手段使得数据达到K-匿名条件,在隐藏特征的同时考虑数据的隐私性与分类性能,可以视为一种特殊的特征选择方法,即K-匿名特征选择。K-匿名特征选择方法结合K-匿名与特征选择的特点使用多个评价准则选出K-匿名特征子集。过滤式K-匿名特征选择方法难以搜索到所有满足K-匿名条件的候选特征子集,不能保证得到的特征子集的分类性能最优,而封装式特征选择方法计算成本很大,因此,结合过滤式特征排序与封装式特征选择的特点,改进已有方法中的前向搜索策略,设计了一种混合式K-匿名特征选择算法,使用分类性能作为评价准则选出分类性能最好的K-匿名特征子集。在多个公开数据集上进行实验,结果表明,所提算法在分类性能上可以超过现有算法并且信息损失更小。