一种半监督对抗鲁棒模型无关元学习方法

元学习期望训练所得的元模型在学习到的“元知识”基础上利用来自新任务的少量标注样本，仅通过较少的梯度下降步骤微调模型就能够快速适应该任务。但是，由于缺乏训练样本，元学习算法在元训练期间对现有任务过度训练时所得的分类器决策边界不够准确，不合理的决策边界使得元模型更容易受到微小对抗扰动的影响，导致元模型在新任务上的鲁棒性能降低。提出一种半监督对抗鲁棒模型无关元学习（semi-ARMAML）方法，在目标函数中分别引入半监督的对抗鲁棒正则项和基于信息熵的任务无偏正则项，以此优化决策边界，其中对抗鲁棒正则项的计算允许未标注样本包含未见过类样本，从而使得元模型能更好地适应真实应用场景，降低对输入扰动的敏感性，提高对抗鲁棒性。实验结果表明，相比ADML、R-MAML-TRADES等当下主流的对抗元学习方法，semi-ARMAML方法在干净样本上准确率较高，在MiniImageNet数据集的5-way 1-shot与5-way 5-shot任务上对抗鲁棒性能分别约提升1.8%和2.7%，在CIFAR-FS数据集上分别约提升5.2%和8.1%。     

BaMBNet: A Blur-Aware Multi-Branch Network for Dual-Pixel Defocus Deblurring

Reducing the defocus blur that arises from the finite aperture size and short exposure time is an essential problem in computational photography. It is very challenging because the blur kernel is spatially varying and difficult to estimate by traditional methods. Due to its great breakthrough in low-level tasks, convolutional neural networks (CNNs) have been introduced to the defocus deblurring problem and achieved significant progress. However, previous methods apply the same learned kernel for different regions of the defocus blurred images, thus it is difficult to handle nonuniform blurred images. To this end, this study designs a novel blur-aware multi-branch network (BaMBNet), in which different regions are treated differentially. In particular, we estimate the blur amounts of different regions by the internal geometric constraint of the dual-pixel (DP) data, which measures the defocus disparity between the left and right views. Based on the assumption that different image regions with different blur amounts have different deblurring difficulties, we leverage different networks with different capacities to treat different image regions. Moreover, we introduce a meta-learning defocus mask generation algorithm to assign each pixel to a proper branch. In this way, we can expect to maintain the information of the clear regions well while recovering the missing details of the blurred regions. Both quantitative and qualitative experiments demonstrate that our BaMBNet outperforms the state-of-the-art (SOTA) methods. For the dual-pixel defocus deblurring (DPD)-blur dataset, the proposed BaMBNet achieves 1.20 dB gain over the previous SOTA method in term of peak signal-to-noise ratio (PSNR) and reduces learnable parameters by 85%. The details of the code and dataset are available at https://github.com/junjun-jiang/BaMBNet.      

Contrastive Meta-learning on Heterogeneous Information Networks for Cold-start Recommendation

In recommender systems, the cold-start issue is challenging due to the lack of interactions between users or items. Such an issue can be alleviated via data-level and model-level strategies. Traditional data-level methods employ auxiliary information like feature information to enhance the learning of user and item embeddings. Recently, Heterogeneous Information Networks (HINs) have been incorporated into the recommender system as they provide more fruitful auxiliary information and meaningful semantics. However, these models are unable to capture the structural and semantic information comprehensively and neglect the unlabeled information of HINs during training. Model-level methods propose to apply the meta-learning framework which naturally fits into the cold-start issue, as it learns the prior knowledge from similar tasks and adapts to new tasks quickly with few labeled samples. Therefore, we propose a contrastive meta-learning framework on HINs named CM-HIN, which addresses the cold-start issue at both data level and model level. In particular, we explore meta-path and network schema views to describe the higher-order and local structural information of HINs. Within meta-path and network schema views, contrastive learning is adopted to mine the unlabeled information of HINs and incorporate these two views. Extensive experiments on three benchmark datasets demonstrate that CM-HIN outperforms all state-of-the-art baselines in three cold-start scenarios.     

元学习的不确定性特征构建及初步分析

元学习即应用机器学习的方法(元算法)寻求问题的特征(元特征)与算法相对性能测度间的映射,从而形成元知识的学习过程,如何构建和提取元特征是其重要的研究内容.针对目前相关研究所用到的元特征大部分是数据的统计特征的问题,提出不确定性建模并研究不确定性对于学习系统的影响.根据样本的不一致性、边界的复杂性、模型输出的不确定性、线...     

Introduction to the Special Issue on Meta-Learning

Recent advances in meta-learning are providing the foundations to construct meta-learning assistants and task-adaptive learners. The goal of this special issue is to foster an interest in meta-learning by compiling representative work in the field. The contributions to this special issue provide strong insights into the construction of future meta-learning tools. In this introduction we present a common frame of reference to address work in meta-learning through the concept of meta-knowledge. We show how meta-learning can be simply defined as the process of exploiting knowledge about learning that enables us to understand and improve the performance of learning algorithms.     

Fast Theta-Subsumption with Constraint Satisfaction Algorithms

Relational learning and Inductive Logic Programming (ILP) commonly use as covering test the -subsumption test defined by Plotkin. Based on a reformulation of -subsumption as a binary constraint satisfaction problem, this paper describes a novel -subsumption algorithm named Django,¹ which combines well-known CSP procedures and -subsumption-specific data structures. Django is validated using the stochastic complexity framework developed in CSPs, and imported in ILP by Giordana et Saitta. Principled and extensive experiments within this framework show that Django improves on earlier -subsumption algorithms by several orders of magnitude, and that different procedures are better at different regions of the stochastic complexity landscape. These experiments allow for building a control layer over Django, termed Meta-Django, which determines the best procedures to use depending on the order parameters of the -subsumption problem instance. The performance gains and good scalability of Django and Meta-Django are finally demonstrated on a real-world ILP task (emulating the search for frequent clauses in the mutagenesis domain) though the smaller size of the problems results in smaller gain factors (ranging from 2.5 to 30).     

融合MAML和CBAM的安卓恶意应用家族分类模型

为满足对新兴安卓恶意应用家族的快速检测需求,提出一种融合MAML(model-agnostic meta-learning)和CBAM(convolutional block attention module)的安卓恶意应用家族分类模型MAML-CAS。将安卓恶意应用样本集中的DEX文件可视化为灰度图,并构建任务集;融合混合域注意力机制CBAM,设计两个具有同等结构的卷积神经网络,分别作为基学习器和元学习器,这两个学习器在自动提取任务集中样本特征的同时,可从通道和空间两个维度来增强关键特征表达;利用元学习方法 MAML对两个学习器进行训练,其中基学习器完成特定恶意家族分类任务的属性学习,元学习器则学习不同任务的共性;在两个学习器训练完成后,MAML-CAS将获得初始化参数,在面对新的安卓恶意应用家族分类任务时,不需要重新训练,只需要少量样本就可以快速迭代;利用训练完成的基学习器提取安卓恶意应用家族特征,并利用SVM进行恶意家族分类。实验结果表明,MAML-CAS模型对新兴小样本安卓恶意应用家族具有良好的检测效果,检测速度较快,并具有较好的稳定性。     

小样本轴承故障诊断研究综述

随着数据时代的来临,基于数据驱动的轴承故障诊断方法表现出了优越的性能,但是此类方法依赖大量标记数据,而在实际生产过程中很难收集到大量的数据,因此小样本的轴承故障诊断具有很高的研究价值。对小样本条件下的轴承故障诊断方法进行了回顾,并将其分为两类：基于数据的方法和基于模型的方法。其中基于数据的方法是从数据角度对原始样本进行扩充;基于模型的方法是指利用模型优化特征提取或者提高分类精度等。总结了当前小样本条件下故障诊断方法的不足,并展望了小样本轴承故障诊断的未来。     

Teachers cooperation: team-knowledge distillation for multiple cross-domain few-shot learning

Although few-shot learning (FSL) has achieved great progress, it is still an enormous challenge especially when the source and target set are from different domains, which is also known as cross-domain few-shot learning (CD-FSL). Utilizing more source domain data is an effective way to improve the performance of CD-FSL. However, knowledge from different source domains may entangle and confuse with each other, which hurts the performance on the target domain. Therefore, we propose team-knowledge distillation networks (TKD-Net) to tackle this problem, which explores a strategy to help the cooperation of multiple teachers. Specifically, we distill knowledge from the cooperation of teacher networks to a single student network in a meta-learning framework. It incorporates task-oriented knowledge distillation and multiple cooperation among teachers to train an efficient student with better generalization ability on unseen tasks. Moreover, our TKD-Net employs both response-based knowledge and relation-based knowledge to transfer more comprehensive and effective knowledge. Extensive experimental results on four fine-grained datasets have demonstrated the effectiveness and superiority of our proposed TKD-Net approach.     

MAML2: meta reinforcement learning via meta-learning for task categories

Meta-learning has been widely applied to solving few-shot reinforcement learning problems, where we hope to obtain an agent that can learn quickly in a new task. However, these algorithms often ignore some isolated tasks in pursuit of the average performance, which may result in negative adaptation in these isolated tasks, and they usually need sufficient learning in a stationary task distribution. In this paper, our algorithm presents a hierarchical framework of double meta-learning, and the whole framework includes classification, meta-learning, and re-adaptation. Firstly, in the classification process, we classify tasks into several task subsets, considered as some categories of tasks, by learned parameters of each task, which can separate out some isolated tasks thereafter. Secondly, in the meta-learning process, we learn category parameters in all subsets via meta-learning. Simultaneously, based on the gradient of each category parameter in each subset, we use meta-learning again to learn a new meta-parameter related to the whole task set, which can be used as an initial parameter for the new task. Finally, in the re-adaption process, we adapt the parameter of the new task with two steps, by the meta-parameter and the appropriate category parameter successively. Experimentally, we demonstrate our algorithm prevents the agent from negative adaptation without losing the average performance for the whole task set. Additionally, our algorithm presents a more rapid adaptation process within re-adaptation. Moreover, we show the good performance of our algorithm with fewer samples as the agent is exposed to an online meta-learning setting.