一种基于知识蒸馏的神经网络鲁棒性迁移方法

近几年来,深度神经网络在多个领域展现了非常强大的应用能力,但是研究者们发现,通过在输入上添加难以察觉的扰动,可以改变神经网络的输出决策,这类样本被称为对抗样本.目前防御对抗样本,最常见的方法是对抗训练,但是对抗训练有着非常高的训练代价.我们提出了一种知识蒸馏的鲁棒性迁移方案(Robust-KD),结合特征图与雅克比矩阵...

A Robust Transfer Method of Neural Network based on Knowledge Distillation

In recent years, neural networks have shown very powerful performance in many fields, but researchers have found that by adding imperceptible interference to the input, neural network decisions can be changed. Such samples are called adversarial samples. At present, the most common method for defending adversarial examples is adversarial training, but the training cost of adversarial training is very high. We propose a knowledge purification scheme (Robust-KD) combining feature maps and Jacobian matrix constraints. By migrating robust features from a robust network, we can obtain considerable white box defense capabilities at relatively low training costs. We have conducted a lot of experiments on the Cifar10, Cifar100 and ImageNet datasets. Experiments have proved the effectiveness of the scheme. Even under a very powerful white box attack, our model still has good classification accuracy.