面向边缘智能的两阶段对抗知识迁移方法

对抗样本的出现,对深度学习的鲁棒性提出了挑战.随着边缘智能的兴起,如何在计算资源有限的边缘设备上部署鲁棒的精简深度学习模型,是一个有待解决的问题.由于精简模型无法通过常规的对抗训练获得良好的鲁棒性,提出两阶段对抗知识迁移的方法,先将对抗知识从数据向模型迁移,然后将复杂模型获得的对抗知识向精简模型迁移.对抗知识以对抗样本的数据形式蕴含,或以模型决策边界的形式蕴含.具体而言,利用云平台上的GPU集群对复杂模型进行对抗训练,实现对抗知识从数据向模型迁移;利用改进的蒸馏技术将对抗知识进一步从复杂模型向精简模型的迁移,最后提升边缘设备上精简模型的鲁棒性.在MNIST,CIFAR-10和CIFAR-100这3个数据集上进行验证,实验结果表明：提出的这种两阶段对抗知识迁移方法可以有效地提升精简模型的性能和鲁棒性,同时加快训练过程的收敛性.

Two-stage Adversarial Knowledge Transfer for Edge Intelligence

The emergence of adversarial examples brings challenges to the robustness of deep learning. With the development of edge intelligence, how to train a robust and compact deep learning mode on edge devices with limited computing resources is also a challenging problem. Since compact models cannot obtain sufficient robustness through conventional adversarial training, a method called two-stage adversarial knowledge transfer is proposed. The method transfers adversarial knowledge from data to models and complex models to compact models. The so-called adversarial knowledge has two forms, one is contained in data with the form of adversarial examples, and the other is contained in models with the form of decision boundary. The GPU clusters of cloud center is first leveraged to train the complex model with adversarial examples to realize the transfer of adversarial knowledge from data to models, and then an improved distillation approach is leveraged to realize the further transfer of adversarial knowledge from complex models to compact models on edge nodes. The experiments over MNIST and CIFAR-10 show that this two-stage adversarial knowledge transfers can efficiently improve the robustness and convergence of compact models.