基于特征分布差异的对抗样本检测

诸多神经网络模型已被证明极易遭受对抗样本攻击。对抗样本则是攻击者为模型所恶意构建的输入,通过对原始样本输入添加轻微的扰动,导致其极易被机器学习模型错误分类。这些对抗样本会对日常生活中的高要求和关键应用的安全构成严重威胁,如自动驾驶、监控系统和生物识别验证等应用。研究表明在模型的训练期间,检测对抗样本方式相比通过增强模型来预防对抗样本攻击更为有效,且训练期间神经网络模型的中间隐层可以捕获并抽象样本信息,使对抗样本与干净样本更容易被模型所区分。因此,本文针对神经网络模型中的不同隐藏层,其对抗样本输入和原始自然输入的隐层表示进行统计特征差异进行研究。本文研究表明,统计差异可以在不同层之间进行区别。本文通过确定最有效层识别对抗样本和原始自然训练数据集统计特征之间的差异,并采用异常值检测方法,设计一种基于特征分布的对抗样本检测框架。该框架可以分为广义对抗样本检测方法和条件对抗样本检测方法,前者通过在每个隐层中提取学习到的训练数据表示,得到统计特征后,计算测试集的异常值分数,后者则通过深层神经网络模型对测试数据的预测结果比较,得到对应训练数据的统计特征。本文所计算的统计特征包括到原点的范数距离L2和样本协方差矩阵的顶奇异向量的相关性。实验结果显示了两种检测方法均可以利用隐层信息检测出对抗样本,且对由不同攻击产生的对抗样本均具有较好的检测效果,证明了本文所提的检测框架在检测对抗样本中的有效性。     

针对卷积神经网络流量分类器的对抗样本攻击防御

随着深度学习的兴起,深度神经网络被成功应用于多种领域,但研究表明深度神经网络容易遭到对抗样本的恶意攻击。作为深度神经网络之一的卷积神经网络(CNN)目前也被成功应用于网络流量的分类问题,因此同样会遭遇对抗样本的攻击。为提高CNN网络流量分类器防御对抗样本的攻击,本文首先提出批次对抗训练方法,利用训练过程反向传播误差的特点,在一次反向传播过程中同时完成样本梯度和参数梯度的计算,可以明显提高训练效率。同时,由于训练用的对抗样本是在目标模型上生成,因此可有效防御白盒攻击;为进一步防御黑盒攻击,克服对抗样本的可转移性,提出增强对抗训练方法。利用多个模型生成样本梯度不一致的对抗样本,增加对抗样本的多样性,提高防御黑盒攻击的能力。通过真实流量数据集USTC-TFC2016上的实验,我们生成对抗样本的网络流量进行模拟攻击,结果表明针对白盒攻击,批次对抗训练可使对抗样本的分类准确率从17.29%提高到75.37%;针对黑盒攻击,增强对抗训练可使对抗样本的分类准确率从26.37%提高到68.39%。由于深度神经网络的黑箱特性,其工作机理和对抗样本产生的原因目前没有一致的认识。下一步工作对CNN的脆弱性机...     

面向个人信息保护的对抗性图像扰动算法研究

通过研究对抗性图像扰动算法,应对深度神经网络对图像中个人信息的挖掘和发现以保护个人信息安全.将对抗样本生成问题转换为一个含有限制条件的多目标优化问题,考虑神经网络的分类置信度、扰动像素的位置以及色差等目标,利用差分进化算法迭代得到对抗样本.在MNIST和CIFAR-10数据集上,基于深度神经网络LeNet和ResNet进行了对抗样本生成实验,并从对抗成功率、扰动像素数目、优化效果和对抗样本的空间特征等方面进行了对比和分析.结果表明,算法在扰动像素极少的情况下(扰动均值为5)依然可以保证对深度神经网络的有效对抗,并显著优化了扰动像素的位置及色差,达到不破坏原图像的情况下保护个人信息的目的.该研究有助于促进信息技术红利共享与个人信息安全保障之间的平衡,也为对抗样本生成及深度神经网络中分类空间特征的研究提供了技术支撑.     

A novel approach to generating high-resolution adversarial examples

Deep neural networks (DNNs) have improved expressive performance in many artificial intelligence (AI) fields in recent years. However, they can easily induce incorrect behavior due to adversarial examples. The state-of-the-art strategies for generating adversarial examples were established as generative adversarial nets (GAN). Due to a large amount of data and the high computational resources required, previous GAN-based work has only generated adversarial examples for small datasets, resulting in a less favorable visualization of the generated images. To address this problem, we propose a feasible approach, which improves on the AdvGAN framework through data augmentation, combined with PCA and KPCA to map the input instance’s main features onto the latent variables. Experimental results indicate that our approach can generate more natural perturbations on high-resolution images while maintaining 96%?+?of the features of the original input instance. Moreover, we measured 90.30% attack success rates on CIFAR-10 against the target model ResNet152, a small improvement compared to 88.69% for AdvGAN. We applied the same idea to ImageNet and LSUN, and the results showed that it not only achieves a high attack success rate,but can generate strongly semantically adversarial examples with better transferability on prevailing DNNs classification models. We also show that our approach yields competitive results compared to sensitivity analysis-based or optimization-based attacks notable in the literature.

     

Robust graph convolutional networks with directional graph adversarial training

Graph convolutional networks (GCNs), an emerging type of neural network model on graphs, have presented state-of-the-art performance on the node classification task. However, recent studies show that neural networks are vulnerable to the small but deliberate perturbations on input features. And GCNs could be more sensitive to the perturbations since the perturbations from neighbor nodes exacerbate the impact on a target node through the convolution. Adversarial training (AT) is a regularization technique that has been shown capable of improving the robustness of the model against perturbations on image classification. However, directly adopting AT on GCNs is less effective since AT regards examples as independent of each other and does not consider the impact from connected examples. In this work, we explore AT on graph and propose a graph-specific AT method, Directional Graph Adversarial Training (DGAT), which incorporates the graph structure into the adversarial process and automatically identifies the impact of perturbations from neighbor nodes. Concretely, we consider the impact from the connected nodes to define the neighbor perturbation which restricts the perturbation direction on node features towards their neighbor nodes, and additionally introduce an adversarial regularizer to defend the worst-case perturbations. In this way, DGAT can resist the impact of worst-case adversarial perturbations and reduce the impact of perturbations from neighbor nodes. Extensive experiments demonstrate that DGAT can effectively improve the robustness and generalization performance of GCNs. Specially, GCNs with DGAT can provide better performance when there are rare few labels available for training.

     

一种基于局部扰动的图像对抗样本生成方法

近年来, 随着人工智能的研究和发展, 深度学习被广泛应用。深度学习在自然语言处理、计算机视觉等多个领域表现出良好的效果。特别是计算机视觉方面, 在图像识别和图像分类中, 深度学习具备非常高的准确性。然而越来越多的研究表明, 深度神经网络存在着安全隐患, 其中就包括对抗样本攻击。对抗样本是一种人为加入特定扰动的数据样本, 这种特殊样本在传递给已训练好的模型时, 神经网络模型会输出与预期结果不同的结果。在安全性要求较高的场景下, 对抗样本显然会对采用深度神经网络的应用产生威胁。目前国内外对于对抗样本的研究主要集中在图片领域, 图像对抗样本就是在图片中加入特殊信息的图片数据, 使基于神经网络的图像分类模型做出错误的分类。已有的图像对抗样本方法主要采用全局扰动方法,即将这些扰动信息添加在整张图片上。相比于全局扰动, 局部扰动将生成的扰动信息添加到图片的非重点区域, 从而使得对抗样本隐蔽性更强, 更难被人眼发现。本文提出了一种生成局部扰动的图像对抗样本方法。该方法首先使用 Yolo 目标检测方法识别出图片中的重点位置区域, 然后以 MIFGSM 方法为基础, 结合 Curls 方法中提到的先梯度下降再梯度上升的思想,在非重点区域添加扰动信息, 从而生成局部扰动的对抗样本。实验结果表明, 在对抗扰动区域减小的情况下可以实现与全局扰动相同的攻击成功率。     

基于Rectified Adam和颜色不变性的对抗迁移攻击

深度神经网络在物体检测、图像分类、自然语言处理、语音识别等众多领域上得到广泛应用.然而,深度神经网络很容易受到对抗样本(即在原有样本上施加人眼无法察觉的微小扰动)的攻击,而且相同的扰动可以跨模型、甚至跨任务地欺骗多个分类器.对抗样本这种跨模型迁移特性,使得深度神经网络在实际生活的应用受到了很大限制.对抗样本对神经网络的威胁,激发了研究者对对抗攻击的研究兴趣.虽然研究者们已提出了不少对抗攻击方法,但是大多数这些方法(特别是黑盒攻击方法)的跨模型的攻击能力往往较差,尤其是对经过对抗训练、输入变换等的防御模型.为此,提出了一种提高对抗样本可迁移性的方法:RLI-CI-FGSM. RLI-CI-FGSM是一种基于迁移的攻击方法,在替代模型上,使用基于梯度的白盒攻击RLI-FGSM生成对抗样本,同时使用CIM扩充源模型,使RLI-FGSM能够同时攻击替代模型和扩充模型.具体而言,RLI-FGSM算法将Radam优化算法与迭代快速符号下降法相结合,并利用目标函数的二阶导信息来生成对抗样本,避免优化算法陷入较差的局部最优.基于深度神经网络具有一定的颜色变换不变性,CIM算法通过优化对颜色变换图像集合...     

基于自适应噪声添加的防御对抗样本的算法

深度神经网络容易受到对抗样本的攻击。为了解决这个问题,一些工作通过向图像中添加高斯噪声来训练网络,从而提高网络防御对抗样本的能力,但是该方法在添加噪声时并没有考虑到神经网络对图像中不同区域的敏感性是不同的。针对这一问题,提出了梯度指导噪声添加的对抗训练算法。该算法在训练网络时,根据图像中不同区域的敏感性向其添加自适应噪声,在敏感性较大的区域上添加较大的噪声抑制网络对图像变化的敏感程度,在敏感性较小的区域上添加较小的噪声提高其分类精度。在Cifar-10数据集上与现有算法进行比较,实验结果表明,该方法有效地提高了神经网络在分类对抗样本时的准确率。     

Shallow and deep learning for image classification

The paper is focused on the idea to demonstrate the advantages of deep learning approaches over ordinary shallow neural network on their comparative applications to image classifying from such popular benchmark databases as FERET and MNIST. An autoassociative neural network is used as a standalone program realized the nonlinear principal component analysis for prior extracting the most informative features of input data for neural networks to be compared further as classifiers. A special study of the optimal choice of activation function and the normalization transformation of input data allows to improve efficiency of the autoassociative program. One more study devoted to denoising properties of this program demonstrates its high efficiency even on noisy data. Three types of neural networks are compared: feed-forward neural net with one hidden layer, deep network with several hidden layers and deep belief network with several pretraining layers realized restricted Boltzmann machine. The number of hidden layer and the number of hidden neurons in them were chosen by cross-validation procedure to keep balance between number of layers and hidden neurons and classification efficiency. Results of our comparative study demonstrate the undoubted advantage of deep networks, as well as denoising power of autoencoders. In our work we use both multiprocessor graphic card and cloud services to speed up our calculations. The paper is oriented to specialists in concrete fields of scientific or experimental applications, who have already some knowledge about artificial neural networks, probability theory and numerical methods.     

图像对抗样本检测综述

深度神经网络是人工智能领域的一项重要技术, 它被广泛应用于各种图像分类任务. 但是, 现有的研究表明深度神经网络存在安全漏洞, 容易受到对抗样本的攻击, 而目前并没有研究针对图像对抗样本检测进行体系化分析. 为了提高深度神经网络的安全性, 针对现有的研究工作, 全面地介绍图像分类领域的对抗样本检测方法. 首先根据检测器的构建方式将检测方法分为有监督检测与无监督检测, 然后根据其检测原理进行子类划分. 最后总结对抗样本检测领域存在的问题, 在泛化性和轻量化等方面提出建议与展望, 旨在为人工智能安全研究提供帮助.     

基于多粒度特征蒸馏的遥感图像场景分类研究

深度神经网络广泛应用于遥感图像场景分类任务中并能大幅提高分类精度,但隐藏层数较少的神经网络在标记数据不足的遥感场景分类中泛化能力较低,而隐层较多的网络往往需要较大的计算量和模型存储空间,限制了其在嵌入式设备上的应用.提出一种针对遥感图像场景分类的多粒度特征蒸馏方法,将深度网络不同阶段的特征与最终的类别概率同时作为浅层模...     

Improving transferability of adversarial examples with powerful affine-shear transformation attack

Image classification models based on deep neural networks have made great improvements on various tasks, but they are still vulnerable to adversarial examples that could increase the possibility of misclassification. Various methods are proposed to generate adversarial examples under white-box attack circumstances that have achieved a high success rate. However, most existing adversarial attacks only achieve poor transferability when attacking other unknown models with the black-box scenario settings. In this paper, we propose a new method that generates adversarial examples based on affine-shear transformation from the perspective of deep model input layers and maximizes the loss function during each iteration. This method could improve the transferability and the input diversity of adversarial examples, and we also optimize the above adversarial examples generation process with Nesterov accelerated gradient. Extensive experiments on ImageNet Dataset indicate that our proposed method could exhibit higher transferability and achieve higher attack success rates on both single model settings and ensemble-model settings. It can also combine with other gradient-based methods and image transformation-based methods to further build more powerful attacks.     

Entropy-based generation of supervised neural networks for classification of structured patterns

Sperduti and Starita proposed a new type of neural network which consists of generalized recursive neurons for classification of structures. In this paper, we propose an entropy-based approach for constructing such neural networks for classification of acyclic structured patterns. Given a classification problem, the architecture, i.e., the number of hidden layers and the number of neurons in each hidden layer, and all the values of the link weights associated with the corresponding neural network are automatically determined. Experimental results have shown that the networks constructed by our method can have a better performance, with respect to network size, learning speed, or recognition accuracy, than the networks obtained by other methods.     

基于图像着色的无限制攻击

深度学习目前被广泛应用于计算机视觉、机器人技术和自然语言处理等领域。然而,已有研究表明,深度神经网络在对抗样本面前很脆弱,一个精心制作的对抗样本就可以使深度学习模型判断出错。现有的研究大多通过产生微小的Lp范数扰动来误导分类器的对抗性攻击,但是取得的效果并不理想。本文提出一种新的对抗攻击方法——图像着色攻击,将输入样本转为灰度图,设计一种灰度图上色方法指导灰度图着色,最终利用经过上色的图像欺骗分类器实现无限制攻击。实验表明,这种方法制作的对抗样本在欺骗几种最先进的深度神经网络图像分类器方面有不俗表现,并且通过了人类感知研究测试。     

联合多重对抗与通道注意力的高安全性图像隐写

目的 现有基于对抗图像的隐写算法大多只能针对一种隐写分析器设计对抗图像,且无法抵御隐写分析残差网络(steganalysis residual network,SRNet)、Zhu-Net等最新基于卷积神经网络隐写分析器的检测。针对这一现状,提出了一种联合多重对抗与通道注意力的高安全性图像隐写方法。方法 采用基于U-Net结构的生成对抗网络生成对抗样本图像,利用对抗网络的自学习特性实现多重对抗隐写网络参数迭代优化,并通过针对多种隐写分析算法的对抗训练,生成更适合内容隐写的载体图像。同时,通过在生成器中添加多个轻量级通道注意力模块,自适应调整对抗噪声在原始图像中的分布,提高生成对抗图像的抗隐写分析能力。其次,设计基于多重判别损失和均方误差损失相结合的动态加权组合方案,进一步增强对抗图像质量,并保障网络快速稳定收敛。结果 实验在BOSS Base 1.01数据集上与当前主流的4种方法进行比较,在使用原始隐写图像训练后,相比于基于U-Net结构的生成式多重对抗隐写算法等其他4种方法,使得当前性能优异的5种隐写分析器平均判别准确率降低了1.6%;在使用对抗图像和增强隐写图像再训练后,相比其他4种方法,仍使得当前性能优异的5种隐写分析器平均判别准确率降低了6.8%。同时也对对抗图像质量进行分析,基于测试集生成的2 000幅对抗图像的平均峰值信噪比(peak signal-tonoise ratio,PSNR)可达到39.925 1 dB,实验结果表明本文提出的隐写网络极大提升了隐写算法的安全性。结论 本文方法在隐写算法安全性领域取得了较优秀的性能,且生成的对抗图像具有很高的视觉质量。     

基于模型的深度学习通信信号鲁棒识别算法

深度学习现在是计算机视觉和自然语言处理的热门话题.在许多应用中,深度神经网络(DNN)的性能都优于传统的方法,并且已经成功应用于调制分类和无线电信号表示等任务的学习.近几年研究发现深度神经网络极易受到对抗性攻击,对“对抗性示例”缺乏鲁棒性.笔者就神经网络的通信信号识别算法的鲁棒性问题,将经过PGD攻击的数据看作基于模型...     

TopoAct: Visually Exploring the Shape of Activations in Deep Learning

Deep neural networks such as GoogLeNet, ResNet, and BERT have achieved impressive performance in tasks such as image and text classification. To understand how such performance is achieved, we probe a trained deep neural network by studying neuron activations, i.e.combinations of neuron firings, at various layers of the network in response to a particular input. With a large number of inputs, we aim to obtain a global view of what neurons detect by studying their activations. In particular, we develop visualizations that show the shape of the activation space, the organizational principle behind neuron activations, and the relationships of these activations within a layer. Applying tools from topological data analysis, we present TopoAct , a visual exploration system to study topological summaries of activation vectors. We present exploration scenarios using TopoAct that provide valuable insights into learned representations of neural networks. We expect TopoAct to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection.     

基于孪生结构的对抗样本攻击动态防御方法

神经网络模型已被广泛应用于多个研究领域,但神经网络模型本身存在易受到对抗样本攻击的缺点,如在图像分类中,只需在原始图片中添加微小的对抗扰动生成对抗样本,就可以轻易欺骗神经网络分类模型,这给许多领域的应用安全带来严重的威胁。因此,研究如何提高神经网络分类模型对对抗样本攻击的防御能力成为深度学习安全领域的研究热点。目前常用的对抗样本攻击防御方法往往只侧重于提高模型对对抗样本分类的鲁棒性,或者只侧重于检测拦截对抗样本,而对抗训练需要收集大量对抗样本,且难以防御新类型的对抗样本攻击,对于使用额外的分类器去检测对抗样本的方法,则存在着易受到二次攻击等缺点。针对这些问题,提出一种基于孪生神经网络结构的对抗样本攻击动态防御方法,利用孪生结构可比较两个输入相似性的特点,从孪生神经网络两侧的预测差异着手,检测图片在动态滤波前后是否存在不同的攻击效果,从而筛选出带有动态扰动的对抗样本。实验结果表明,在没有收集特定种类的对抗样本进行训练的情况下,该方法对多种对抗样本攻击取得了良好的通用防御效果,其中在FGSM对抗样本测试集上的防御准确率达到95.35%,在DeepFool和JSMA对抗样本测试集上的防御准确...     

A dyadic multi-resolution deep convolutional neural wavelet network for image classification

For almost the past four decades, image classification has gained a lot of attention in the field of pattern recognition due to its application in various fields. Given its importance, several approaches have been proposed up to now. In this paper, we will present a dyadic multi-resolution deep convolutional neural wavelets’ network approach for image classification. This approach consists of performing the classification of one class versus all the other classes of the dataset by the reconstruction of a Deep Convolutional Neural Wavelet Network (DCNWN). This network is based on the Neural Network (NN) architecture, the Fast Wavelet Transform (FWT) and the Adaboost algorithm. It consists, first, of extracting features using the FWT based on the Multi-Resolution Analysis (MRA). These features are used to calculate the inputs of the hidden layer. Second, those inputs are filtered by using the Adaboost algorithm to select the best ones corresponding to each image. Third, we create an AutoEncoder (AE) using wavelet networks of all images. Finally, we apply a pooling for each hidden layer of the wavelet network to obtain a DCNWN that permits the classification of one class and rejects all other classes of the dataset. Classification rates given by our approach show a clear improvement compared to those cited in this article.

     

一种面向人脸活体检测的对抗样本生成算法

近年来,基于深度卷积神经网络的人脸活体检测技术取得了较好的性能.然而,深度神经网络被证明容易受到对抗样本的攻击,影响了人脸系统的安全性.为了建立更好的防范机制,需充分研究活体检测任务对抗样本的生成机理.相对于普通分类问题,活体检测任务具有类间距离小,且扰动操作难度大等特性.在此基础上,提出了基于最小扰动维度和人眼视觉特性的活体检测对抗样本生成算法,将扰动集中在少数几个维度上,并充分考虑人眼的视觉连带集中特性,加入扰动点的间距约束,以便最后生成的对抗样本更不易被人类察觉.该方法只需平均改变输入向量总维度的1.36%,即可成功地欺骗网络,使网络输出想要的分类结果.通过志愿者的辨认,该方法的人眼感知率比DeepFool方法降低了20%.