面向深度神经网络训练的数据差分隐私保护随机梯度下降算法

针对传统深度神经网络所采用的随机梯度下降算法忽略了对数据集隐私性保护的缺点,提出一种基于数据差分隐私保护的随机梯度下降算法。引入范数剪切与附加高斯噪声操作,对传统梯度更新策略进行改进。为衡量每次迭代过程中对数据隐私性的破坏,提出隐私损失累积函数在迭代过程中对数据隐私性的侵犯程度进行度量。MNIST手写数字识别和CIFAR-10图像分类实验表明,该算法在保护数据集隐私性的同时,对手写数字以及图像分类的识别准确率分别超过了90%和70%,且相较于传统的随机梯度下降算法,其准确率提升了5%以上。该算法在实际工程中能够有效兼顾数据隐私性保护与神经网络辨识准确度。

DATA DIFFERENTIAL PRIVACY PROTECTION STOCHASTIC GRADIENT DESCENT ALGORITHM FOR DEEP NEURAL NETWORK TRAINING

In allusion to the shortcoming of stochastic gradient descent algorithm for traditional deep neural network,which ignores the privacy protection of datasets,we propose a stochastic gradient descent algorithm based on data differential privacy protection.The traditional gradient update strategy was improved by introducing norm cutting and Gaussian noise operations.Then,in order to measure the destruction of data privacy in each iteration,a privacy loss accumulation function was proposed to measure the degree of data privacy violation in the iteration process.MNIST handwritten digit recognition and the CIFAR-10 image classification experiments show that the recognition accuracy of the algorithm for handwritten digits and image classification exceeds 90%and 70%respectively while protecting the privacy of datasets.Compared with the traditional stochastic gradient descent algorithm,the accuracy can be improved by more than 5%.It also can effectively balance the data privacy protection and neural network identification accuracy in practical engineering.