改进的弹性网模型在深度神经网络中的应用

由于具有较高的模型复杂度,深层神经网络容易产生过拟合问题,为了减少该问题对网络性能的不利影响,提出一种基于改进的弹性网模型的深度学习优化方法。首先,考虑到变量之间的相关性,对弹性网模型中的L1范数的不同变量进行自适应加权,从而得到L2范数与自适应加权的L1范数的线性组合。其次,将改进的弹性网络模型与深度学习的优化模型相结合,给出在这种新正则项约束下求解神经网络参数的过程。然后,推导出改进的弹性网模型在神经网络优化中具有群组选择能力和Oracle性质,进而从理论上保证该模型是一种更加鲁棒的正则化方法。最后,在多个回归问题和分类问题的实验中,相对于L1、L2和弹性网正则项,该方法的回归测试误差可分别平均降低87.09、88.54和47.02,分类测试准确度可分别平均提高3.98、2.92和3.58个百分点。由此,在理论和实验两方面验证了改进的弹性网模型可以有效地增强深层神经网络的泛化能力,提升优化算法的性能,解决深度学习的过拟合问题。

Improved elastic network model for deep neural network

Deep neural networks tend to suffer from overfitting problem because of the high complexity of the model. To reduce the adverse eeffects of the problem on the network performance, an improved elastic network model based deep learning optimization method was proposed. Firstly, considering the strong correlation between the variables, the adaptive weights were assigned to different variables of L1-norm in elastic network model, so that the linear combination of the L2-norm and the adaptively weighted L1-norm was obtained. Then, the solving process of neural network parameters under this new regularization term was given by combining improved elastic network model with the deep learning optimization model. Moreover, the robustness of this proposed model was theoretically demonstrated by showing the grouping selection ability and Oracle property of the improved elastic network model in the optimization of neural network. At last, in regression and classification experiments, the proposed model was compared with L1-norm, L2-norm and elastic network regularization term, and had the regression error decreased by 87.09, 88.54 and 47.02 and the classification accuracy improved by 3.98, 2.92 and 3.58 percentage points respectively. Thus, theory and experimental results prove that the improved elastic network model can effectively improve the generalization ability of deep neural network model and the performance of optimization algorithm, and solve the overfitting problem of deep learning.