改进型的batch normalization:BNalpha

针对提高卷积神经网络(convolutional neural network,CNN)在图像识别方向的训练速度和识别准确率进行了研究.从BN(batch normalization)着手,通过新增参数对BN的仿射变换进行具体调节,并提出一种改进型的BN——BNalpha.除去带有某些特定结构的神经网络,相对于原始的BN,BNalpha可以在不增加运算复杂度的前提下,提升神经网络的训练速度和识别准确度.通过对BN仿射变换的参数进行分析和对比,尝试解释BN在网络中的运行机理,并以此说明BNalpha相对于BN的改进为何生效.最后通过CIFAR-10和CIFAR-100数据集以及不同类型的卷积神经网络结构对BNalpha和BN进行对比实验分析,实验结果表明BNalpha能够进一步提升训练速度和识别准确度.

Improvement of batch normalization:BNalpha

In order to improve the training speed and certain classification accuracy of CNN in the image classification, this paper started from BN, adjusted the affine transform of BN by adding new parameter, and proposed an improved BN, called BNalpha. Except for the neural networks with some specific structures, compared with the original BN, BNalpha could improve the training speed and certain classification accuracy of general neural networks without increasing the computational complexity. By analyzing and comparing the parameters of BN affine transform, this paper tried to explain the operation mechanism of BN partially, and conducted contrast experiment which covering different periods during training to illustrate that BNalpha was superior to the original BN. Based on the CIFAR-10 and CIFAR-100 datasets, by using various types of CNN structures, it compared and analyzed BNalpha and BN, and the experimental results verify that BNalpha can further improve the training speed and certain classification accuracy.