初始化卷积神经网络的主成分洗牌方法

为了更好地初始化卷积神经网络,提出了一种初始化卷积核的有效方法,称为主成分洗牌方法.该方法首先对第1个卷积层的每个输入特征图的所有感受野进行采样,再对采样得到的图像块按输入特征图分别进行主成分分析,利用主成分分析得到的投影矩阵初始化该层卷积核,最后按上述过程依次对各层卷积核进行初始化.使用该方法在MNIST与CIFAR-10数据集上进行卷积层初始化实验.实验结果表明:与目前常用的随机初始化算法、Xavier初始化算法相比,该方法在提高网络的训练速度和测试集正确率方面均具有优越性.

PCA Shuffling Initialization of Convolutional Neural Networks

To initialize convolutional neural networks better, an effective method named principal component analysis ( PCA) Shuffling initialization was proposed. The method consisted of three steps. First, for the first convolutional layer, all receptive field of each feature map on training set was sampled. Then, principal component analysis of image patches separately for each feature map was conducted, and projection matrix was used to initialize filter of first convolutional layer. Finally, the first two steps on the other convolutional layers layer-wisely were performed. Experimental results on MNIST and CIFAR-10 dataset show that the proposed initialization has advantages of accuracy and speed of convergence compared to the common method such as random initialization and Xavier initialization.