基于多隐层Gibbs采样的深度信念网络训练方法

深度信念网络（Deep belief network，DBN）作为一类非常重要的概率生成模型，在多个领域都有着广泛的用途.现有深度信念网的训练分为两个阶段，首先是对受限玻尔兹曼机（Restricted Boltzmann machine，RBM）层自底向上逐层进行的贪婪预训练，使得每层的重构误差最小，这个阶段是无监督的；随后再对整体的权值使用有监督的反向传播方法进行精调.本文提出了一种新的DBN训练方法，通过多隐层的Gibbs采样，将局部RBM层组合，并在原有的逐层预训练和整体精调之间进行额外的预训练，有效地提高了DBN的精度.本文同时比较了多种隐层的组合方式，在MNIST和ShapeSet以及Cifar10数据集上的实验表明，使用两两嵌套组合方式比传统的方法错误率更低.新的训练方法可以在更少的神经元上获得比以往的训练方法更好的准确度，有着更高的算法效率.

A Deep Belief Networks Training Strategy Based on Multi-hidden Layer Gibbs Sampling

Deep belief network (DBN) is a very important probabilistic generative model that can be used in many areas. The current training approach of DBN involves two phases. The first is a fully unsupervised pre-training process, which is a down-top and layer-by-layer one to train the restricted Boltzmann machine (RBM) layers, making the reconstruction error of each layer minimal. The second is a supervised stage which uses the back propagation to fine-tune the entire parameters of the model. In this paper, a new training strategy for DBN is proposed. Between the current two training phases, this paper introduces another training strategy to combine multiple local RBMs into an overall probability model for multi hidden layer Gibbs sampling, which effectively improves the accuracy of DBN. This paper has compared a variety of combinations of RBM layers, experiments on the MNIST, ShapeSet and Cifar10 dataset show that our method outperforms the existing training algorithms for DBN. The new algorithm can achieve better accuracy with fewer neurons, also achieves higher algorithm efficiency.