Discriminative deep belief networks for visual data classification

Visual data classification using insufficient labeled data is a well-known hard problem. Semi-supervise learning, which attempts to exploit the unlabeled data in additional to the labeled ones, has attracted much attention in recent years. This paper proposes a novel semi-supervised classifier called discriminative deep belief networks (DDBN). DDBN utilizes a new deep architecture to integrate the abstraction ability of deep belief nets (DBN) and discriminative ability of backpropagation strategy. For unsupervised learning, DDBN inherits the advantage of DBN, which preserves the information well from high-dimensional features space to low-dimensional embedding. For supervised learning, through a well designed objective function, the backpropagation strategy directly optimizes the classification results in training dataset by refining the parameter space. Moreover, we apply DDBN to visual data classification task and observe an important fact that the learning ability of deep architecture is seriously underrated in real-world applications, especially in visual data analysis. The comparative experiments on standard datasets of different types and different scales demonstrate that the proposed algorithm outperforms both representative semi-supervised classifiers and existing deep learning techniques. For visual dataset, we can further improve the DDBN performance with much larger and deeper architecture.