Convolutional restricted Boltzmann machines learning for robust visual tracking

It is a critical step to choose visual features in object tracking. Most existing tracking approaches adopt handcrafted features, which greatly depend on people’s prior knowledge and easily become invalid in other conditions where the scene structures are different. On the contrary, we learn informative and discriminative features from image data of tracking scenes itself. Local receptive filters and weight sharing make the convolutional restricted Boltzmann machines (CRBM) suit for natural images. The CRBM is applied to model the distribution of image patches sampled from the first frame which shares same properties with other frames. Each hidden variable corresponding to one local filter can be viewed as a feature detector. Local connections to hidden variables and max-pooling strategy make the extracted features invariant to shifts and distortions. A simple naive Bayes classifier is used to separate object from background in feature space. We demonstrate the effectiveness and robustness of our tracking method in several challenging video sequences. Experimental results show that features automatically learned by CRBM are effective for object tracking.