Background subtraction model based on color and depth cues

Background subtraction consists of segmenting objects in movement in a video captured by a static camera. This is typically performed using color information, but it leads to wrong estimations due to perspective and illumination issues. We show that multimodal approaches based on the integrated use of color and depth cues produce more accurate and robust results than using either data source independently. Depth is less affected by issues such as shadows or foreground objects similar to background. However, objects close to the background may not be detected when using only range information, being color information complementary in those cases. We propose an extension of a well-known background subtraction technique which fuses range and color information, as well as a post-processing mask fusion stage to get the best of each feature. We have evaluated the method proposed using a well-defined dataset and different disparity estimation algorithms, showing the benefits of our method for fusion color and depth cues.