Evaluation of color descriptors for projector-camera systems

Spatial Augmented Reality applications generally use projector-camera systems to control the visual projection appearance by comparing the initial projected and the acquired images. To obtain an accurate geometric compensation, a non-intrusive feature-point matching approach can be exploited which must handle complex photometric distortions due to the spectral devices responses, complex illumination and the mixing of the projected image with the projection surface.This paper first discusses the invariance properties of existing color descriptors in that application for non-intrusive geometric compensation. Their performance is evaluated using the framework of Setkov et al. (2013) extended by adding the several new test cases: modeled synthetic projections, real-world projections under various illuminants on one and two planar surfaces. Our experimental results show two main conclusions: (1) classical color vision models are hardly suitable to model the distortions in a projector-camera system, and (2) the LHE-based descriptor (Local Histogram Equalization) is the most reliable to compensate real-projections.