Flexible and Accurate Transparent‐Object Matting and Compositing Using Refractive Vector Field

In digital image editing, environment matting and compositing are fundamental and interesting operations that can capture and simulate the refraction and reflection effects of light from an environment. The state‐of‐the‐art real‐time environment matting and compositing method is short of flexibility, in the sense that it has to repeat the entire complex matte acquisition process if the distance between the object and the background is different from that in the acquisition stage, and also lacks accuracy, in the sense that it can only remove noises but not errors. In this paper, we introduce the concept of refractive vector and propose to use a refractive vector field as a new representation for environment matte. Such refractive vector field provides great flexibility for transparent‐object environment matting and compositing. Particularly, with only one process of the matte acquisition and the refractive vector field extraction, we are able to composite the transparent object into an arbitrary background at any distance. Furthermore, we introduce a piecewise vector field fitting algorithm to simultaneously remove both noises and errors contained in the extracted matte data. Experimental results show that our method is less sensitive to artefacts and can generate perceptually good composition results for more general scenarios.