An industrial portrait background removal solution based on knowledge infusion

Background removal of an identity (ID) picture consists in separating the foreground (face, body, hair and clothes) from the background of the image. It is a necessary groundwork for all modern identity documents that also has many benefits for improving ID security. State of the art image processing techniques encountered several segmentation issues and offer only partial solutions. It is due to the presence of erratic components like hairs, poor contrast, luminosity variation, shadow, color overlap between clothes and background. In this paper, a knowledge infused approach is proposed that hybridizes smart image processing tasks and prior knowledge. The research is based on a divide and conquer strategy aiming at simulating the sequential attention of human when performing a manual segmentation. Knowledge is infused by considering the spatial relation between anatomic elements of the ID image (face feature, forehead, body and hair) as well as their “signal properties”. The process consists in first determining a convex hull around the person’s body including all the foreground while keeping very close to the contour between the background and the foreground. Then, a body map generated from biometric analysis associated to an automatic grab cut process is applied to reach a finer segmentation. Finally, a heuristic-based post-processing step consisting in correcting potential hair and fine boundary issues leads to the final segmentation. Experimental results show that the newly proposed architecture achieves better performances than tested current state-of-the-art methodologies including active contours, generalist popular deep learning techniques, and also two other ones considered as the smartest for portrait segmentation. This new technology has been adopted by an international company as its industrial ID foreground solution.