Bilinear factorisation for facial expression analysis and synthesis

The paper addresses the issue of face representations for facial expression recognition and synthesis. In this context, a global active appearance model is used in conjunction with two bilinear factorisation models to separate expression and identity factors from the global appearance parameters. Although active appearance models and bilinear modelling are not new concepts, the paper's main contribution consists in combining both techniques to improve facial expression recognition and synthesis (control). Indeed, facial expression recognition is performed through linear discriminant analysis of the global appearance parameters extracted by active appearance model search. Results are compared to ones obtained for the same training and test images using classification of the expression factors extracted by bilinear factorisation. This experiment highlights the advantages of bilinear factorisation. Finally, it is proposed to exploit bilinear factorisation to synthesise facial expressions through replacement of the extracted expression factors. This yields very interesting synthesis performances in terms of visual quality of the synthetic faces. Indeed, synthetic open mouth reconstruction, either with or without teeth appearing, is of better quality than with classical linear-regression-based synthesis.