Semi‐supervised classification of facial expression using a mixture of multivariate t distributions

Abstract: This paper addresses the semi‐supervised classification of facial expression images using a mixture of multivariate t distributions. The facial expression features are first extracted into labelled graph vectors using the Gabor wavelet transformation. We then learn a mixture of multivariate t distributions by using the labelled graph vectors, and set correspondence between the component distributions and the basic facial emotions. According to this correspondence, the classification of a given testing image is implemented in a probabilistic way according to its fitted posterior probabilities of component memberships. Specifically, we perform hard classification of the testing image by assigning it into an emotional class that the corresponding mixture component has the highest posterior probability, or softly use the posterior probabilities as the estimates of the semantic ratings of expressions. The experimental results on the Japanese female facial expression database, Ekman's Pictures of Facial Affect database and the AR database demonstrate the effectiveness of the proposed method.