A system for facial expression recognition of a speaker using front-view face judgment, vowel judgment, and thermal image processing

For facial expression recognition, we previously selected three images: (1) just before speaking, and speaking (2) the first vowel and (3) the last vowel in an utterance. A frame of the front-view face in a dynamic image was selected by estimating the face direction. Based on our method, we have been developing an on-line system for recognizing the facial expression of a speaker using front-view face judgment, vowel judgment, and thermal image processing. In the proposed system, we used three personal computers connected by cables to form a local area network. As an initial investigation, we adopted the utterance of the Japanese name ??Taro,?? which is semantically neutral. Using the proposed system, the facial expressions of one male subject were discriminable with 76?% accuracy when he exhibited one of the intentional facial expressions of ??angry,?? ??happy,?? ??neutral,?? ??sad,?? and ??surprised.??