融合背景上下文特征的视觉情感识别与预测方法 |
| |
引用本文: | 冯月华,魏若岩,朱晓庆.融合背景上下文特征的视觉情感识别与预测方法[J].计算机应用研究,2024,41(5). |
| |
作者姓名: | 冯月华 魏若岩 朱晓庆 |
| |
作者单位: | 河北经贸大学信息技术学院、河北省跨境电商技术创新中心,河北经贸大学信息技术学院、河北省跨境电商技术创新中心,北京工业大学信息学部 |
| |
基金项目: | 国家自然科学基金资助项目(62103009);河北省重点研发计划资助项目(17216108);河北省自然基金资助项目(F2018207038);河北省高等教育教学改革研究与实践项目(2022GJJG178);河北省教育厅科研资助项目(QN2020186);河北经贸大学重点研究项目(ZD20230001) |
| |
摘 要: | 为解决基于视觉的情感识别无法捕捉人物所处环境和与周围人物互动对情感识别的影响、单一情感种类无法更丰富地描述人物情感、无法对未来情感进行合理预测的问题,提出了融合背景上下文特征的视觉情感识别与预测方法。该方法由融合背景上下文特征的情感识别模型(Context-ER)和基于GRU与Valence-Arousal连续情感维度的情感预测模型(GRU-mapVA)组成。Context-ER同时综合了面部表情、身体姿态和背景上下文(所处环境、与周围人物互动行为)特征,进行26种离散情感类别的多标签分类和3个连续情感维度的回归。GRU-mapVA根据所提映射规则将Valence-Arousal的预测值投影到改进的Valence-Arousal模型上,使得情感预测类间差异更为明显。Context-ER在Emotic数据集上进行了测试,结果表明,识别情感的平均精确率比现有最优方法提高4%以上;GRU-mapVA在三段视频样本上进行了测试,结果表明情感预测效果相较于现有方法有很大提升。
|
关 键 词: | 情感识别 背景上下文 多标签分类 GRU 情感预测 |
收稿时间: | 2023/8/9 0:00:00 |
修稿时间: | 2024/4/8 0:00:00 |
Visual emotion recognition and prediction based on fusion of background contextual features |
| |
Abstract: | To address the problems of inability to capture the impact of environmental factors and interaction with surrounding individuals on emotion recognition in vision-based affective computing, limitations of describing emotions with a single category, and inability to predict future emotions, this paper proposed a visual emotion recognition and prediction method integrating background context features. This method consisted of an emotion recognition model that integrated background context features(Context-ER) and an emotion prediction model based on GRU and continuous emotion dimensions of Valence-Arousal(GRU mapVA). Context-ER combined facial expressions, body posture, and background context(environment, interaction behavior with surrounding people) features to perform multi-label classification for 26 discrete emotion categories and regression for 3 continuous emotion dimensions. GRU mapVA projected the predicted values of Valence-Arousal onto the improved ValenceArousal model based on the proposed mapping rules, making the differences between sentiment prediction classes more pronounced. Context-ER was tested on the Emotic dataset, and the results show an average precision improvement of over 4% compared to the state-of-the-art methods. GRU-mapVA was tested on three video samples, and the results demonstrate a signi-ficant improvement in emotion prediction compared to existing methods. |
| |
Keywords: | emotion recognition background context multi-label classification GRU emotion prediction |
|
| 点击此处可从《计算机应用研究》浏览原始摘要信息 |
|
点击此处可从《计算机应用研究》下载全文 |