基于主动样本精选与跨模态语义挖掘的图像情感分析

图像情感分析是机器视觉领域的研究热点,它面临的关键问题是:标注者的主观差异导致情感标签明确的高质量样本匮乏,且异构图像特征间跨模态语义未有效利用.为此,提出基于主动样本精选与跨模态语义挖掘的图像情感分析模型ASRF²(active sample refinement & feature fusion):融合主动学习与样本精选思想,设计主动样本精选策略,优选情感标签明确的样本;对异构图像特征执行判别相关分析,生成能准确刻画图像情感内容的低维跨模态语义;采用跨模态语义训练Catboost模型,实现图像情感分析.在TwitterI与FI数据集上验证ASRF²模型,识别准确率分别达90.06%和75.77%,优于主流基线且实时效率良好.与基线相比,ASRF²模型仅需两类特征,参数调制简单,更易复现.ASR策略还具备一定的泛化性,可为基线模型提供优质训练样本,以改善识别性能.

Image sentiment analysis via active sample refinement and cross-modal semantics mining

Image sentiment analysis is a research focus in the field of computer vision. However, we are faced with the following key problems: First, owing to the subjective differences of different annotators, high-quality samples with definite sentimental annotations are very scarce. Second, the implicit cross-modal semantics among heterogeneous features has not been fully explored. To address these two problems, we propose an active sample refinement & feature fusion (ASRF²) via active sample refinement and cross-modal semantics mining: an active sample refinement strategy is designed by fusing the active learning and sample refinement ideas. High-quality samples with definite sentimental annotations are obtained in turn. Then, the state-of-the-art discriminant correlation analysis (DCA) algorithm is employed to fully mine the cross-modal correlations among the heterogeneous features. Low-dimensional but more discriminant cross-modal semantics that can better depict the key sentimental contents of images are generated. The cross-modal semantics is used to train a Catboost classifier and complete image sentiment analysis. We validate the proposed ASRF² model on the TwitterI and FI datasets. The corresponding accuracies reach about 90.06% and 75.77%, respectively, which outperform other state-of-the-art baselines as well as the real-time efficiency. Compared with the baselines, the proposed model only needs two image features, and it is easy to tune and reproduced the ASRF² model. Moreover, the ASR strategy is robust, which can offer many more high-quality samples for the baselines to improve the final recognition performance.