个性化图像美学评价的研究进展与趋势

图像美学评价方法是当前研究的热点问题。图像美学评价分为大众化和个性化两种。大众化图像美学评价主要研究大多数人对图像共同的审美感知评估,而个性化图像美学评价可以针对用户的个性化审美感知进行评估。现有的研究工作主要集中在大众化图像美学评价上,但是由于人们对图像的审美体验具有高度主观性,研究针对特定用户的个性化图像美学评价方法更加符合现实意义。目前研究人员针对个性化图像美学评价展开了相关研究,并取得了一定的研究进展。但是现有的文献中缺少对个性化图像美学评价方法的综述,本文针对个性化图像美学评价的研究进展与趋势进行概述。首先分析图像美学评价的研究现状与发展趋势;然后针对现阶段的个性化图像美学评价模型进行概述,将现有的个性化图像美学评价模型总结为基于协同过滤的模型、基于用户交互的模型和基于审美差异的模型,并分析这3类模型主要的设计思路以及优缺点;最后介绍个性化图像美学评价在精准营销、个性化推荐系统、个性化视觉增强和个性化艺术设计上的应用前景,并指出未来研究工作在主观特性分析和知识驱动建模等方面的发展方向。

The review of personalized image aesthetics assessment

The multimedia imaging technology can meet people''s visual demands to a certain extent. People can easily obtain high-quality images through mobile devices, so people begin to pay more attention to their aesthetic experience of images, which makes the image aesthetics assessment (IAA) method become a hotspot issue and frontier technology in the current image processing and computer vision fields. Intelligent IAA can be developed to imitate people''s aesthetic perception of images and predict the results of aesthetic evaluation automatically. Aesthetic-preference images can be screened out. Consequently, IAA is critical to be applied in photography, beauty, photo album management, interface design, and marketing. Generally, IAA can be classified into two categories, including generic image aesthetics assessment (GIAA) and personalized image aesthetics assessment (PIAA). Early researches believe that people have a consensus on the aesthetic experience of images, and leverage the general photography rules to describe most people''s visual aesthetics on images, which are usually affected by many factors, such as light intensity, color richness, and composition. Most of the current GIAA model can predict most people''s aesthetic evaluation results of images. GIAA models can be divided into three aesthetic-related tasks like classification, score regression and distribution prediction. The aesthetic classification task is focused on dividing the image into two classes of "high" and "low" according to the aesthetic experience of most people. The research goal of the aesthetic score regression task can predict the aesthetic score of an image. This task leverages the average aesthetic ratings of most people as the image aesthetic score for regression modeling. However, the two tasks shown above need to convert different people''s aesthetic ratings of images into a unified result ("high" or "low" and score). Label uncertainty is derived from people''s aesthetic experience of images, which makes it difficult for the consensus result. Therefore, the predictable aesthetic distribution is more concerned to reflect people''s subjectivity. The goal of the aesthetic distribution prediction task can predict the aesthetic distribution results of multiple people''s ratings of an image. This task predicts the aesthetic distribution straightforward and converts the aesthetic distribution result into aesthetic scores and aesthetic classes. Consequently, recent GIAA models researches are mainly focused on the task of aesthetic distribution prediction. Although the aesthetic distribution prediction task of the GIAA model can reflect people''s subjectivity of image aesthetics to a certain extent, the task can realize people''s visual aesthetic preferences from the image level only. Besides, it is more realistic to develop the PIAA model for specification beneficial from the growth of customized services. Therefore, the PIAA model has received great attention recently, which can predict the accurate aesthetic results for customized users. We introduce the existing PIAA models published from 2014 to 2020 due to the lack of reviews on PIAA models. Generally, the PIAA model faces two challenges for specific users as mentioned below:First, PIAA is a typical small sample learning task. This is because the PIAA model is a real-time system for specific users, which cannot force users to annotate a large number of images aesthetically. In addition, a small number of image samples can just be obtained for model training. Second, the user''s subjective characteristics become important factors to affect their aesthetic perception of images since the user''s aesthetic experience of images is highly subjective. Meanwhile, users'' aesthetic experience is influenced by many subjective factors like emotion and personality traits. Therefore, the existing framework of the PIAA model is mainly divided into two stages based on machine learning or deep learning. In the first stage, the GIAA dataset rated by a large number of users is used to obtain the prior knowledge of the PIAA model through supervision training for the smalls sized sample learning issue of the PIAA task. In the second stage, a user''s PIAA dataset is used for fine-tuning on the prior knowledge model for the high subjectivity of users'' image aesthetic experience, and the subjective knowledge of users is integrated to obtain the PIAA model that conforms to the user''s aesthetic perception. The existing PIAA models can be divided into three categories like collaborative filtering based PIAA models, user interaction based PIAA models, and aesthetic difference based PIAA models. To demonstrate the differences between these three PIAA models, we first introduce each of the three PIAA models separately. Then, we summarize the design clues, pros and cons of existing PIAA models. Meanwhile, we introduce the commonly used datasets and evaluation criteria of PIAA models, as well as the application prospect of PIAA models in precision marketing, personalized recommendation systems, personalized visual enhancement, and personalized art design. Finally, the future direction of the PIAA model is predicted in subjective characteristic analysis and knowledge-driven modeling.