视觉特征对比解耦的广义零样本学习

广义零样本学习通常利用在ImageNet上预训练的深度模型来提取相应的视觉特征，然而预训练模型提取到的视觉特征不可避免地包含和语义无关的信息，这将导致语义—视觉对齐的偏差以及对不可见类的负迁移，从而影响分类结果。为解决上述问题，提出了视觉特征对比解耦的广义零样本学习模型(visual feature contrast decoupling for generalized zero-shot learning, VFCD-GZSL),通过解耦出视觉特征中的语义相关表示来降低冗余信息对分类结果的影响。具体来说，首先用条件变分自编码器生成不可见类的视觉特征。然后通过解耦模块将视觉特征解耦语义相关和语义无关的潜层表示，同时添加总相关惩罚和对比损失来鼓励两者间的相互独立，并用语义关系匹配模型衡量其语义一致性，从而指导模型学习语义相关表示。最后使用特征细化模块细化后的特征和语义相关表示联合学习一个广义零样本学习分类器。在四个数据集上的实验均取得较优的结果，证实了所提方法的有效性。

Visual feature contrast decoupling for generalized zero-shot learning

Generalized zero-shot learning usually uses the deep model pre-trained on ImageNet to extract corresponding visual features. However, visual features extracted by the pre-trained model inevitably contain semantically irrelevant information, which will lead to the deviation of semantic-visual alignment and negative transfer to unseen classes, thus affecting the classification results. To solve the above problems, this paper proposed a generalized zero-shot learning model for visual feature contrast decoupling, which reduced the impact of redundant information on classification results by decoupling out the semantic-related representation of visual features. Specifically, conditional variational auto-encoder firstly generated the visual features of unseen classes. Then decoupling module decoupled them into semantic-related and semantic-unrelated latent representations. Meanwhile, it applied total correlation penalty and contrastive loss to encourage the mutual independence of the two representations, and used semantic relationship matching model to measure its semantic consistency and thus guiding the model to learn semantic-related representations. Finally, it used features refined by feature refinement module and semantic-related representations to jointly learn a GZSL classifier. The experiments of the proposed model on all four data sets yielded superior results, confirming the effectiveness of the proposed method.