运用模态融合的半监督广义零样本学习

映射域漂移和偏见性预测问题使得现有的方案无法很好地应对广义零样本学习挑战.在CADA-VAE模型的基础上,提出了基于模态融合的半监督学习方案,就如何利用未标注样本及语义辅助模型进行模态内自学习提供了一种思路.该方案使用潜层向量空间作为视觉和语义模态融合的桥梁,提出了视觉质心和异类语义潜层向量概念,用以指导模态间互学习;...

Semi-supervised Generalized Zero-Shot Learning Using Modal Fusion

Projection domain drift and prejudice prediction problems make the existing schemes unable to meet the challenge of generalized zero-shot learning well.Based on the CADA-VAE,this article proposes a semi-supervised learning scheme based on modal fusion which provides a way of how to use unlabeled samples and semantic help the model for intra-modal self-learning.This solution uses the latent layer vector space as a bridge for the fusion of visual and semantic modalities,and proposes the concept of visual centroid and heterogeneous semantic latent layer vectors to guide mutual learning between modalities.In the cross-reconstruction link,the semantic latent layer vector is cross-reconstructed into visual features by taking the visual centroid as the axis;in the feature coding link,the visual feature is coded as a latent layer vector along the opposite direction of the heterogeneous semantic latent layer vector.This scheme ensures the generated samples have diversity while not losing the discrimination between classes.Comparative experiments on three benchmark data sets proves that this model is superior to the current mainstream solutions in recognition accuracy,and it can cope with the scarcity of labeled samples.