可解释的视觉问答研究进展

在视觉问答(VQA)任务中,“可解释”是指在特定的任务中通过各种方法去解释模型为什么有效。现有的一些VQA模型因为缺乏可解释性导致模型无法保证在生活中能安全使用,特别是自动驾驶和医疗相关的领域,将会引起一些伦理道德问题,导致无法在工业界落地。主要介绍视觉问答任务中的各种可解释性实现方式,并分为了图像解释、文本解释、多模态解释、模块化解释和图解释五类,讨论了各种方法的特点并对其中的一些方法进行了细分。除此之外,还介绍了一些可以增强可解释性的视觉问答数据集,这些数据集主要通过结合外部知识库、标注图片信息等方法来增强可解释性。对现有常用的视觉问答可解释方法进行了总结,最后根据现有视觉问答任务中可解释性方法的不足提出了未来的研究方向。

Research advances in explainable visual question answering

In the context of visual question answering(VQA) tasks, "explainability" refers to the various ways in which researchers can explain why a model works in a given task. The lack of explainability of some existing VQA models has led to a lack of assurance that the models can be used safely in real-life applications, especially in fields such as autonomous driving and healthcare. This would raise ethical and moral issues that hinder their implementation in industry. This paper introduced various implementations for enhancing explainability in VQA tasks and categorized them into four main categories: image interpretation, text interpretation, multi-modal interpretation, modular interpretation, and graph interpretation. This paper discussed the characteristics of each approach, and further presented the subdivisions for some of them. Furthermore, it presented several VQA datasets that aimed to enhance explainability. These datasets primarily focused on incorporating external knowledge bases and annotating image information to improve explainability. In summary, this paper provided an overview of existing commonly used interpretable methods for VQA tasks and proposed future research directions based on the identified shortcomings of the current approaches.