IATG: 基于解释分析的自动驾驶软件测试方法

以深度神经网络(deep neural network, DNN)为基础构建的自动驾驶软件已成为最常见的自动驾驶软件解决方案. 与传统软件一样, DNN也会产生不正确输出或意想不到的行为, 基于DNN的自动驾驶软件已经导致多起严重事故, 严重威胁生命和财产安全. 如何有效测试基于DNN的自动驾驶软件已成为亟需解决的问题. 由于DNN的行为难以预测和被人类理解, 传统的软件测试方法难以适用. 现有的自动驾驶软件测试方法通常对原始图片加入像素级的扰动或对图片整体进行修改来生成测试数据, 所生成的测试数据通常与现实世界差异较大, 所进行扰动的方式也难以被人类理解. 为解决上述问题, 提出测试数据生成方法IATG (interpretability-analysis-based test data generation), 使用DNN的解释方法获取自动驾驶软件所做出决策的视觉解释, 选择原始图像中对决策产生重要影响的物体, 通过将其替换为语义相同的其他物体来生成测试数据, 使生成的测试数据更加接近真实图像, 其过程也更易于理解. 转向角预测模型是自动驾驶软件决策模块重要组成部分, 以此类模型为例进行实验, 结果表明解释方法的引入有效增强IATG对转向角预测模型的误导能力. 此外, 在误导角度相同时IATG所生成测试数据比DeepTest更加接近真实图像; 与semSensFuzz相比, IATG具有更高误导能力, 且IATG中基于解释分析的重要物体选择技术可有效提高semSensFuzz的误导能力.

IATG: Interpretation-analysis-based Testing Method for Autonomous Driving Software

Autonomous driving software based on deep neural network (DNN) has become the most popular solution. Like traditional software, DNN can also produce incorrect output or unexpected behaviors, and DNN-based autonomous driving software has caused serious accidents, which seriously threaten life and property safety. Therefore, how to effectively test DNN-based autonomous driving software has become an urgent problem. Since it is difficult to predict and understand the behavior of DNNs, traditional software testing methods are no longer applicable. Existing autonomous driving software testing methods are implemented byadding pixel-level perturbations to original images or modifying the whole image to generate test data. The generated test data are quite different from the real images, and the perturbation-based methods are difficult to be understood. To solve the above problem, this study proposes a test data generation method, namely interpretability-analysis-based test data generation (IATG). Firstly, it uses the interpretation method for DNNs to generate visual explanations of decisions made by autonomous driving software and chooses objects in the original images that have significant impacts on the decisions. Then, it generates test data by replacing the chosen objects with other objects with the same semantics. The generated test data are more similar to the real image, and the process is more understandable. As an important part of the autonomous driving software’s decision-making module, the steering angle prediction model is used to conduct experiments. Experimental results show that the introduction of the interpretation method effectively enhances the ability of IATG to mislead the steering angle prediction model. Furthermore, when the misleading angle is the same, the test data generated by IATG are more similar to the real image than DeepTest; IATG has a stronger misleading ability than semSensFuzz, and the interpretation analysis based important object selection method of IATG can effectively improve the misleading ability of semSensFuzz.