Deep-SBFL:基于频谱的深度神经网络缺陷定位方法

深度神经网络已经在自动驾驶和智能医疗等领域取得了广泛的应用.与传统软件一样,深度神经网络也不可避免地包含缺陷,如果做出错误决定,可能会造成严重后果.因此,深度神经网络的质量保障受到了广泛关注.然而,深度神经网络与传统软件存在较大差异,传统软件质量保障方法无法直接应用于深度神经网络,需要设计有针对性的质量保障方法.软件缺陷定位是保障软件质量的重要方法之一,基于频谱的缺陷定位方法在传统软件的缺陷定位中取得了很好的效果,但无法直接应用于深度神经网络.在传统软件缺陷定位方法的基础上提出了一种基于频谱的深度神经网络缺陷定位方法 Deep-SBFL.该方法首先通过收集深度神经网络的神经元输出信息和预测结果作为频谱信息;然后将频谱信息进行处理作为贡献信息,以用于量化神经元对预测结果所做的贡献;最后提出了针对深度神经网络缺陷定位的怀疑度公式,基于贡献信息计算深度神经网络中神经元的怀疑度并进行排序,以找出最有可能存在缺陷的神经元.为验证该方法的有效性,以E_Inspect@n (结果排序列表前n个位置内成功定位的缺陷数)和EXAM (在找到缺陷元素之前必须检查元素的百分比)作为评测指...

Deep-SBFL: Spectrum-based Fault Localization Approach for Deep Neural Networks

Deep neural networks have been widely used in fields such as autonomous driving and smart healthcare. Like traditional software, deep neural networks inevitably contain defects, and it may cause serious consequences if they make wrong decisions. Therefore, the quality assurance of deep neural networks has received extensive attention. However, deep neural networks are quite different from traditional software. Traditional software quality assurance methods cannot be directly applied to deep neural networks, and targeted quality assurance methods need to be designed. Software fault localization is one of the important methods to ensure software quality. The spectrum-based fault localization method has achieved good results in traditional software fault localization methods, but it cannot be directly applied to deep neural networks. In this study, based on the traditional software fault localization methods, a spectrum-based fault localization approach named Deep-SBFL for deep neural network is proposed. The approach firstly collects the neuron output information and the prediction results of deep neural network as the spectrum. The spectrum is then further calculated as the contribution information, which can be used to quantify the contribution of neurons to the predicted results. Finally, a suspicious formula for the defect localization of deep neural network is proposed. Based on the contribution information, the suspiciousness scores of neurons in deep neural network are calculated and ranked to find out the most likely defective neurons. To verify the effectiveness of the method, E_Inspect@n (the number of defects successfully located by inspecting the first n positions of the sorted list) and EXAM (the percentage of elements that must be checked before finding defect elements) are evaluated on a deep neural network trained by the MNIST data set. Experimental results show that this approach can effectively locate different types of defects in deep neural networks.