基于改良程序谱的软件故障定位方法

故障定位是软件调试过程中一项耗时耗力的工作,自动化查错的应用对于提高软件调试效率具有重要的现实意义。近年来,基于程序谱的故障定位方法得到了研究人员的大量关注。针对单错误现象,提出了基于改良程序谱的软件故障定位新方法,该方法基于“在单错误情况下,若测试用例运行错误,则该测试用例运行必定覆盖了故障语句”这一论断,将所有的故障测试用例对程序语句的覆盖情况做交运算,从而得到故障基,再利用故障基定位故障。最后,以西门子测试程序集为测试数据,对比了不同方法对故障定位的效果和效率的影响,其结果表明所提出的方法可以有效地提高故障定位的效果和效率。     

基于程序频谱的两阶段缺陷定位方法

缺陷定位是软件质量保证中关键且困难的一项工作,随着软件规模的增大,人工进行缺陷定位的成本越来越高,自动化缺陷定位技术成为研究热点。现有的基于程序频谱的缺陷定位技术可以将缺陷定位到程序语句,但对于大型复杂的软件系统,这种定位方法将带来较大的时间花销。针对此问题,提出一种基于程序频谱的两阶段缺陷定位方法,第一阶段为粗粒度定位,将缺陷定位到程序模块;第二阶段为细粒度定位,在定位的程序模块中再将缺陷定位到语句;最后输出可疑语句推荐列表,辅助开发人员的调试工作。实验结果表明,相比于传统的方法,该方案在保证定位效果的前提下平均减少了10.24%的定位时间。     

IPSETFUL: an iterative process of selecting test cases for effective fault localization by exploring concept lattice of program spectra

Fault localization is an important and challenging task during software testing. Among techniques studied in this field, program spectrum based fault localization is a promising approach. To perform spectrum based fault localization, a set of test oracles should be provided, and the effectiveness of fault localization depends highly on the quality of test oracles. Moreover, their effectiveness is usually affected when multiple simultaneous faults are present. Faced with multiple faults it is difficult for developers to determine when to stop the fault localization process. To address these issues, we propose an iterative fault localization process, i.e., an iterative process of selecting test cases for effective fault localization (IPSETFUL), to identify as many faults as possible in the program until the stopping criterion is satisfied. It is performed based on a concept lattice of program spectrum (CLPS) proposed in our previous work. Based on the labeling approach of CLPS, program statements are categorized as dangerous statements, safe statements, and sensitive statements. To identify the faults, developers need to check the dangerous statements. Meantime, developers need to select a set of test cases covering the dangerous or sensitive statements from the original test suite, and a new CLPS is generated for the next iteration. The same process is proceeded in the same way. This iterative process ends until there are no failing tests in the test suite and all statements on the CLPS become safe statements. We conduct an empirical study on several subject programs, and the results show that IPSETFUL can help identifymost of the faults in the program with the given test suite. Moreover, it can save much effort in inspecting unfaulty program statements compared with the existing spectrum based fault localization techniques and the relevant state of the art technique.     

Combining mutation and fault localization for automated program debugging

This paper proposes a strategy for automatically fixing faults in a program by combining the ideas of mutation and fault localization. Statements ranked in order of their likelihood of containing faults are mutated in the same order to produce potential fixes for the faulty program. The proposed strategy is evaluated using 8 mutant operators against 19 programs each with multiple faulty versions. Our results indicate that 20.70% of the faults are fixed using selected mutant operators, suggesting that the strategy holds merit for automatically fixing faults. The impact of fault localization on efficiency of the overall fault-fixing process is investigated by experimenting with two different techniques, Tarantula and Ochiai, the latter of which has been reported to be better at fault localization than Tarantula, and also proves to be better in the context of fault-fixing using our proposed strategy. Further experiments are also presented to evaluate stopping criteria with respect to the mutant examination process and reveal that a significant fraction of the (fixable) faults can be fixed by examining a small percentage of the program code. We also report on the relative fault-fixing capabilities of mutant operators used and present discussions on future work.     

An evaluation of pure spectrum-based fault localization techniques for large-scale software systems

Pure spectrum-based fault localization (SBFL) is a well-studied statistical debugging technique that only takes a set of test cases (some failing and some passing) and their code coverage as input and produces a ranked list of suspicious program elements to help the developer identify the location of a bug that causes a failed test case. Studies show that pure SBFL techniques produce good ranked lists for small programs. However, our previous study based on the iBugs benchmark that uses the Aspect J repository shows that, for realistic programs, the accuracy of the ranked list is not suitable for human developers. In this paper, we confirm this based on a combined empirical evaluation with the iBugs and the Defects4 J benchmark. Our experiments show that, on average, at most ∼40%, ∼80%, and ∼90% of the bugs can be localized reliably within the first 10, 100, and 1000 ranked lines, respectively, in the Defects4 J benchmark. To reliably localize 90% of the bugs with the best performing SBFL metric D^∗, ∼450 lines have to be inspected by the developer. For human developers, this remains unsuitable, although the results improve compared with the results for the Aspect J benchmark. Based on this study, we can clearly see the need to go beyond pure SBFL and take other information, such as information from the bug report or from version history of the code lines, into consideration.     

软件故障定位技术研究综述

软件故障定位旨在利用程序信息以及测试用例找到导致程序出现故障的语句,以提高程序的安全性与健壮性。首先介绍了常见的单故障定位技术和现有的多故障定位技术,并对比分析了两者之间的差异,然后介绍了常用的测试用例集合,并列举了一些用来测试评价故障定位技术效率的方法,最后对现有的故障定位技术进行总结并提出未来研究方向的展望。     

Simulink fault localization: an iterative statistical debugging approach

Debugging Simulink models presents a significant challenge in the embedded industry. This paper proposes SimFL, a fault localization approach for Simulink models by combining statistical debugging and dynamic model slicing. Simulink models, being visual and hierarchical, have multiple outputs at different hierarchy levels. Given a set of outputs to observe for localizing faults, we generate test execution slices, for each test case and output, of the Simulink model. In order to further improve fault localization accuracy, we propose iSimFL, an iterative fault localization algorithm. At each iteration, iSimFL increases the set of observable outputs by including outputs at lower hierarchy levels, thus increasing the test oracle cost but offsetting it with significantly more precise fault localization. We utilize a heuristic stopping criterion to avoid unnecessary test oracle extension. We evaluate our work on three industrial Simulink models from Delphi Automotive. Our results show that, on average, SimFL ranks faulty blocks in the top 8.9% in the list of suspicious blocks. Further, we show that iSimFL significantly improves this percentage down to 4.4% by requiring engineers to observe only an average of five additional outputs at lower hierarchy levels on top of high‐level model outputs. Copyright © 2016 John Wiley & Sons, Ltd.     

组合认知复杂度的程序谱软件错误定位方法

基于程序谱的软件错误定位(spectrum-based fault localization,SBFL)技术收集测试用例结果和语句覆盖信息,用以计算每条语句的可疑度值.认知复杂度是软件复杂性度量工具,其值高的代码较易出错.为提升错误定位性能,提出一种语句级认知复杂度和SBFL相组合的方法对语句排序.当多条语句可疑度值相等时,新方法优先检查认知复杂度高的语句.测试数据集有925个错误版本,包含Java、C和C++项目.实验结果证实,加入认知复杂度后,传统的SBFL技术能减少待排查语句.     

基于线性分类算法的软件错误定位模型

基于谱的错误定位（SBFL）方法能帮助程序员减小软件调试的困难。作为一种轻量方法,SBFL只需收集测试用例的覆盖信息和测试结果,计算程序每条语句的运行特征。众多SBFL方法,将四个运行特征组合成不同的可疑度计算公式。然而,这些公式受固定参数的影响,无法适应不同的程序集。因此,提出一种机器学习方法,能自动确定特定程序集的可疑度计算公式。首先,收集已标注错误语句的程序旧版本;再将错误语句与正确语句的运行特征两两相减,构造为训练集的一个样本;最后基于Weka的分类算法,学习到线性函数,作为该程序的错误定位模型。在Siemens程序包、space和gzip三个基准数据集上,使用Logistic、SGD、SMO和LibLinear学习到的模型,性能都要优于SBFL方法。     

Model-based fault localization in bottling plants

The bottling of beverages is carried out in complex plants that consist of several machines and material flows. To realize an efficient bottling process and high quality products, operators try to avoid plant downtimes. With actual non-productive times of between 10% and 60%, the operators require diagnosis tools that allow them to locate plant components that cause downtime by exploiting automatically acquired machine data.This paper presents a model-based solution for automatic fault diagnosis in bottling plants. There are currently only a few plant-specific solutions (based on statistical calculations or artificial neural networks) for automatic bottling plant diagnosis. In order to develop a customizable solution, we followed the model-based diagnosis approach which allows the automatic generation of diagnosis solutions for individual plants. The existing stochastic and discrete-event models for bottling plants are not adequate for model-based diagnosis. Therefore, we developed new first-principle models for the relevant plant components, validated them numerically, and abstracted them to qualitative diagnosis models. Based on the diagnosis engine OCC’M Raz’r, application systems for two real plants and one virtual plant (based on discrete-event simulation) were generated and evaluated. Compared to the reasons for downtime identified by experts, we obtained up to 87.1% of compliant diagnosis results. The diagnosis solution was tested by practitioners and judged as a useful tool for plant optimization.     

TQM implementation factors for software development: an empirical study

Software Quality Journal - There is a need for methodologies which can enhance the quality of software and improve software developmental efficiencies. One such approach commonly used by software...     

More efficient automatic repair of large-scale programs using weak recompilation

Automatically repairing a bug can be a time-consuming process especially for large-scale programs owing to the significant amount of time spent recompiling and reinstalling the patched program.To reduce this time overhead and speed up the repair process,in this paper we present a recompilation technique called weak recompilation.In weak recompilation,we assume that a program consists of a set of components,and for each candidate patch only the altered components are recompiled to a shared library.The original program is then dynamically updated by a function indirection mechanism.The advantage of weak recompilation is that redundant recompilation cost can be avoided,and while the reinstallation cost is completely eliminated as the original executable program is not modified at all.For maximum applicability of weak recompilation we created WAutoRepair,a scalable system for fixing bugs with high efficiency in large-scale C programs.The experiments on real bugs in widely used programs show that our repair system significantly outperforms Genprog,a wellknown approach to automatic program repair.For the wireshark program containing over 2 million lines of code,WAutoRepair is over 128 times faster in terms of recompilation cost than Genprog.     

Forecasting technical debt evolution in software systems: an empirical study

Technical debt is considered detrimental to the long-term success of software development, but despite the numerous studies in the literature, there are still many aspects that need to be investigated for a better understanding of it. In particular, the main problems that hinder its complete understanding are the absence of a clear definition and a model for its identification, management, and forecasting. Focusing on forecasting technical debt, there is a growing notion that preventing technical debt build-up allows you to identify and address the riskiest debt items for the project before they can permanently compromise it. However, despite this high relevance, the forecast of technical debt is still little explored. To this end, this study aims to evaluate whether the quality metrics of a software system can be useful for the correct prediction of the technical debt. Therefore, the data related to the quality metrics of 8 different open-source software systems were analyzed and supplied as input to multiple machine learning algorithms to perform the prediction of the technical debt. In addition, several partitions of the initial dataset were evaluated to assess whether prediction performance could be improved by performing a data selection. The results obtained show good forecasting performance and the proposed document provides a useful approach to understanding the overall phenomenon of technical debt for practical purposes.     

一种新型的故障检测定位方法研究

针对大型复杂系统故障定位耗时长等问题,在分析故障树特征的基础上,改进了遗传算法,设计了基于故障树底事件的故障检测定位方法。该方法将顺序检测融入到改进的遗传算法检测当中,在工程应用领域体现了一定的优越性。仿真实验结果表明,该方法能够准确定位故障,有助于提高故障检测定位的效率。     

多层信息网络故障定位综述

本文对多层网络（覆盖网、虚拟网等）故障定位问题进行了分析和总结。讨论了多层网络探测故障信息获取策略和故障定位模型的发展状况,即介绍了被动监测、主动探测、主被动结合探测和终端用户观察等探测信息获取策略,以及基于图论故障传播模型、依赖矩阵模型、症状-故障-行动模型等故障传播模型的原理以及其优缺点。从故障定位模型、探测信息获取策略、故障定位计算、网络异构性、运行效率与成本多个方面重点综述了多层网络故障定位方法,讨论了每种方法的优点和局限性。最后,对多层网络故障定位研究的不足和亟待进一步研究解决的问题进行了探讨。     

多粒度光网络故障定位

为解决多种粒度光通道所带来的光交换系统中新的故障定位问题,通过对具有多粒度交换能力的光节点进行分析,依据业务请求所建立的光路在节点内以及节点之间的传输路径,提出了信道模型。将二叉树算法应用于该模型中,从而可对不同部件所产生的告警信息进行处理,实现故障的定位。实例仿真计算结果表明,该算法对于故障点的定位精度高、时间短,并且对网络的拓扑没有特殊要求。     

Sequential variable neighborhood descent variants: an empirical study on the traveling salesman problem

In a single local search algorithm, several neighborhood structures are usually explored. The simplest way is to define a single neighborhood as the union of all predefined neighborhood structures; the other possibility is to make an order (or sequence) of the predefined neighborhoods, and to use them in the first improvement or the best improvement fashion, following that order. In this work, first we classify possible variants of sequential use of neighborhoods and then, empirically analyze them in solving the classical traveling salesman problem (TSP). We explore the most commonly used TSP neighborhood structures, such as 2‐opt and insertion neighborhoods. In our empirical study, we tested 76 different such heuristics on 15,200 random test instances. Several interesting observations are derived. In addition, the two best of 76 heuristics (used as local searches within a variable neighborhood search) are tested on 23 test instances taken from the TSP library (TSPLIB). It appears that the union of neighborhoods does not perform well.     

Effect of fault distribution and execution patterns on fault exposure in software: a simulation study

Execution patterns and fault distribution characteristics of a program will affect the failure process and thus reliability estimates. The failure process of a software system is influenced by many factors, and traditional software reliability engineering has found it difficult to isolate the effect of each individual factor. A simulation approach is used to investigate the effects of fault distribution, execution pattern and program structure on software reliability estimates. A reliability simulation environment (RSIM) is extended by introducing variable fault distribution patterns in its code generation phase. Flow control points allow varying the execution frequency of different parts of a program. The simulation results show that fault distribution patterns and execution patterns have dramatic effects on fault exposure rate. If the fault distribution is non‐uniform, a non‐uniform code execution exposes faults more efficiently and effectively than uniform execution. Results also show that the structure of a program affects fault exposure rate and testing time required. Copyright © 2000 John Wiley & Sons, Ltd.     

神经程序修复领域数据泄露问题的实证研究

修复软件缺陷是软件工程领域一个无法回避的重要问题,而程序自动修复技术则旨在自动、准确且高效地修复存在缺陷的程序,以缓解软件缺陷所带来的问题.近年来,随着深度学习的快速发展,程序自动修复领域兴起了一种使用深度神经网络去自动捕捉缺陷程序和其补丁之间关系的方法,被称为神经程序修复.从在基准测试上被正确修复的缺陷的数量上看,神经程序修复工具的修复性能已经显著超过了非学习的程序自动修复工具.然而,近期有研究发现神经程序修复系统性能的提升可能得益于测试数据在训练数据中存在,即数据泄露.受此启发,为进一步探究神经程序修复系统数据泄露的原因及影响,更公平地评估现有的系统,本文(1)对现有神经程序修复系统进行了系统的分类和总结,根据分类结果定义了神经程序修复系统的数据泄露,并为每个类别的系统设计了数据泄露的检测方法;(2)依照上一步骤中的数据泄露检测方法对现有模型展开了大规模检测,并探究了数据泄露对模型真实性能与评估性能间差异的影响以及对模型本身的影响;(3)分析现有神经程序修复系统数据集的收集和过滤策略,加以改进和补充,在现有流行的数据集上基于改进后的策略构建了一个纯净的大规模程序修复训练数据集,并验证了该数据集避免数据泄露的有效性.实验结果发现:本次调研的10个神经程序修复系统在基准测试集上均出现了数据泄露,其中神经程序修复系统RewardRepair的数据泄露问题较为严重,在基准测试集Defects4J(v1.2.0)上的数据泄露达24处,泄露比例高达53.33%.此外,数据泄露对神经程序修复系统的鲁棒性也造成了影响,调研的5个神经程序修复系统均因数据泄露产生了鲁棒性降低的问题.由此可见,数据泄露是一个十分常见的问题,且会使神经程序修复系统得到不公平的性能评估结果,以及影响系统本身的性质.研究人员在训练神经程序修复模型时,应尽可能避免出现数据泄露,且要考虑数据泄露问题对神经程序修复系统性能评估产生的影响,尽可能更公平地评估系统.     

Evaluating the usage of predefined interactive behaviors for writing user stories: an empirical study with potential product owners

Cognition, Technology & Work - Behavior-driven development (BDD) has gained popularity in agile software development as a means of specifying user requirements through user stories and...