面向语句的MBFL变异体约减策略

在软件调试过程中如何高效、精确地定位程序中的错误代码是软件开发人员普遍关注的问题。MBFL是一种基于变异分析的错误定位技术,它在获得较高错误定位精度的同时会生成大量变异体,并在变异体上执行测试用例集,开销庞大。为了减少MBFL的变异执行开销,提出面向语句的变异体约减策略,通过分析测试用例的执行信息, 按一定比例 对每条由失败测试用例覆盖的语句生成的变异体集合进行约减。实验结果表明,在7个程序包的112个错误版本上,应用面向语句的变异体约减策略的MBFL,在保持较高错误定位精度的同时,能够有效减少73.51%~79.98%的变异执行开销。     

针对基于变异错误定位的一种动态变异执行策略

在软件调试过程中,如何快速、精确地定位程序中的错误代码是软件开发人员普遍关注的问题。基于变异的错误定位方法是一种通过分析被测程序与程序变异体之间的行为相似性来估计语句出错概率、进行错误定位的方法。该方法有较高的错误定位精确度,但由于需对大量程序变异体执行测试用例集,因此其变异执行开销较大。为此提出了一种动态变异执行策略,它通过搜集测试用例执行信息,动态地调整变异体及测试用例的执行顺序,以减少其变异执行开销。实验结果表明,在6个程序包的127个错误版本上,应用提出的动态变异执行策略可在保证错误定位精确度的前提下,减少23%~78%的变异执行开销,显著提高了基于变异的错误定位方法的效率。     

基于改良程序谱的软件故障定位方法

故障定位是软件调试过程中一项耗时耗力的工作,自动化查错的应用对于提高软件调试效率具有重要的现实意义。近年来,基于程序谱的故障定位方法得到了研究人员的大量关注。针对单错误现象,提出了基于改良程序谱的软件故障定位新方法,该方法基于“在单错误情况下,若测试用例运行错误,则该测试用例运行必定覆盖了故障语句”这一论断,将所有的故障测试用例对程序语句的覆盖情况做交运算,从而得到故障基,再利用故障基定位故障。最后,以西门子测试程序集为测试数据,对比了不同方法对故障定位的效果和效率的影响,其结果表明所提出的方法可以有效地提高故障定位的效果和效率。     

On the adoption of MC/DC and control-flow adequacy for a tight integration of program testing and statistical fault localization

ContextTesting and debugging consume a significant portion of software development effort. Both processes are usually conducted independently despite their close relationship with each other. Test adequacy is vital for developers to assure that sufficient testing effort has been made, while finding all the faults in a program as soon as possible is equally important. A tight integration between testing and debugging activities is essential.ObjectiveThe paper aims at finding whether three factors, namely, the adequacy criterion to gauge a test suite, the size of a prioritized test suite, and the percentage of such a test suite used in fault localization, have significant impacts on integrating test case prioritization techniques with statistical fault localization techniques.MethodWe conduct a controlled experiment to investigate the effectiveness of applying adequate test suites to locate faults in a benchmark suite of seven Siemens programs and four real-life UNIX utility programs using three adequacy criteria, 16 test case prioritization techniques, and four statistical fault localization techniques. We measure the proportion of code needed to be examined in order to locate a fault as the effectiveness of statistical fault localization techniques. We also investigate the integration of test case prioritization and statistical fault localization with postmortem analysis.ResultThe main result shows that on average, it is more effective for a statistical fault localization technique to utilize the execution results of a MC/DC-adequate test suite than those of a branch-adequate test suite, and is in turn more effective to utilize the execution results of a branch-adequate test suite than those of a statement-adequate test suite. On the other hand, we find that none of the fault localization techniques studied can be sufficiently effective in suggesting fault-relevant statements that can fit easily into one debug window of a typical IDE.ConclusionWe find that the adequacy criterion and the percentage of a prioritized test suite utilized are major factors affecting the effectiveness of statistical fault localization techniques. In our experiment, the adoption of a stronger adequacy criterion can lead to more effective integration of testing and debugging.     

基于程序频谱的动态缺陷定位方法研究

基于程序频谱的动态缺陷定位是软件自动化调试研究中的一个热点问题,通过搜集测试用例的程序频谱和执行结果,基于特定模型以定位缺陷语句在被测程序内的可能位置.对近些年来国内外学者在该研究领域取得的成果进行系统总结:首先,给出预备知识和基本假设;随后,提出缺陷定位研究框架并识别出框架内一系列可影响缺陷定位效果的内在影响因素,包括程序频谱构造方式、测试套件构成和维护、内在缺陷数量、测试用例预言设置、用户反馈和缺陷修复开销等;接着,对实证研究中采用的评测指标和评测程序进行总结和分析;然后,对缺陷定位方法在一些特定测试领域中的应用进行总结;最后,对该领域未来值得关注的研究方向进行了展望.     

分析语句命中谱的自动故障定位方法研究

软件故障的自动定位能提高测试过程的效率,对改善软件的可信性也相当重要。在原有程序谱分析故障定位基础上,提出了基于语句命中谱分析的自动故障定位方法,通过引进聚类函数克服了基于模型的故障定位计算量过大的问题,并和现有的程序谱定位工具Pinpoint、Tarantula进行了比较,最终通过实验证明了这种方法的高效性和优越性。     

Mutation-oriented test data augmentation for GUI software fault localization

ContextFault localization lies at the heart of program debugging and often proceeds by contrasting the statistics of program constructs executed by passing and failing test cases. A vital issue here is how to obtain these “suitable” test cases. Techniques presented in the literature mostly assume the existence of a large test suite a priori. However, developers often encounter situations where a failure occurs, but where no or no appropriate test suite is available for use to localize the fault.ObjectiveThis paper aims to alleviate this key limitation of traditional fault localization techniques for GUI software particularly, namely, it aims at enabling cost-effective fault localization process for GUI software in the described scenario.MethodTo address this scenario, we propose a mutation-oriented test data augmentation technique, which actually is directed by the “similarity” criterion in GUI software’s test case context towards the generation of test suite with excellent fault localization capabilities. More specifically, the technique mainly uses four proposed novel mutation operators to iteratively mutate some failing GUI test cases’ event sequences to derive new test cases potentially useful to localize the specific encountered fault. We then compare the fault localization performance of the test suite generated using this technique with that of an original provided large event-pair adequate test suite on some GUI applications.ResultsThe results indicate that the proposed technique is capable of generating a test suite that has comparable, if not better, fault localization effectiveness to the event-pair adequate test suite, but it is much smaller and it is generated immediately once a failure is encountered by developers.ConclusionIt is concluded that the proposed technique can truly enable quick-start cost-effective fault localization process under the investigated all-too-common scenario, greatly alleviating one key limitation of traditional fault localization techniques and prompting the test–diagnose–repair cycle.     

基于程序频谱的两阶段缺陷定位方法

缺陷定位是软件质量保证中关键且困难的一项工作,随着软件规模的增大,人工进行缺陷定位的成本越来越高,自动化缺陷定位技术成为研究热点。现有的基于程序频谱的缺陷定位技术可以将缺陷定位到程序语句,但对于大型复杂的软件系统,这种定位方法将带来较大的时间花销。针对此问题,提出一种基于程序频谱的两阶段缺陷定位方法,第一阶段为粗粒度定位,将缺陷定位到程序模块;第二阶段为细粒度定位,在定位的程序模块中再将缺陷定位到语句;最后输出可疑语句推荐列表,辅助开发人员的调试工作。实验结果表明,相比于传统的方法,该方案在保证定位效果的前提下平均减少了10.24%的定位时间。     

融合语句复杂度的软件错误定位轻量级方法

在程序调试过程中,基于程序谱的软件错误定位(SBFL)技术能提供有效的帮助。为改善SBFL的性能,提出一种组合程序谱、代码行静态属性的软件错误定位排序学习方法,由线性排序支持向量机学习最优错误定位模型。代码行静态属性包括局部变量、类属性、逻辑运算符和方法调用等程序实体的个数。在使用C、C++和Java语言开发的22个实际故障项目上,采用跨工程的形式训练错误定位模型。实验结果表明,新方法比最优SBFL减少了37.1%的最坏策略EXAM和22.6%的平均策略EXAM。还比较了程序语句的3类轻量级特征：结构化类别、变量谱和静态属性。新方法的时间复杂度低,能实时地推荐可能出现故障的语句序列。     

面向软件错误定位与理解的测试执行离散特征筛选

软件错误定位与错误理解是软件调试过程中的重要步骤,然而调试人员利用基于覆盖分析的软件错误定位获取的可疑度,从高到低静态分析每条程序语句的检查方式,与实际软件调试过程并不相符。为了能够筛选更有助于理解错误根源的测试执行,尤其是致使程序失效的失效执行,帮助调试人员进行动态差异化分析,针对失效执行提出基于高可疑度覆盖率、揭示错误潜力和覆盖语句可疑度离散特征的3种优先级策略,针对成功执行提出加权余弦相似度匹配策略。通过将3种失效执行优先级策略与随机选择在常用错误定位技术中进行实验对比,验证了基于覆盖语句可疑度离散特征的失效执行筛选策略能够对筛选前后的错误理解工作量变化产生更强的积极影响和更弱的消极影响,并能够在相同工作量下理解更多的错误,进而更有助于将错误定位结果应用于错误根源的理解。     

Improving spectral‐based fault localization using static analysis

Debugging is crucial for producing reliable software. One of the effective bug localization techniques is spectral‐based fault localization (SBFL). It helps to locate a buggy statement by applying an evaluation metric to program spectra and ranking program components on the basis of the score it computes. SBFL is an example of a dynamic analysis – an analysis of computer program that is performed by executing it with sufficient number of test cases. Static analysis, on the other hand, is performed in a non‐runtime environment. We introduce a weighting technique by combining these two kinds of program analysis. Static analysis is performed to categorize program statements into different classes and giving them weights based on the likelihood of being buggy statement. Statements are finally ranked on the basis of the weights computed by statements' categorization (static analysis) and scores computed by SBFL metrics (dynamic analysis). We evaluate the performance of our technique on Siemens test suite and Flex (having seeded bugs seeded by expert developers), Sed (having mixture of real and seeded bugs), and Space (having real bugs). In our evaluation, proposed weighting technique improves the performance of a wide variety of fault localization metrics up to 20% on single bug datasets and up to 42% on multi‐bug datasets. Copyright © 2017 John Wiley & Sons, Ltd.     

一种随机TBFL方法

许多学者研究了运用测试集对程序错误语句定位的问题,并提出了许多行之有效的方法,这些方法统称为TBFL(testing based fault localization)方法。后来人们发现,测试集里如果出现冗余,则这些冗余测试用例会伤害这些定位方法的功效。为了解决这个问题,Hao等人提出了SAFL(similarity aware fault localization)方法。实际上完全避免冗余是不可能的,因此从另一个角度构造了一个新的TBFL方法,称为随机TBFL方法。该方法的基本思想是:测试前对程序的语句错误概率进行先验分布,并把测试集看成随机变量,用测试用例反映的程序语句有关信息对程序语句的概率作一些调整,调整后的概率称为后验校正概率,最后根据这个后验概率对错误语句进行定位。将传统的TB-FL方法如Dicing方法、TARANTULA方法、SAFL方法纳入随机信息分析并通过几个实例进行分析和比较,结果表明,随机TBFL方法不仅能够正确定位错误语句,而且冗余对该方法的功效伤害不大。     

Statistical Debugging: A Hypothesis Testing-Based Approach

Manual debugging is tedious, as well as costly. The high cost has motivated the development of fault localization techniques, which help developers search for fault locations. In this paper, we propose a new statistical method, called SOBER, which automatically localizes software faults without any prior knowledge of the program semantics. Unlike existing statistical approaches that select predicates correlated with program failures, SOBER models the predicate evaluation in both correct and incorrect executions and regards a predicate as fault-relevant if its evaluation pattern in incorrect executions significantly diverges from that in correct ones. Featuring a rationale similar to that of hypothesis testing, SOBER quantifies the fault relevance of each predicate in a principled way. We systematically evaluate SOBER under the same setting as previous studies. The result clearly demonstrates the effectiveness: SOBER could help developers locate 68 out of the 130 faults in the Siemens suite by examining no more than 10 percent of the code, whereas the cause transition approach proposed by Holger et al. [2005] and the statistical approach by Liblit et al. [2005] locate 34 and 52 faults, respectively. Moreover, the effectiveness of SOBER is also evaluated in an "imperfect world", where the test suite is either inadequate or only partially labeled. The experiments indicate that SOBER could achieve competitive quality under these harsh circumstances. Two case studies with grep 2.2 and bc 1.06 are reported, which shed light on the applicability of SOBER on reasonably large programs     

一种基于谓词分层覆盖矩阵的缺陷定位方法

基于程序频谱的动态缺陷定位(spectrum based dynamic fault localization,简称SFL)可分为基于可执行语句覆盖的方法和基于谓词覆盖的方法。通过分析以上两类方法可以发现：(1) 基于可执行语句覆盖的方法未考虑谓词错误和执行结果之间的关联。(2)基于谓词覆盖的方法只针对谓词进行插桩,最后只计算谓词的可疑度并对谓词进行排序。如果缺陷是非谓词,此类方法无法准确定位缺陷位置。(3) 忽略了基本块之间的关联和层次特性,将各个基本块看成相互独立的个体。为解决上述问题,首先,本文将谓词错误与执行结果之间的关联性这一有用信息加入到算法的设计中;其次,加入谓词分层覆盖与分析的思想,对覆盖矩阵中的基本块进行细分和分层;最后,将二者结合,提出一种基于谓词分层覆盖矩阵的缺陷定位方法,提出了谓词分层覆盖算法Phcm。本文将西门子程序集作为目标程序,通过与其他三种缺陷定位方法进行对比实验,验证了该方法在提高缺陷定位的精准度和减小代码检查率上的有效性。     

Inforence: effective fault localization based on information-theoretic analysis and statistical causal inference

In this paper, a novel approach, Inforence, is proposed to isolate the suspicious codes that likely contain faults. Inforence employs a feature selection method, based on mutual information, to identify those bug-related statements that may cause the program to fail. Because the majority of a program faults may be revealed as undesired joint effect of the program statements on each other and on program termination state, unlike the state-of-the-art methods, Inforence tries to identify and select groups of interdependent statements which altogether may affect the program failure. The interdependence amongst the statements is measured according to their mutual effect on each other and on the program termination state. To provide the context of failure, the selected bug-related statements are chained to each other, considering the program static structure. Eventually, the resultant causeeffect chains are ranked according to their combined causal effect on program failure. To validate Inforence, the results of our experimentswith seven sets of programs include Siemens suite, gzip, grep, sed, space, make and bash are presented. The experimental results are then compared with those provided by different fault localization techniques for the both single-fault and multi-fault programs. The experimental results prove the outperformance of the proposed method compared to the state-of-the-art techniques.     

Test input reduction for result inspection to facilitate fault localization

Testing-based fault-localization (TBFL) approaches often require the availability of high-statement-coverage test suites that sufficiently exercise the areas around the faults. However, in practice, fault localization often starts with a test suite whose quality may not be sufficient to apply TBFL approaches. Recent capture/replay or traditional test-generation tools can be used to acquire a high-statement-coverage test collection (i.e., test inputs only) without expected outputs. But it is expensive or even infeasible for developers to manually inspect the results of so many test inputs. To enable practical application of TBFL approaches, we propose three strategies to reduce the test inputs in an existing test collection for result inspection. These three strategies are based on the execution traces of test runs using the test inputs. With the three strategies, developers can select only a representative subset of the test inputs for result inspection and fault localization. We implemented and applied the three test-input-reduction strategies to a series of benchmarks: the Siemens programs, DC, and TCC. The experimental results show that our approach can help developers inspect the results of a smaller subset (less than 10%) of test inputs, whose fault-localization effectiveness is close to that of the whole test collection.     

基于程序频谱的缺陷定位方法

软件测试是生产可靠软件的重要保障,对测试所发现缺陷的解决可以分为缺陷定位和缺陷修改两个步骤^[1],其中的缺陷定位是最耗时的.通常情况下,测试套件中成功执行的测试用例都占绝大多数,对基于程序频谱的缺陷定位方法,应该具备自主调节成功测试用例覆盖比重的能力,以提高方法的可用性.即,随着语句被成功测试用例覆盖的次数增多,该语句的覆盖次数对怀疑率的贡献度应逐渐减小,成功测试用例数的有效处理能提高缺陷定位方法的效果.基于此,本文提出EPStar（EP*）缺陷定位方法,该方法可以有效调整成功执行用例数的影响,以避免成功用例数量对缺陷定位效果的过度影响,从而提高缺陷定位的准确性,通过实验对比,说明了EP*方法比现有的几种缺陷定位方法具有更高的缺陷定位精度.     

A dynamic code coverage approach to maximize fault localization efficiency

Spectrum-based fault localization is amongst the most effective techniques for automatic fault localization. However, abstractions of program execution traces, one of the required inputs for this technique, require instrumentation of the software under test at a statement level of granularity in order to compute a list of potential faulty statements. This introduces a considerable overhead in the fault localization process, which can even become prohibitive in, e.g., resource constrained environments. To counter this problem, we propose a new approach, coined dynamic code coverage (DCC), aimed at reducing this instrumentation overhead. This technique, by means of using coarser instrumentation, starts by analyzing coverage traces for large components of the system under test. It then progressively increases the instrumentation detail for faulty components, until the statement level of detail is reached. To assess the validity of our proposed approach, an empirical evaluation was performed, injecting faults in six real-world software projects. The empirical evaluation demonstrates that the dynamic code coverage approach reduces the execution overhead that exists in spectrum-based fault localization, and even presents a more concise potential fault ranking to the user. We have observed execution time reductions of 27% on average and diagnostic report size reductions of 77% on average.     

A family of code coverage-based heuristics for effective fault localization

Locating faults in a program can be very time-consuming and arduous, and therefore, there is an increased demand for automated techniques that can assist in the fault localization process. In this paper a code coverage-based method with a family of heuristics is proposed in order to prioritize suspicious code according to its likelihood of containing program bugs. Highly suspicious code (i.e., code that is more likely to contain a bug) should be examined before code that is relatively less suspicious; and in this manner programmers can identify and repair faulty code more efficiently and effectively. We also address two important issues: first, how can each additional failed test case aid in locating program faults; and second, how can each additional successful test case help in locating program faults. We propose that with respect to a piece of code, the contribution of the first failed test case that executes it in computing its likelihood of containing a bug is larger than or equal to that of the second failed test case that executes it, which in turn is larger than or equal to that of the third failed test case that executes it, and so on. This principle is also applied to the contribution provided by successful test cases that execute the piece of code. A tool, χDebug, was implemented to automate the computation of the suspiciousness of the code and the subsequent prioritization of suspicious code for locating program faults. To validate our method case studies were performed on six sets of programs: Siemens suite, Unix suite, space, grep, gzip, and make. Data collected from the studies are supportive of the above claim and also suggest Heuristics III(a), (b) and (c) of our method can effectively reduce the effort spent on fault localization.     

软件故障定位技术进展

故障定位是调试过程中一项耗时费力的工作。为了降低调试成本,并辅助开发人员定位和修复软件故障,软件故障定位技术通过审查源代码、分析测试过程的软件行为和测试结果来定位包含故障的代码片段。综述了近期故障定位领域相关成就,分类介绍了各种代表性的故障定位方法的基本原理和建模技术,讨论了这些故障定位技术的贡献以及它们之间的主要区别,给出了常用的故障定位效果基准测试集和度量方法,展望了故障定位技术的研究方向。