面向有效错误定位的测试用例优选方法

针对已有测试用例选择方法在提高错误定位有效性方面存在局限性的问题,首先,定义"失效覆盖向量相似度优先排序"准则,将执行路径与失效执行路径相似的成功测试用例赋予较高的优先级;然后定义"失效覆盖等价划分优化选择"准则,选择能够最大区分失效执行语句的成功测试用例集合;在此基础上,建立测试用例优选模型(effective selection,ES).不同于已有方法,ES充分利用失效执行路径来提高错误定位的有效性.该模型被应用于优选Siemens测试用例集合,其结果被应用于Tarantula等4种错误定位方法.结果表明,ES在约简率Reduction和衡量错误定位有效性的Expense_increase两个指标方面,均优于已有的基于语句和基于向量的测试用例约简方法.ES不但可以获得97%以上的约简率,提高错误定位的效率,而且具有较低的Expense_increase,显著提高了错误定位的有效性.     

结合用例约简与联合依赖概率建模的错误定位

现有的测试用例约简方法不能有效提高错误定位精度,现有的软件错误定位方法不能充分分析元素间的依赖关系.针对以上问题,提出结合测试用例约简和联合依赖概率建模的软件错误自动定位方法,将测试用例约简与软件错误定位统一为一个整体.不同于一般的测试用例约简方法,所提出的测试用例约简方法在程序执行路径的基础上充分考虑了错误测试用例对错误定位的影响,能够为错误定位提供有效的测试用例,为快速、准确地定位软件错误奠定基础.定义了一种新的统计模型——联合依赖概率模型,充分分析了程序元素间的控制依赖、数据依赖以及语句执行状态,并提出基于联合依赖概率模型的错误自动定位方法.通过计算联合依赖关系的可疑度,对可疑节点进行排序,准确定位错误语句.实验结果表明：与SBI,SOBER,Tarantula,SF和RankCP方法相比,该算法可以更加有效地定位软件错误.     

基于深度卷积网络的多错误定位方法

目前的错误定位方法大多数解决的是单错误定位.然而,错误之间是相互关联的,如何找到这些语句与测试结果之间的关联关系和错误之间的关联关系,并减轻偶然性正确的测试用例和相似测试用例对语句可疑度的影响,对提高多错误定位的效率至关重要.为了解决以上问题,提出了基于深度卷积网络的多错误定位方法,通过一种特殊结构的深度卷积网络得到一组准确度比较高的语句可疑度,再将其应用于前向切片和后向切片中,寻找到错误与错误之间的关联定位多错误.实验表明,所提方法的多错误定位效率高于目前存在的经典的错误定位方法的错误定位效率.     

通过增大边际权重提高基于频谱的错误定位效率

基于频谱的错误定位技术通常利用覆盖信息来求出程序中每条语句的可疑度,并将语句按照可疑度降序排序以寻找错误语句.文中对已有的基于频谱的错误定位算法进行改进,将失败测试用例的边际权重引入到可疑度计算的过程中,即针对某一特定语句,令失败测试用例的权重随着其对该语句覆盖次数的增加而增大.实验结果表明,相对于其它方法,文中提出的方法对错误定位效率有一定的促进作用,即只需检查更少的语句即可找到出错位置.     

基于程序变异分析的软件错误定位

基于覆盖的错误定位(CBFL)方法通过获取成功和失败测试用例的覆盖信息和执行结果对程序中的错误进行定位,但该方法未考虑偶然性成功测试用例的影响,降低了错误定位的准确率。为此,提出一种新的软件错误定位方法,通过分析程序变异减少偶然性成功测试用例的影响,改进怀疑度计算公式,并加入对变异影响的计算。实验结果表明,与传统CBFL方法相比,该方法能够有效提高错误定位的准确率。     

组合认知复杂度的程序谱软件错误定位方法

基于程序谱的软件错误定位(spectrum-based fault localization,SBFL)技术收集测试用例结果和语句覆盖信息,用以计算每条语句的可疑度值.认知复杂度是软件复杂性度量工具,其值高的代码较易出错.为提升错误定位性能,提出一种语句级认知复杂度和SBFL相组合的方法对语句排序.当多条语句可疑度值相等时,新方法优先检查认知复杂度高的语句.测试数据集有925个错误版本,包含Java、C和C++项目.实验结果证实,加入认知复杂度后,传统的SBFL技术能减少待排查语句.     

基于圈复杂度的静态测试用例排序方法

针对如何排列测试用例的问题,提出一种基于圈复杂度的静态测试用例排序方法。首先介绍圈复杂度和基于方法覆盖的测试用例排序算法;然后将圈复杂度应用到排序算法中,设计了两种新的排序算法;最后通过实验,检测改进方法在错误检测方面的有效性。实验结果表明,与已有的几种优先级技术对比,上述改进方法能够达到更高的错误检测效率,有利于提高测试效率。     

基于定值-引用链的测试用例优先级排序算法

测试用例优先级排序作为一种高效实用的回归测试技术,通常以测试用例的覆盖度作为优先级排序的量化指标,忽略了测试用例的其他测试性能。针对该问题,提出一种基于DU链的测试用例优先级排序算法。该算法 综合考虑 测试用例的DU链覆盖度和回归测试的错误检测能力,对测试用例优先级进行量化。与已有算法相比,该算法基于数据流覆盖,充分利用了测试执行的历史信息和程序模块的耦合信息,在排序过程中动态计算测试用例的优先级量化值。实验结果表明,采用优先级排序算法的测试用例集能在测试过程中以较短的时间发现更多的错误,有效地提高了回归测试的检错效率。     

面向MBFL的测试用例约减策略

基于变异的错误定位(MBFL)是最近提出的一种自动化程序错误定位技术, 错误定位精度高, 但伴随着庞大的执行开销, 严重制约了其在工业领域的应用. 研究人员主要从减少变异体数量、减少测试用例数量和优化变异体的执行过程三个方面优化MBFL的执行效率. 前两种方法被广泛研究并取得很好的定位效果, 但对MBFL测试用例方面的研究较少, 且存在错误定位精度损失的问题. 为解决该问题, 本文提出了一种基于信息熵的测试用例约减方法(IETCR). IETCR首先计算出测试用例的信息熵, 然后根据信息熵对测试用例进行排序, 最后选择少量有价值的测试用例执行变异体. 在SIR中 6个程序100个版本上的实验结果表明, IETCR能够约减56.3%~88.6%的MBFL执行开销, 而且几乎保持与原始MBFL相同的错误定位精度.     

基于增强径向函数神经网络的错误定位方法

结合径向基函数神经网络与正交实验设计理论,提出了一种增强径向基函数神经网络错误定位算法.根据选择的测试用例执行得到源程序的语句覆盖信息和执行结果;通过神经网络计算出每条语句的可疑度值,并通过正交实验设计方法自适应调整神经网络中的参数值;最后按照可疑度值由高到低的顺序逐条检查程序的可疑语句进行错误定位.通过实验对所提出方法与径向基函数神经网络算法以及反向传播神经网络算法进行比较分析,结果表明,基于增强径向基函数神经网络算法具有更精确的错误定位效果和更显著的定位效率.     

面向软件错误定位与理解的测试执行离散特征筛选

软件错误定位与错误理解是软件调试过程中的重要步骤,然而调试人员利用基于覆盖分析的软件错误定位获取的可疑度,从高到低静态分析每条程序语句的检查方式,与实际软件调试过程并不相符。为了能够筛选更有助于理解错误根源的测试执行,尤其是致使程序失效的失效执行,帮助调试人员进行动态差异化分析,针对失效执行提出基于高可疑度覆盖率、揭示错误潜力和覆盖语句可疑度离散特征的3种优先级策略,针对成功执行提出加权余弦相似度匹配策略。通过将3种失效执行优先级策略与随机选择在常用错误定位技术中进行实验对比,验证了基于覆盖语句可疑度离散特征的失效执行筛选策略能够对筛选前后的错误理解工作量变化产生更强的积极影响和更弱的消极影响,并能够在相同工作量下理解更多的错误,进而更有助于将错误定位结果应用于错误根源的理解。     

基于改良程序谱的软件故障定位方法

故障定位是软件调试过程中一项耗时耗力的工作,自动化查错的应用对于提高软件调试效率具有重要的现实意义。近年来,基于程序谱的故障定位方法得到了研究人员的大量关注。针对单错误现象,提出了基于改良程序谱的软件故障定位新方法,该方法基于“在单错误情况下,若测试用例运行错误,则该测试用例运行必定覆盖了故障语句”这一论断,将所有的故障测试用例对程序语句的覆盖情况做交运算,从而得到故障基,再利用故障基定位故障。最后,以西门子测试程序集为测试数据,对比了不同方法对故障定位的效果和效率的影响,其结果表明所提出的方法可以有效地提高故障定位的效果和效率。     

一种基于频谱信息并结合碰集和遗传算法的缺陷定位方法

在软件研制过程中,缺陷定位是一个重要的研究课题。但是,实际软件中的缺陷数量无法被预先判定,且已有的单缺陷定位方法不易使用,已有的多缺陷定位方法存在定位效率不高的问题。基于此,文中对多缺陷定位方法GAMFL进行了研究和改进,提出了基于频谱信息并结合碰集和遗传算法的缺陷定位方法GAHIT。该方法定义了定位基本块,并用其替代语句进行缺陷定位,缩小了搜索范围;在初始种群的构造过程中,提出了采用求解失败用例执行路径碰集的方法,优化了初始种群的生成,并给出了新的适应度函数的计算方法,提高了算法的整体执行效率;最后针对遗传算法的结果,给出了缺陷检查策略,提高了在最优种群中查找缺陷的准确性。实验结果表明,所提方法能够有效处理缺陷数量未知情况下的定位问题,在单缺陷和多缺陷程序中都有较好的定位效果。     

Mutation-oriented test data augmentation for GUI software fault localization

ContextFault localization lies at the heart of program debugging and often proceeds by contrasting the statistics of program constructs executed by passing and failing test cases. A vital issue here is how to obtain these “suitable” test cases. Techniques presented in the literature mostly assume the existence of a large test suite a priori. However, developers often encounter situations where a failure occurs, but where no or no appropriate test suite is available for use to localize the fault.ObjectiveThis paper aims to alleviate this key limitation of traditional fault localization techniques for GUI software particularly, namely, it aims at enabling cost-effective fault localization process for GUI software in the described scenario.MethodTo address this scenario, we propose a mutation-oriented test data augmentation technique, which actually is directed by the “similarity” criterion in GUI software’s test case context towards the generation of test suite with excellent fault localization capabilities. More specifically, the technique mainly uses four proposed novel mutation operators to iteratively mutate some failing GUI test cases’ event sequences to derive new test cases potentially useful to localize the specific encountered fault. We then compare the fault localization performance of the test suite generated using this technique with that of an original provided large event-pair adequate test suite on some GUI applications.ResultsThe results indicate that the proposed technique is capable of generating a test suite that has comparable, if not better, fault localization effectiveness to the event-pair adequate test suite, but it is much smaller and it is generated immediately once a failure is encountered by developers.ConclusionIt is concluded that the proposed technique can truly enable quick-start cost-effective fault localization process under the investigated all-too-common scenario, greatly alleviating one key limitation of traditional fault localization techniques and prompting the test–diagnose–repair cycle.     

On similarity-awareness in testing-based fault localization

In the process of software development and maintenance, software debugging is an inevitable and time-consuming task. To accelerate software debugging, various approaches have been proposed to automate fault localization. Among them, testing-based fault-localization approaches are most promising, which use the execution information of many test cases to localize the faults. However, these existing testing-based fault-localization approaches ignore the similarity between test cases, which may harm the effectiveness of these approaches according to our previous research. Therefore, in this paper we propose a similarity-aware fault-localization approach, which takes each test case as a fuzzy set to deal with the similarity between test cases and calculates statements’ suspicions based on the probability theory. To investigate whether SAFL can address the similarity issue effectively, we manually injected redundant test cases in a test suite and performed an experimental study on the original test suite and the test suite with redundancy, respectively. The experimental results demonstrate that in our experiments SAFL is an effective fault-localization approach, whether there is manually injected redundancy in the test suite. To compare SAFL with most existing testing-based fault-localization approaches, we performed another experimental study on Siemens program suite, which is extensively used in the evaluation of many other testing-based fault-localization approaches. This experimental study confirms the effectiveness of SAFL. Based on the two experimental studies, it seems that in our experiments SAFL cannot only deal with test suites containing much redundancy effectively but also perform effectively for test suites without much redundancy. A preliminary version of this paper appears in (Hao et al. 2005a).     

HSFal: Effective fault localization using hybrid spectrum of full slices and execution slices

Most of the existing fault localization approaches use execution coverage of test cases to isolate the suspicious codes that likely contain faults. Program slicing can extract the dependencies of program entities with respect to a specific criterion. Therefore this technique is expected to have a beneficial effect on fault localization. In this paper, we propose a novel approach using a hybrid spectrum of full slices and execution slices to improve the effectiveness of fault localization. In particular, our approach firstly computes full slices of failed test cases and execution slices of passed test cases respectively. Secondly it constructs the hybrid spectrum by intersecting full slices and execution slices. Finally it computes the suspiciousness of each statement in the hybrid slice spectrum and generates a fault location report with descending suspiciousness of each statement. We also implement our proposed approach in our prototype tool HSFal by Java programming language. To verify the effectiveness of our approach, we performed an empirical study by the prototype on several widely used open source programs. Our approach is compared with eight representative coverage-based and slice-based fault localization approaches. Final experimental results show that our proposed approach is more effective in fault localization than other compared approaches, and can reduce almost 2.98–31.79% of the average cost of examined code significantly.     

Factor oriented requirement coverage based system test case prioritization of new and regression test cases

Test case prioritization involves scheduling test cases in an order that increases the effectiveness in achieving some performance goals. One of the most important performance goals is the rate of fault detection. Test cases should run in an order that increases the possibility of fault detection and also that detects the most severe faults at the earliest in its testing life cycle. In this paper, we propose to put forth a model for system level test case prioritization (TCP) from software requirement specification to improve user satisfaction with quality software that can also be cost effective and to improve the rate of severe fault detection. The proposed model prioritizes the system test cases based on the six factors: customer priority, changes in requirement, implementation complexity, completeness, traceability and fault impact. The proposed prioritization technique is validated with two different validation techniques and is experimented in three phases with student projects and two sets of industrial projects and the results show convincingly that the proposed prioritization technique improves the rate of severe fault detection.     

A test-suite reduction approach to improving fault-localization effectiveness

In order to improve the effectiveness of fault localization, researchers are interested in test-suite reduction to provide suitable test-suite inputs. Different test-suite reduction approaches have been proposed. However, the results are usually not ideal. Reducing the test-suite improperly or excessively can even negatively affect fault-localization effectiveness. In this paper, we propose a two-step test-suite reduction approach to remove the test cases which have little or no effect on fault localization, and improve the distribution evenness of concrete execution paths of test cases. This approach consists of coverage matrix based reduction and path vector based reduction, so it analyzes not only the test cases coverage but also the concrete path information. We design and implement experiments to verify the effect of our approach. The experimental results show that our reduced test-suite can improve fault-localization effectiveness. On average, our approach can reduce the size of a test-suite in 47.87% (for Siemens programs) and 23.03% (for space program). At the same time, on average our approach can improve the fault-localization effectiveness, 2.12 on Siemens programs and 0.13 on space program by Tarantula approach.     

针对基于变异错误定位的一种动态变异执行策略

在软件调试过程中,如何快速、精确地定位程序中的错误代码是软件开发人员普遍关注的问题。基于变异的错误定位方法是一种通过分析被测程序与程序变异体之间的行为相似性来估计语句出错概率、进行错误定位的方法。该方法有较高的错误定位精确度,但由于需对大量程序变异体执行测试用例集,因此其变异执行开销较大。为此提出了一种动态变异执行策略,它通过搜集测试用例执行信息,动态地调整变异体及测试用例的执行顺序,以减少其变异执行开销。实验结果表明,在6个程序包的127个错误版本上,应用提出的动态变异执行策略可在保证错误定位精确度的前提下,减少23%~78%的变异执行开销,显著提高了基于变异的错误定位方法的效率。     

Prioritizing test cases for early detection of refactoring faults

Refactoring edits are error‐prone, requiring cost‐effective testing. Regression test suites are often used as a safety net for decreasing the chances of behavioural changes. Because of the high costs related to handling massive test suites, prioritization techniques can be applied to reorder test case execution, fostering early fault detection. However, traditional prioritization techniques are not specifically designed for detecting refactoring‐related faults. This article proposes refactoring‐based approach (RBA), a refactoring‐aware strategy for prioritizing regression test cases. RBA reorders an existing test sequence, using a set of proposed refactoring fault models that define the refactoring's impact on program methods. Refactoring‐based approach's evaluation shows that it promotes early detection of refactoring faults and outperforms well‐known prioritization techniques in 71% of the cases. Moreover, it prioritizes fault‐revealing test cases close to one another in 73% of the cases, which can be useful for fault localization. Those findings show that RBA can considerably improve prioritization of test cases during perfective evolution, both by increasing fault‐detection rates as well as by helping to pinpoint defects introduced by an incorrect refactoring. Copyright © 2016 John Wiley & Sons, Ltd.