基于定值-引用链的测试用例优先级排序算法

测试用例优先级排序作为一种高效实用的回归测试技术,通常以测试用例的覆盖度作为优先级排序的量化指标,忽略了测试用例的其他测试性能。针对该问题,提出一种基于DU链的测试用例优先级排序算法。该算法 综合考虑 测试用例的DU链覆盖度和回归测试的错误检测能力,对测试用例优先级进行量化。与已有算法相比,该算法基于数据流覆盖,充分利用了测试执行的历史信息和程序模块的耦合信息,在排序过程中动态计算测试用例的优先级量化值。实验结果表明,采用优先级排序算法的测试用例集能在测试过程中以较短的时间发现更多的错误,有效地提高了回归测试的检错效率。     

基于函数调用路径的测试用例优先级排序

基于覆盖的优先级排序技术通常以代码覆盖信息作为测试用例的特征加以度量,忽略了其他优先级的影响因素,缺乏全面性和动态性。针对该问题,提出基于函数调用路径的测试用例优先级排序方法。以函数调用路径为基础,通过对源代码新旧版本的对比,分析回归测试影响域,确定回归测试用例集的范围。将测试用例函数调用路径覆盖能力、单元测试时函数中检测出缺陷的个数,以及函数的扇入系数等影响因素应用于优先级排序,确定测试用例优先级量化方法,并在测试执行过程中,通过调整算法实现优先级的动态调整,优化优先级排序。实验结果表明,优先级量化方法能提高测试的缺陷检测率,发现程序中的缺陷,降低测试成本。     

基于OTT策略的组合测试用例优先级排序方法

在组合测试用例优先级排序问题中,通常采用组合覆盖率为排序标准,该方法能够尽快满足覆盖率的要求,但其排序影响因子较为单一,缺陷检测能力不稳定。针对此问题,该文结合One-Test-at-a-Time（OTT）策略提出了一种在线调整的组合测试用例优先级排序方法,引入多重待覆盖率、测试用例失效率和测试用例重要程度3个影响因子用于衡量组合测试用例优先级。根据测试用例的在线测试反馈信息,该方法能够实时调整组合测试用例的优先级,增加了优先级排序的准确性。实验结果表明,相较于original、基于单一覆盖和多重覆盖的组合测试优先级排序方法,该方法在缺陷检测能力和稳定性上具有更强的竞争力。     

基于函数调用路径的回归测试用例选择排序方法研究

针对在回归测试过程中,因为不断修复软件中存在的缺陷所造成的测试工作量大、测试效率低等问题,论文将测试用例选择与优先级排序技术相结合,以面向函数调用的路径覆盖生成方法为基础,提出了一种面向函数调用路径（Functions Calling Path, FCP）的测试用例选择与排序方法。首先根据函数调用关系图,对程序中被修改函数与其他函数的关联性进行分析,从初始测试用例集中选择测试用例,形成回归测试用例集;然后对这些测试用例进行优先级排序,并动态地调整优先级排序结果;最后,对优先级排序结果进行再次选择,确定最小的回归测试用例集。实验结果表明,测试用例选择与排序方法对优化回归测试用例是有效的,大大减少了回归测试用例数量,降低了回归测试成本。     

基于需求的测试用例优先级排序

为提高软件测试用例执行效率,研究了测试用例优先级排序技术,提出了基于需求的测试用例优先级排序及动态调整算法.基于需求的测试用例优先级排序将与软件需求相关的影响因素应用于测试用例的排序,并且在测试执行过程中可以通过调整算法对测试用例的优先级进行动态调整.实验结果表明,采用基于需求的测试用例优先级排序可以提高软件缺陷检测率...     

基于OTT策略的可变力度组合测试用例优先级排序方法

如何选取组合力度用于测试是可变力度组合测试用例优先级排序方法中迫于解决的关键问题。采用组合覆盖率为排序标准能够满足组合覆盖率,但其排序因素单一,测试用例优先级排序结果差异性较大,且无法根据测试结果反馈信息及时调整组合测试用例优先级。针对上述问题,该文结合One-test-at-a-time(OTT)策略、利用局部组合覆盖率、测试用例失效率和测试用例重要程度对组合测试用例优先级排序方法进行了研究。在测试过程中,实时关注测试用例的执行结果用于在线调整测试用例排序因素的取值情况,以达到实时更新组合测试用例优先级的目的。实验结果表明：相较于Random、ICBP、GISVSP和LISVSP方法,该方法使得组合测试用例优先级排序结果相对稳定,且在缺陷检测率上具有竞争力。     

基于改进遗传算法的回归测试用例优先级排序

由于优先级排序测试效果较差、执行效率低,为此提出基于改进遗传算法的回归测试用例优先级排序方法.通过自然选择与遗传理论作为基本条件,结合群体中染色体任意信息交换机制和生物进化过程内适者生存规则,完成遗传算法,接着引入禁忌搜索方法,增强排序的局部超强搜索能力,然后在回归测试的过程内,利用多个目标函数对回归测试用例优先级排序以及评价,完成回归测试用例优先级排序.实验结果表明,经过时间以及覆盖的角度考察,执行时间较短,平均分支较大,说明该序列优良,本文方法的回归测试效果较好,执行效率高.     

基于圈复杂度的静态测试用例排序方法

针对如何排列测试用例的问题,提出一种基于圈复杂度的静态测试用例排序方法。首先介绍圈复杂度和基于方法覆盖的测试用例排序算法;然后将圈复杂度应用到排序算法中,设计了两种新的排序算法;最后通过实验,检测改进方法在错误检测方面的有效性。实验结果表明,与已有的几种优先级技术对比,上述改进方法能够达到更高的错误检测效率,有利于提高测试效率。     

基于回归测试的软件测试方法的研究与应用

为节省回归测试的时间和资源,提出了一种基于回归测试的软件测试方法（Hierarchy-PS-TCP）。该方法首先按照分层程序切片技术搜索程序的不同版本,利用UML类图及顺序图,从而定位不同版本程序间的差别。仅对差别部分生成测试用例集,节省大量开支。再利用优先级技术按照测试用例优先级公式对测试用例划分优先级,根据优先级高低进行排序,按照次序进行回归测试。从而提高了回归测试的效率,有效地缩减了回归测试用例集,同时降低了回归测试的成本。     

一种风险驱动的迭代开发需求优先级排序方法

需求优先级排序是指系统参与者为需求指定实现的优先次序,是迭代开发过程中开发者制定项目迭代计划的基础.现存的需求优先级排序方法对系统参与者之间的协商和调整优先级的支持能力不足,导致根据需求优先级所制定的迭代计划难以作出符合需求变更和环境改变的调整.提出一种风险驱动的需求优先级自适应排序方法.该方法将自适应计划方法学与风险驱动相结合,将风险作为排序决策的依据,以自适应的过程为迭代开发排序需求优先级.该方法能够改善需求优先级排序过程中系统参与者之间的协商和调整需求优先级的能力,增强在迭代开发中对需求的控制,降低因需求导致项目失败的可能性.     

回归测试中的测试用例优先排序技术述评

测试用例优先排序(test case prioritization,简称TCP)问题是回归测试研究中的一个热点.通过设定特定排序准则,对测试用例进行排序以优化其执行次序,旨在最大化排序目标,例如最大化测试用例集的早期缺陷检测速率.TCP问题尤其适用于因测试预算不足以致不能执行完所有测试用例的测试场景.首先对TCP问题进行描述,并依次从源代码、需求和模型这3个角度出发对已有的TCP技术进行分类;然后对一类特殊的TCP问题(即测试资源感知的TCP问题)的已有研究成果进行总结;随后依次总结实证研究中常用的评测指标、评测数据集和缺陷类型对实证研究结论的影响;接着依次介绍TCP技术在一些特定测试领域中的应用,包括组合测试、事件驱动型应用测试、Web服务测试和缺陷定位等;最后对下一步工作进行展望.     

融合聚类算法和缺陷预测的测试用例优先排序方法

持续集成环境下,软件快速更新加快了回归测试执行的频率,但缺陷快速反馈的需求对回归测试又提出了更高要求。测试用例优先排序技术研究测试用例的重要性,通常将缺陷探测能力强的测试用例优先执行,使其提早发现软件缺陷,其可解决持续集成环境下的快速反馈需求。缺陷预测技术可通过被测系统代码特征和历史缺陷来预估信息预测软件在新版本中发现缺陷的可能性,传统基于聚类的测试用例优先排序方法大多未考虑不同类簇数和特征子集对聚类结果的影响。文中将缺陷预测应用到聚类优先排序方法,构建测试用例和代码关联矩阵,对测试用例进行聚类分析,结合缺陷预测结果和最大最小距离策略指导簇间和簇内排序。通过实验验证发现,类簇数和聚类特征子集选择对排序效果有一定影响,当未能获取最佳类簇数和特征子集时,相比单一的聚类优先排序方法,所提方法可更有效地提高回归测试效率。     

Bio-inspired metaheuristics: evolving and prioritizing software test data

Software testing is both a time and resource-consuming activity in software development. The most difficult parts of software testing are the generation and prioritization of test data. Principally these two parts are performed manually. Hence introducing an automation approach will significantly reduce the total cost incurred in the software development lifecycle. A number of automatic test case generation (ATCG) and prioritization approaches have been explored. In this paper, we propose two approaches: (1) a pathspecific approach for ATCG using the following metaheuristic techniques: the genetic algorithm (GA), particle swarm optimization (PSO) and artificial bee colony optimization (ABC); and (2) a test case prioritization (TCP) approach using PSO. Based on our experimental findings, we conclude that ABC outperforms the GA and PSO-based approaches for ATC.G Moreover, the results for PSO on TCP arguments demonstrate biased applicability for both small and large test suites against random, reverse and unordered prioritization schemes. Therefore, we focus on conducting a comprehensive and exhaustive study of the application of metaheuristic algorithms in solving ATCG and TCP problems in software engineering.     

A history-based cost-cognizant test case prioritization technique in regression testing

Software testing is typically used to verify whether the developed software product meets its requirements. From the result of software testing, developers can make an assessment about the quality or the acceptability of developed software. It is noted that during testing, the test case is a pair of input and expected output, and a number of test cases will be executed either sequentially or randomly. The techniques of test case prioritization usually schedule test cases for regression testing in an order that attempts to increase the effectiveness. However, the cost of test cases and the severity of faults are usually varied. In this paper, we propose a method of cost-cognizant test case prioritization based on the use of historical records. We gather the historical records from the latest regression testing and then propose a genetic algorithm to determine the most effective order. Some controlled experiments are performed to evaluate the effectiveness of our proposed method. Evaluation results indicate that our proposed method has improved the fault detection effectiveness. It can also been found that prioritizing test cases based on their historical information can provide high test effectiveness during testing.     

基于蚁群算法的测试用例优先排序

测试用例优先排序技术通过优化测试用例的执行次序来提高软件测试的效率,是增强型软件测试和回归测试的重要研究课题。针对基于需求的测试用例优先排序问题,提出了一种基于蚁群算法的求解方法,采用不同的测试用例间距离及用例序列评价策略,给出了该方法的2种不同实现方式。首先,针对黑盒测试特点,设计了基于需求的一般性测试用例序列评价指标;其次,提出测试用例吸引度概念,基于测试用例吸引度定义了测试用例间的距离;然后,给出了信息素更新策略、最优解集更新策略、局部最优解突变策略等主要设计策略,分别实现了该方法基于距离和基于指标的2种实现方式。实验结果表明,该方法具有很好的全局寻优能力,整体效果上优于粒子群算法、遗传算法和随机测试。     

Factor oriented requirement coverage based system test case prioritization of new and regression test cases

Test case prioritization involves scheduling test cases in an order that increases the effectiveness in achieving some performance goals. One of the most important performance goals is the rate of fault detection. Test cases should run in an order that increases the possibility of fault detection and also that detects the most severe faults at the earliest in its testing life cycle. In this paper, we propose to put forth a model for system level test case prioritization (TCP) from software requirement specification to improve user satisfaction with quality software that can also be cost effective and to improve the rate of severe fault detection. The proposed model prioritizes the system test cases based on the six factors: customer priority, changes in requirement, implementation complexity, completeness, traceability and fault impact. The proposed prioritization technique is validated with two different validation techniques and is experimented in three phases with student projects and two sets of industrial projects and the results show convincingly that the proposed prioritization technique improves the rate of severe fault detection.     

Static test case prioritization using topic models

Software development teams use test suites to test changes to their source code. In many situations, the test suites are so large that executing every test for every source code change is infeasible, due to time and resource constraints. Development teams need to prioritize their test suite so that as many distinct faults as possible are detected early in the execution of the test suite. We consider the problem of static black-box test case prioritization (TCP), where test suites are prioritized without the availability of the source code of the system under test (SUT). We propose a new static black-box TCP technique that represents test cases using a previously unused data source in the test suite: the linguistic data of the test cases, i.e., their identifier names, comments, and string literals. Our technique applies a text analysis algorithm called topic modeling to the linguistic data to approximate the functionality of each test case, allowing our technique to give high priority to test cases that test different functionalities of the SUT. We compare our proposed technique with existing static black-box TCP techniques in a case study of multiple real-world open source systems: several versions of Apache Ant and Apache Derby. We find that our static black-box TCP technique outperforms existing static black-box TCP techniques, and has comparable or better performance than two existing execution-based TCP techniques. Static black-box TCP methods are widely applicable because the only input they require is the source code of the test cases themselves. This contrasts with other TCP techniques which require access to the SUT runtime behavior, to the SUT specification models, or to the SUT source code.     

An improved method for test case prioritization by incorporating historical test case data

Test case prioritization reorders test cases from a previous version of a software system for the current release to optimize regression testing. We have previously introduced a technique for test case prioritization using historical test case performance data. The technique was based on a test case prioritization equation, which directly computes the priority of each test case using the historical information of the test case using an equation with constant coefficients. This technique was compared just with random ordering approach. In this paper, we present an enhancement of the aforementioned technique in two ways. First, we propose a new prioritization equation with variable coefficients gained according to the available historical performance data, which acts as a feedback from the previous test sessions. Second, a family of comprehensive empirical studies has been conducted to evaluate the performance of the technique. We have compared the proposed technique with our previous technique and the technique proposed by Kim and Porter. The experimental results demonstrate the effectiveness of the proposed technique in accelerating the rate of fault detection in history-based test case prioritization.     

Test case prioritization techniques for model-based testing: a replicated study

Recently, several test case prioritization (TCP) techniques have been proposed to order test cases for achieving a goal during test execution, particularly, revealing faults sooner. In the model-based testing (MBT) context, such techniques are usually based on heuristics related to structural elements of the model and derived test cases. In this sense, techniques’ performance may vary due to a number of factors. While empirical studies comparing the performance of TCP techniques have already been presented in literature, there is still little knowledge, particularly in the MBT context, about which factors may influence the outcomes suggested by a TCP technique. In a previous family of empirical studies focusing on labeled transition systems, we identified that the model layout, i.e., amount of branches, joins, and loops in the model, alone may have little influence on the effectiveness of TCP techniques investigated, whereas characteristics of test cases that actually fail definitely influences this aspect. However, we considered only synthetic artifacts in the study, which reduced the ability of representing properly the reality. In this paper, we present a replication of one of these studies, now with a larger and more representative selection of techniques and considering test suites from industrial systems as experimental objects. Our objective is to find out whether the results remain while increasing the validity in comparison to the original study. Results reinforce that there is no best performer among the investigated techniques and characteristics of test cases that fail represent an important factor, although adaptive random-based techniques are less affected by it.     

Experiments with test case prioritization using relevant slices

Software testing and retesting occurs continuously during the software development lifecycle to detect errors as early as possible and to gain confidence that changes to software do not introduce defects. Once developed, test suites are reused and updated frequently, and their sizes grow as software evolves. Due to time and resource constraints, an important goal during regression testing of software is to prioritize the execution of test cases in a suite so as to improve the chances of increasing the rate of fault detection. Prior techniques for test case prioritization are based on the total number of coverage requirements exercised by the test cases. In this paper, we present a new approach to prioritize test cases that takes into account the coverage requirements present in the relevant slices of the outputs of test cases. We have implemented three different heuristics based on our relevant slicing based approach to prioritize test cases and conducted experiments to compare the effectiveness of our techniques with those of the traditional techniques that only account for the total requirement coverage. Our detailed experimental study and results provide interesting insights into the effectiveness of using relevant slices for test case prioritization in terms of ability to achieve high rate of fault detection.