回归测试中测试用例集扩充技术研究进展

测试用例集扩充问题(Test Suite Augmentation)是回归测试研究的一个最新研究热点.在完成代码修改影响分析后,对已有测试用例集的充分性进行评估,若不充分则设计新的测试用例,以确保对代码修改的充分测试.但到目前为止,国内外学者并未对该研究问题的已有研究成果进行系统总结和展望.首先介绍了测试用例集扩充问题的研究背景和问题描述,然后总结出研究框架并对已有研究工作进行分类和系统比较,接着对常用评测数据集和评测指标进行了分析,最后对该问题值得关注的未来研究方向进行展望.     

基于混合遗传算法的回归测试用例集最小化研究

为缩减测试用例规模,降低回归测试成本,将遗传算法和贪心算法相结合,提出了一种混合遗传算法用于解决测试用例最小化问题.算法对标准遗传算法中的选择、交叉和变异操作进行改善,提高了算法的全局寻优能力.同时,利用贪心算法处理可行解和不可行解,提高了算法的局部寻优能力.实验结果表明:与标准遗传算法相比,在保证测试完备性的前提下,混合遗传算法能够得到更优的缩减效果和更快的收敛速度.     

回归测试中测试用例集优化方法的研究

回归测试是软件演化过程中频繁进行的且开销巨大的一项任务,测试用例集的优化程度直接影响着测试的成本和效率。针对回归测试过程的特点,提出一种对测试用例集优化的新方法,即通过对测试用例集进行必要的消除冗余和调整排序,完成了对初始测试用例集的精简以及执行顺序的确定过程,使得有限的测试资源得到科学合理的分配。实验结果表明,相对于以往的测试用例集优化方法,新方法的效率和资源分配的合理性均有了显著的提高。     

基路径覆盖测试用例自动生成方法研究

传统基路径覆盖测试用例生成方法通过程序图求出圈复杂度,然后再得出程序的一组基路径,最后分别针对基路径组中的每条路径求出相应的测试用例,不仅繁琐,而且忽视了代码的语义相关性,导致存在路径不可达问题,也就无法生成对应的测试用例.提出了一种新的方法,利用遗传算法动态运行程序,逐渐逼近被测程序的真实逻辑圈复杂度,直接生成满足基路径覆盖测试用例的最小集合,不存在路径不可达问题.实验结果表明,该算法能够有效地生成满足基路径覆盖的测试用例.     

基于函数调用路径的回归测试用例选择排序方法研究

针对在回归测试过程中,因为不断修复软件中存在的缺陷所造成的测试工作量大、测试效率低等问题,论文将测试用例选择与优先级排序技术相结合,以面向函数调用的路径覆盖生成方法为基础,提出了一种面向函数调用路径（Functions Calling Path, FCP）的测试用例选择与排序方法。首先根据函数调用关系图,对程序中被修改函数与其他函数的关联性进行分析,从初始测试用例集中选择测试用例,形成回归测试用例集;然后对这些测试用例进行优先级排序,并动态地调整优先级排序结果;最后,对优先级排序结果进行再次选择,确定最小的回归测试用例集。实验结果表明,测试用例选择与排序方法对优化回归测试用例是有效的,大大减少了回归测试用例数量,降低了回归测试成本。     

基于动态切片和UML图的回归测试用例生成

针对基于UML设计的面向对象软件在修改后进行回归测试的情况,提出一种回归测试用例生成的新方法。获取软件修改后进行回归测试时必须重新测试的类和方法,分析与需要重测试类有关的UML顺序图,对顺序图中每个包含不等关系的条件断言生成与其有关的动态程序切片,根据该动态程序切片生成与其相应的回归测试用例。     

基于冗余测试用例的最小测试用例集生成方法

提出一种最小测试用例集的生成方法。保留在某个测试标准下冗余,但在其他测试标准下不冗余的测试用例,即在测试用例集简化期间通过有选择性地保留测试用例来生成一个测试用例集。与已有方法相比,该方法能在不影响测试组大小范围的情况下有效提高错误检测效率。     

基于蚁群算法的测试用例集优化方法

为了达到以尽可能少的测试用例满足测试需求的目的,提出了一种先对测试用例集进行完全划分,再利用蚁群算法对其优化的方法。首先根据测试需求间的相互关系,将最初的测试用例集划分成多个互不相交的子集,每个子集中的元素为等价测试用例;其次从各个子集中选取一个测试用例,组成一个新的集合,该集合已经摒弃了部分冗余测试用例;然后利用蚁群算法对测试用例集进行最优的简化;最后通过实例证明了该方法可以产生比原有的方法更优的测试用例集。     

测试用例演化研究进展

由于功能增加、性能调优、错误修复等原因,软件常常动态演化.现有测试技术难以满足软件演化过程中变化的测试需求,因此需要系统的测试用例演化技术,以有效保障演化软件的质量.回顾测试用例演化技术的研究现状,分别对测试用例选择、测试用例修复和测试用例集扩增这3部分内容进行了详细的比较和分析.最后提出测试用例演化技术领域存在的挑战和未来的研究方向.     

基于搜索的分层回归测试数据集扩增方法

针对在回归测试中原有的测试数据集往往难以满足新版本软件的测试需求问题,提出一种基于搜索的分层回归测试数据生成方法。方法主要包含覆盖目标方法集获取模块和测试数据生成模块。首先对新版本程序进行抽象分析,提取出方法调用图,利用方法调用轨迹和已有测试数据建立方法覆盖信息,获取目标方法集,并通过计算贝叶斯条件概率对目标方法集进行优先选择;利用Hadamard矩阵设计正交种群,同时结合已有测试数据集进行种群初始化,采用文化基因算法对目标集中方法生成测试数据。该方法针对4个基准程序与随机法和遗传算法以及基于粒子群算法测试数据生成方法相比,测试数据的生成效率平均提高了95.2%、78.2%和50.5%,测试数据检错能力平均提高了47.9%、33.6%和18.2%,实验结果表明,该方法更适合回归测试数据的生成。     

Directed test suite augmentation: an empirical investigation

Test suite augmentation techniques are used in regression testing to identify code elements in a modified program that are not adequately tested and to generate test cases to cover those elements. A defining feature of test suite augmentation techniques is the potential for reusing existing regression test suites. Our preliminary work suggests that several factors influence the efficiency and effectiveness of augmentation techniques that perform such reuse. These include the order in which target code elements are considered while generating test cases, the manner in which existing regression test cases and newly generated test cases are used, and the algorithm used to generate test cases. In this work, we present the results of two empirical studies examining these factors, considering two test case generation algorithms (concolic and genetic). The results of our studies show that the primary factor affecting augmentation using these approaches is the test case generation algorithm utilized; this affects both cost and effectiveness. The manner in which existing and newly generated test cases are utilized also has a substantial effect on efficiency and in some cases a substantial effect on effectiveness. The order in which target code elements are considered turns out to have relatively few effects when using concolic test case generation but in some cases influences the efficiency of genetic test case generation. The results of our first study, on four relatively small programs using a large number of test suites, are supported by our second study of a much larger program available in multiple versions. Together, the studies reveal a potential opportunity for creating a more cost‐effective hybrid augmentation approach leveraging both concolic and genetic test case generation techniques, while appropriately utilizing our understanding of the factors that affect them. Copyright © 2014 John Wiley & Sons, Ltd.     

基于子域的构件软件充分性测试样本生成策略

基于构件的软件开发给软件测试带来了新的挑战。现有基于构件的软件测试充分性的研究表明,集成测试时的软件测试充分性不能保证构件的测试充分性,同时,构件的测试充分性也不能保证软件的测试充分性。提出了将构件测试充分性对集成测试的限制转化为子域的思想,将测试标准产生的软件子域和构件子域映射的软件子域进行统一的处理,从而对基于构件的软件测试样本集生成与优化提出了一个新的思路。     

用于回归测试数据验算的筛选方法

通过利用覆盖数据技术和回归测试集选择技术,提出一种用于回归测试数据验算的筛选方法,该方法通过深度优先遍历程序的相关记录来筛选测试用例,能有效地提高回归测试的准确性,减少回归测试的测试时间和省略无需测试的测试用例,以达到降低回归测试的成本的目的.对顺序、循环和分支3类程序设计了相关的实验,比较了该算法在这3类程序上的使用效果.     

测试用例集约简问题研究及其进展

测试用例集约简问题是软件测试中的关键问题之一,其目的是使用尽可能少的测试用例充分满足给定的测试目标,从而提高测试效率、降低测试成本。在简要介绍了测试用例集约简问题基本概念的基础上,总结了求解该问题的几种主要方法,分析比较了这些方法的效率和特性。随后探讨了与测试用例集约简问题强相关的测试用例集错误检测效率的问题,并研究了测试用例优先级技术。最后指出了测试用例集约简问题的下一步研究方向。     

On guiding the augmentation of an automated test suite via mutation analysis

Mutation testing has traditionally been used as a defect injection technique to assess the effectiveness of a test suite as represented by a “mutation score.” Recently, mutation testing tools have become more efficient, and industrial usage of mutation analysis is experiencing growth. Mutation analysis entails adding or modifying test cases until the test suite is sufficient to detect as many mutants as possible and the mutation score is satisfactory. The augmented test suite resulting from mutation analysis may reveal latent faults and provides a stronger test suite to detect future errors which might be injected. Software engineers often look for guidance on how to augment their test suite using information provided by line and/or branch coverage tools. As the use of mutation analysis grows, software engineers will want to know how the emerging technique compares with and/or complements coverage analysis for guiding the augmentation of an automated test suite. Additionally, software engineers can benefit from an enhanced understanding of efficient mutation analysis techniques. To address these needs for additional information about mutation analysis, we conducted an empirical study of the use of mutation analysis on two open source projects. Our results indicate that a focused effort on increasing mutation score leads to a corresponding increase in line and branch coverage to the point that line coverage, branch coverage and mutation score reach a maximum but leave some types of code structures uncovered. Mutation analysis guides the creation of additional “common programmer error” tests beyond those written to increase line and branch coverage. We also found that 74% of our chosen set of mutation operators is useful, on average, for producing new tests. The remaining 26% of mutation operators did not produce new test cases because their mutants were immediately detected by the initial test suite, indirectly detected by test suites we added to detect other mutants, or were not able to be detected by any test.

Test suite reduction methods that decrease regression testing costs by identifying irreplaceable tests

ContextIn software development and maintenance, a software system may frequently be updated to meet rapidly changing user requirements. New test cases will be designed to ensure the correctness of new or modified functions, thus gradually increasing the test suite’s size. Test suite reduction techniques aim to decrease the cost of regression testing by removing the redundant test cases from the test suite and then obtaining a representative set of test cases that still yield a high level of code coverage.ObjectiveMost of the existing reduction algorithms focus on decreasing the test suite’s size. Yet, the differences in execution costs among test cases are usually significant and it may take a lot of execution time to run a test suite consisting of a few long-running test cases. This paper presents and empirically evaluates cost-aware algorithms that can produce the representative sets with lower execution costs.MethodWe first use a cost-aware test case metric, called Irreplaceability, and its enhanced version, called EIrreplaceability, to evaluate the possibility that each test case can be replaced by others during test suite reduction. Furthermore, we construct a cost-aware framework that incorporates the concept of test irreplaceability into some well-known test suite reduction algorithms.ResultsThe effectiveness of the cost-aware framework is evaluated via the subject programs and test suites collected from the Software-artifact Infrastructure Repository — frequently chosen benchmarks for experimentally evaluating test suite reduction methods. The empirical results reveal that the presented algorithms produce representative sets that normally incur a low cost to yield a high level of test coverage.ConclusionThe presented techniques indeed enhance the capability of the traditional reduction algorithms to reduce the execution cost of a test suite. Especially for the additional Greedy algorithm, the presented techniques decrease the costs of the representative sets by 8.10–46.57%.     

基于自适应粒子群优化算法的测试数据扩增方法

针对在回归测试中原有测试数据集往往难以满足新版本软件测试需求的问题,提出一种基于自适应粒子群算法（APSO）的测试数据扩增方法。首先,根据原有测试数据在新版本程序上的穿越路径与目标路径的相似度,在原有的测试数据集中选择合适的测试数据,作为初始种群的进化个体;然后,利用初始测试数据的穿越路径与目标路径的不同子路径,确定造成两者路径偏离的输入分量;最后,根据路径相似度构建适应度函数,利用APSO操作输入分量,生成新的测试数据。该方法针对四个基准程序与基于遗传算法（GA）和随机法的测试数据扩增方法相比,测试数据扩增效率分别平均提高了约56%和81%。实验结果表明,所提方法在回归测试方面有效地提高了测试数据扩增的效率,增强了其稳定性。     

面向路径的测试数据自动生成工具的设计

设计了一个通用的基于控制流和数据流的结构测试数据自动生成的工具。该工具根据控制流和数据流测试中所采用的覆盖标准来选取测试路径,并以改进后的迭代松弛法为核心,对所选取的路径生成测试数据。同时工具采用Fibonacci法优化选取路径,对不可达路径进行处理,并对测试数据的分支覆盖率、DCP覆盖率等进行了统计。实验结果表明该工具是可行的。     

基于混合测试和动态分析的分段代码测试

传统混合执行测试方法无法对源代码不可见函数进行符号执行。针对该问题,将符号执行、分段式符号执行以及具体执行按需结合,提出一种分段式混合执行测试方法,将源代码不可见函数以分段式分析法截取为单独代码片段,结合动态执行和回归分析方法推导其相应的程序语义。为验证该方法的有效性,实现sCREST原型系统,并对5个应用广泛的开源系统进行测试。实验结果表明,该方法能够产生比传统方法覆盖更多分支数的测试数据。