An empirical investigation of the relationship between spectra differences and regression faults

Many software maintenance and testing tasks involve comparing the behaviours of program versions. Program spectra have recently been proposed as a heuristic for use in performing such comparisons. To assess the potential usefulness of spectra in this context an experiment was conducted, examining the relationship between differences in program spectra and the exposure of regression faults (faults existing in a modified version of a program that were not present prior to modifications, or not revealed in previous testing), and empirically comparing several types of spectra. The results reveal that certain types of spectra differences correlate with high frequency—at least in one direction—with the exposure of regression faults. That is, when regression faults are revealed by particular inputs, spectra differences are likely also to be revealed by those inputs, though the reverse is not true. The results also suggest that several types of spectra that appear, analytically, to offer greater precision in predicting the presence of regression faults than other, cheaper, spectra may provide no greater precision in practice. These results have ramifications for future research on, and for the practical uses of, program spectra. Copyright © 2000 John Wiley & Sons, Ltd.     

用于回归测试数据验算的筛选方法

通过利用覆盖数据技术和回归测试集选择技术,提出一种用于回归测试数据验算的筛选方法,该方法通过深度优先遍历程序的相关记录来筛选测试用例,能有效地提高回归测试的准确性,减少回归测试的测试时间和省略无需测试的测试用例,以达到降低回归测试的成本的目的.对顺序、循环和分支3类程序设计了相关的实验,比较了该算法在这3类程序上的使用效果.     

Using component metadata to regression test component‐based software

Increasingly, modern‐day software systems are being built by combining externally‐developed software components with application‐specific code. For such systems, existing program‐analysis‐based software engineering techniques may not directly apply, due to lack of information about components. To address this problem, the use of component metadata has been proposed. Component metadata are metadata and metamethods provided with components, that retrieve or calculate information about those components. In particular, two component‐metadata‐based approaches for regression test selection are described: one using code‐based component metadata and the other using specification‐based component metadata. The results of empirical studies that illustrate the potential of these techniques to provide savings in re‐testing effort are provided. Copyright © 2006 John Wiley & Sons, Ltd.     

基于路径引导的回归测试用例集扩增方法

为了全面测试演化软件,回归测试通常需要生成新的测试用例。concolic测试是一种沿着具体执行路径进行符号执行的软件验证技术,通过生成测试数据来执行程序的所有可行路径。回归测试中,由于concolic测试关注于程序本身,没有利用已有测试用例和软件演化信息,导致生成大量无效测试数据,浪费资源和时间。为解决此问题,提出一种基于路径引导的回归测试用例集扩增方法。该方法将目标路径作为引导,根据软件演化信息选择有利于覆盖目标路径的测试用例,利用已有测试用例跳过重叠初始子路径,对后续目标子路径进行concolic测试并生成覆盖目标路径的测试数据。案例分析表明,本文方法相比传统concolic测试,本方法在覆盖程序可行路径的同时,可有效减少concolic测试路径,提高测试数据生成效率。     

Directed test suite augmentation: an empirical investigation

Test suite augmentation techniques are used in regression testing to identify code elements in a modified program that are not adequately tested and to generate test cases to cover those elements. A defining feature of test suite augmentation techniques is the potential for reusing existing regression test suites. Our preliminary work suggests that several factors influence the efficiency and effectiveness of augmentation techniques that perform such reuse. These include the order in which target code elements are considered while generating test cases, the manner in which existing regression test cases and newly generated test cases are used, and the algorithm used to generate test cases. In this work, we present the results of two empirical studies examining these factors, considering two test case generation algorithms (concolic and genetic). The results of our studies show that the primary factor affecting augmentation using these approaches is the test case generation algorithm utilized; this affects both cost and effectiveness. The manner in which existing and newly generated test cases are utilized also has a substantial effect on efficiency and in some cases a substantial effect on effectiveness. The order in which target code elements are considered turns out to have relatively few effects when using concolic test case generation but in some cases influences the efficiency of genetic test case generation. The results of our first study, on four relatively small programs using a large number of test suites, are supported by our second study of a much larger program available in multiple versions. Together, the studies reveal a potential opportunity for creating a more cost‐effective hybrid augmentation approach leveraging both concolic and genetic test case generation techniques, while appropriately utilizing our understanding of the factors that affect them. Copyright © 2014 John Wiley & Sons, Ltd.     

Selecting a Cost-Effective Test Case Prioritization Technique

Regression testing is an expensive testing process used to validate modified software and detect whether new faults have been introduced into previously tested code. To reduce the cost of regression testing, software testers may prioritize their test cases so that those which are more important, by some measure, are run earlier in the regression testing process. One goal of prioritization is to increase a test suite's rate of fault detection. Previous empirical studies have shown that several prioritization techniques can significantly improve rate of fault detection, but these studies have also shown that the effectiveness of these techniques varies considerably across various attributes of the program, test suites, and modifications being considered. This variation makes it difficult for a practitioner to choose an appropriate prioritization technique for a given testing scenario. To address this problem, we analyze the fault detection rates that result from applying several different prioritization techniques to several programs and modified versions. The results of our analyses provide insights into which types of prioritization techniques are and are not appropriate under specific testing scenarios, and the conditions under which they are or are not appropriate. Our analysis approach can also be used by other researchers or practitioners to determine the prioritization techniques appropriate to other workloads.     

回归测试中测试用例集优化方法的研究

回归测试是软件演化过程中频繁进行的且开销巨大的一项任务,测试用例集的优化程度直接影响着测试的成本和效率。针对回归测试过程的特点,提出一种对测试用例集优化的新方法,即通过对测试用例集进行必要的消除冗余和调整排序,完成了对初始测试用例集的精简以及执行顺序的确定过程,使得有限的测试资源得到科学合理的分配。实验结果表明,相对于以往的测试用例集优化方法,新方法的效率和资源分配的合理性均有了显著的提高。     

An empirical study of a test for polynomial relationships in randomly right censored regression models

In this paper, a test statistic is constructed to test polynomial relationships in randomly right censored regression models based on the local polynomial smoothing technique. Two bootstrap procedures, namely the residual-based bootstrap and the naive bootstrap procedures, are suggested to derive the p-value of the test. Some simulations are conducted to empirically assess the performance of the two bootstrap procedures. The results demonstrate that the residual-based bootstrap performs much better than the naive bootstrap and the test method with the residual-based bootstrap to derive the p-value works satisfactorily. Although the limiting distribution of the test statistic and the consistency of the bootstrap approximations remain to be investigated, simulation results indicate that the proposed test method may be of some practical use. As a real example, the proposed test is applied to the Stanford heart transplant data.     

ALL-Uses覆盖引导的回归测试用例生成方法

基于扩展有限状态机(EFSM)的回归测试过程需要根据依赖关系变化对软件所做修改的影响域进行分析。为了针对软件某一功能进行修复,通常需要对多处代码进行同步修改,已有依赖分析方法在这种情况下暴露模型中触发条件和行为语句错误的效率不高。提出以ALL-Uses覆盖准则引导回归测试的方法,引入依赖关系变化因素的概念,修改待覆盖子路径的产生规则,对已有测试用例集中能有效覆盖子路径的用例进行选择、排序。针对已有测试用例无法覆盖的子路径,利用AOE活动图中求关键路径的方法将其补充为一条完整的迁移执行序列。选取三个软件进行实验,结果表明,本文方法可以在减小测试用例集规模的前提下有效提升ALL-Uses和植入错误的覆盖率,提高回归测试效率。     

Regression test selection for C++ software

Regression testing is an important but expensive software maintenance activity performed with the aim of providing confidence in modified software. Regression test selection techniques reduce the cost of regression testing by selecting test cases for a modified program from a previously existing test suite. Many researchers have addressed the regression test selection problem for procedural language software, but few have addressed the problem for object‐oriented software. This paper presents a regression test selection technique for use with object‐oriented software. The technique constructs graph representations for software, and uses these graphs to select test cases, from the original test suite, that execute code that has been changed for the new version of the software. The technique is strictly code based, and requires no assumptions about the approach used to specify or test the software initially. The technique applies to modified and derived classes, and to application programs that use modified classes. Copyright © 2000 John Wiley & Sons, Ltd.     

矿用液压泵变频测试系统设计及应用

为满足液压泵故障后对备件液压泵或者修复后的液压泵进行性能测试与调试的需求,解决通用的液压泵测试系统自动化水平低、数据记录处理工作量大等问题,提出了一种矿用液压泵变频测试系统,并详细介绍了其液压系统、变频电控系统、信息采集系统和软件系统的设计。该系统可高速采集液压泵压力、流量、转速、转矩信号,能够自动完成液压泵的效率试验、冲击试验、满载试验、超载试验、超速试验。该系统在满载和超载试验时功率回收率达到60%,节能效果明显。     

基于实例的自动测试维护代价研究

不考虑自动测试的维护代价,盲目进行自动化测试是有风险的。以画板程序升级为例,详细介绍了它的修改过程和相应测试程序的维护代价,然后利用COCOMO度量方法,对测试程序与被测试程序的维护代价作一个比较,说明测试程序和被测试程序一样,需要反复维护才能进行回归测试。通过实例和理论分析使测试者清醒认识自动测试的维护代价,合理利用自动化测试。     

A safe regression testing approach for safety critical systems

Regression testing is important activity during the software maintenance to deal with adverse effects of changes. Our approach is important for safety critical system as usually formal methods are preferred and highly recommended for the safety critical systems but they are also applied for the systems development of other than critical system. Our approach is based on Regression testing using VDM++ which takes two VDM++ specifications, one baseline and other delta (Changed) along with test suite for the baseline version. It compares both versions by using comparator module, identifies the change. By analyzing the change we classify the test cases from original test suite into obsolete, re-testable, and reusable test cases. Our scope is at unit level i.e. at class level. Our approach gets two versions of VDM++ specification and returns regression test suite for the delta version. Our approach distinguishes test cases which are still effective for the delta version of VDM++ specification and it differs from re-test all strategy as it can distinguish the test cases and identifies test cases which are useful for delta version. Test cases reusability and test case reduction is the main objective of our approach. Our approach presents how to perform regression testing using VDM++ specification during the maintenance of systems.     

软件回归测试中的自动测试生成方法

软件回归测试一般使用现有的测试用例集进行测试,如何有效利用这些用例成为回归测试的关键。研究自动对现有测试用例集的扩展,包括自动改变测试用例的执行顺序、自动进行测试用例数目的增减等,提出基于现有测试用例随机生成新的测试用例集的方法。实验证明,该方法生成的测试文件能在原有环境下成功执行,能实现自动测试范围的扩大或缩小,以及测试时间的增加或减少,为软件回归测试提供了有用的自动化工具。     

面向时移电视类业务的组合测试实践研究

时移电视是三网融合的典型多媒体业务之一,具有功能多、操作复杂等特点。由于已有的测试方法尚未充分考虑系统中各参数的相互作用,因而较难发现系统某些功能缺陷和故障。以自适应组合测试方法学为基础,将其应用于机顶盒时移业务的测试。通过建立实用的交互式电视业务组合测试模型,设计测试用例并进行测试,最后对发现的故障进行定位。该实践研究为已有的组合测试理论和方法提供实证的同时,也发现了机顶盒时移业务存在的问题。     

多重关联的静态回归测试用例集构建研究

静态回归测试用例集构建策略是依据程序间的调用关联,分析因代码更改而受影响的模块,进而构建回归测试用例集,该方法并没有考虑程序间的隐式数据关联,对同一数据库操作或者对公共对象数据操作的方法间存在隐式数据关联。针对代码更改不仅会对调用关联的方法产生影响,也会对隐式数据关联的方法产生影响进行研究,提出了一种多重关联的静态回归测试用例集构建策略,通过构建多重方法关联图分析方法间调用关联和隐式数据关联,进而依据关联关系构建因代码更改而受影响的回归测试用例集。通过对4个开源项目进行实验评估,实验结果表明本文提出的静态策略提高了回归测试的安全性和精确性。     

基于偶然正确性概率的回归测试选择方法

数据驱动的智能系统的核心是处理数据的算法,对算法正确性的要求高,导致其测试开销大,需要有效地缩减测试的规模,其中回归测试选择是控制测试规模的有效手段.数据驱动的智能系统由于其动态信息流强度弱的原因,发生偶然正确性现象的概率较高,并且该现象会导致常用的回归测试选择技术所选择出的测试集包含大量检测不到故障的测试用例.因此,...     

自动化回归测试在Java项目中的实现

回归测试是软件测试中一种重要的测试。软件开发过程中出现的各种变化可能会给软件的质量带来副作用。回归测试就是用来监控软件的变化,及时反馈由变化带来的结果。回归测试的自动化可以提高较大规模软件测试的质量和效率。文中首先介绍了软件自动化测试和回归测试的概念,然后以一个用Java语言开发的项目CADS(CrossArchitectureDevelopmentSuite)为例,详细阐述了如何在Java项目中实现回归测试的自动化,最后总结了自动化回归测试的优点及其适用范围。     

Regression testing minimization,selection and prioritization: a survey

Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research. Copyright © 2010 John Wiley & Sons, Ltd.