常用的基于控制流的测试充分性准则

本文介绍了几种在软件测试中常用的基于控制流的测试充分性评价准则。这些测试充分性准则适用于用过程式语言、面向对象语言编写的程序。测试人员可用这些准则判断究竟测试了多少代码,还有多少代码未测试到。     

基于UML状态图的软件测试充分性准则研究

文中详细讨论了一组基于UML状态图的软件测试充分性准则。在软件测试时,还没有测试充分性准则明确定义循环应该被执行多少次,而循环只被执行一次是很不充分的。针对这种情况,在现有测试准则的基础上提出了两种扩展的基于UML状态图测试的充分性准则：ZOT-R循环覆盖准则和全ZOT-R路径覆盖准则,丰富并完善了现有的测试准则体系。并且通过结合一个具体实例给出了这些覆盖准则在具体测试实践中的应用方法,探讨了这些覆盖准则之间的包含关系。     

一种新的变异测试数据自动生成方法

变异测试是一种行之有效的软件测试方法,通过使用变异算子产生变异体系统地模拟软件中的各种缺陷,然后构造能够杀死这些变异体的测试数据集.自动生成能够杀死变异体的测试数据将提高变异测试的效率和有效性.当前的研究工作只考虑生成杀死单个变异体的测试数据.文中根据杀死同一位置的多个变异体的条件相近的特点,提出一种对杀死这些变异体的条件进行组合,然后生成同时杀死该位置多个变异体的测试数据的方法;给出相应的支持工具,并且通过实验验证方法的有效性.     

变异测试:原理、优化和应用

变异测试是一种基于缺陷的软件测试技术,在近四十年得到国内外学者的广泛关注,并取得了一些研究成果。对已有的研究工作进行总结,将其分为变异测试原理、优化和应用三个模块。其中在变异测试原理模块中,给出变异测试的基本假设,对变异测试分析流程进行介绍,并对其中的重要概念依次给出定义,从静态检测和动态检测两个角度对等价变异体检测技术进行总结。在变异测试优化模块中,从变异体选择优化和变异体执行优化两个角度对已有研究工作进行总结。在变异测试应用模块中,选择了测试用例集充分性评估、测试用例生成和回归测试三个应用领域,对研究工作进行分类总结。最后对变异测试的未来研究方向进行了展望。     

An Experimental Evaluation of Data Flow and Mutation Testing

Two experimental comparisons of data flow and mutation testing are presented. These techniques are widely considered to be effective for unit-level software testing, but can only be analytically compared to a limited extent. We compare the techniques by evaluating the effectiveness of test data developed for each. We develop ten independent sets of test data for a number of programs: five to satisfy the mutation criterion and five to satisfy the all-uses data-flow criterion. These test sets are developed using automated tools, in a manner consistent with the way a test engineer might be expected to generate test data in practice. We use these test sets in two separate experiments. First we measure the effectiveness of the test data that was developed for one technique in terms of the other. Second, we investigate the ability of the test sets to find faults. We place a number of faults into each of our subject programs, and measure the number of faults that are detected by the test sets. Our results indicate that while both techniques are effective, mutation-adequate test sets are closer to satisfying the data flow criterion, and detect more faults.     

基于接口参数的黑箱测试用例自动生成算法

测试用例的选择与生成技术是软件测试尤其是黑箱测试的一个重要研究领域,测试用例的质量将直接决定软件测试的科学性和有效性．该文在一般的测试用例选择方法的基础上,提出了一种基于对接口参数进行组合覆盖的黑箱测试用例自动生成算法模型,据此可以生成一个对所有接口参数进行两两组合覆盖的测试用例表．并证明了该方法产生的测试用例具有数量少、能实现对接口参数最大限度组合覆盖的特点,从而可以在提高软件测试质量的同时,降低成本,提高效率．最后介绍了该算法在作者研究开发的测试数据生成工具中的实际效果．     

Mutation testing on an object-oriented framework: An experience report

Context

The increasing presence of Object-Oriented (OO) programs in industrial systems is progressively drawing the attention of mutation researchers toward this paradigm. However, while the number of research contributions in this topic is plentiful, the number of empirical results is still marginal and mostly provided by researchers rather than practitioners.

Objective

This article reports our experience using mutation testing to measure the effectiveness of an automated test data generator from a user perspective.

Method

In our study, we applied both traditional and class-level mutation operators to FaMa, an open source Java framework currently being used for research and commercial purposes. We also compared and contrasted our results with the data obtained from some motivating faults found in the literature and two real tools for the analysis of feature models, FaMa and SPLOT.

Results

Our results are summarized in a number of lessons learned supporting previous isolated results as well as new findings that hopefully will motivate further research in the field.

Conclusion

We conclude that mutation testing is an effective and affordable technique to measure the effectiveness of test mechanisms in OO systems. We found, however, several practical limitations in current tool support that should be addressed to facilitate the work of testers. We also missed specific techniques and tools to apply mutation testing at the system level.     

对逻辑覆盖软件测试准则的公理化评估

由于形式规格说明采用一种精确、一致、容易被机器自动处理的符号系统来描述软件需求,因而形式规格说明为测试用例的自动生成和软件功能的验证提供了基础.在基于形式规格说明的测试过程中逻辑覆盖测试准则是一组常用的测试准则,如何选择和使用其中的每个测试准则是应用这组测试准则时面临的主要问题.因此分析和比较这组测试准则中每个测试准则的性质将为测试工程师选择测试准则提供指导和帮助.对测试充分性准则的公理化评估是一种比较测试准则的方式,这种方式将对理想的测试准则的直觉需求定义为一组公理,然后通过检查测试准则是否满足该组公理来分析和比较相应的测试准则.描述了一组理想的逻辑覆盖测试准则应该具有的性质和用来确定一个测试充分性准则是否完全的生成算法.这组性质被形式化地定义为一组公理.利用这种形式化的定义,用定理的形式精确地给出了这些性质之间的关系.最后通过这组公理系统来评估现有的逻辑覆盖测试准则.评估的结果为测试人员在实际过程中选择逻辑覆盖测试准则提供了指导.     

面向构件接口变异的模型检验技术研究

随着构件技术在软件开发中的广泛应用,人们对构件化软件系统质量的要求也在不断提高。该文运用模型检验技术进行接口变异测试的相关研究工作。该文首先详细介绍了基于模型检验技术的变异测试的基本原理,随后通过对构件集成时接口之间的交互关系进行分析后,提出了通过构造接口函数调用关系模型进行基于模型检验的接口变异测试方法。最后,通过实例进行了对JavaBean构件的接口测试用例的自动生成。     

Selection and Evaluation of Test Data Based on Genetic Programming

In the literature, we find several criteria that consider different aspects of the program to guide the testing, a fundamental activity for software quality assurance. They address two important questions: how to select test cases to reveal as many fault as possible and how to evaluate a test set T and end the test. Fault-based criteria, such as mutation testing, use mutation operators to generate alternatives for the program P being tested. The goal is to derive test cases capable of producing different behaviors in P and its alternatives. However, this approach usually does not allow the test of interaction between faults since the alternative differs from P by a simple modification. This work explores the use of Genetic Programming (GP), a field of Evolutionary Computation, to derive alternatives for testing P and introduces two GP-based procedures for selection and evaluation of test data. The procedures are related to the above questions, usually addressed by most testing criteria and tools. A tool, named GPTesT, is described and results from an experiment using this tool are also presented. The results show the applicability of our approach and allow comparison with mutation testing.     

Can fault‐exposure‐potential estimates improve the fault detection abilities of test suites?

Code‐coverage‐based test data adequacy criteria typically treat all coverable code elements (such as statements, basic blocks or outcomes of decisions) as equal. In practice, however, the probability that a test case can expose a fault in a code element varies: some faults are more easily revealed than others. Thus, several researchers have suggested that if one could estimate the probability that a fault in a code element will cause a failure, one could use this estimate to determine the number of executions of a code element that are required to achieve a certain level of confidence in that element's correctness. This estimate, in turn, could be used to improve the fault‐detection effectiveness of test suites and help testers distribute testing resources more effectively. This conjecture is intriguing; however, like many such conjectures it has never been directly examined empirically. If empirical evidence were to support this conjecture, it would motivate further research into methodologies for obtaining fault‐exposure‐potential estimates and incorporating them into test data adequacy criteria. This paper reports the results of experiments conducted to investigate the effects of incorporating an estimate of fault‐exposure probability into the statement coverage test data adequacy criterion. The results of these experiments, however, ran contrary to the conjectures of previous researchers. Although incorporation of the estimates did produce statistically significant increases in the fault‐detection effectiveness of test suites, these increases were quite small, suggesting that the approach might not be able to produce the gains hoped for and might not be worth the cost of its employment. Copyright © 2002 John Wiley & Sons, Ltd.     

网络管理接口测试中的一种通用数据串行化表示法

提出了网络管理接口测试中的通用数据串行化表示方法,对基于不同技术的网管接口可能使用到的数据类型的共性进行了分析和抽象,采用一种统一的格式来对不同技术中类似的数据类型值进行表示,并给出了这种表示方法的范式定义。此外,还给出了基于这种通用数据串行化表示法的测试系统中数据编解码模块的重用模型。     

数据流测试准则之间关系的研究

软件测试充分性准则是决定一个软件系统能否被充分测试的规则,而测试充分性的关键问题是其检测故障的能力。该文首先分析了数据流测试基本概念,然后讨论了数据流测试准则之间的关系,最后针对测试准则C1和C2,提出了二者的关系R。R(C1,C2)可保证在特定的错误检测度量方法下,C1比C2具有更好的检错能力。     

基于对象约束语言规格的变异分析

由于传统形式化语言的复杂性以及当前变异分析系统性能问题,基于规格的变异分析一直没有得到广泛的应用.介绍辅助UML建模的对象约束语言(OCL)和基于规格变异分析的基本理论.提出一种新型的OCL规格变异系统的设计方案,并详细描述了一些变异算子的具体实现.解决了传统变异系统的性能缺陷,提高了变异分析的实用性.     

Mutation testing from probabilistic and stochastic finite state machines

Specification mutation involves mutating a specification, and for each mutation a test is derived that distinguishes the behaviours of the mutated and original specifications. This approach has been applied with finite state machine based models. This paper extends mutation testing to finite state machine models that contain non-functional properties. The paper describes several ways of mutating a finite state machine with probabilities (PFSM) or stochastic time (PSFSM) attached to its transitions and shows how we can generate test sequences that distinguish between such a model and its mutants. Testing then involves applying each test sequence multiple times, observing the resultant behaviours and using results from statistical sampling theory in order to compare the observed frequency and execution time of each output sequence with that expected.     

一种Web服务的测试数据自动生成方法

软件测试是保证Web服务质量的重要技术手段.测试数据生成是Web服务测试的重要内容.测试数据的质量将直接影响Web服务测试的效率和成本.文章基于合约式设计的 Web服务测试技术,提出一种 Web服务的测试数据自动生成方法.首先根据WSDL文档采用随机法自动生成初始测试数据,然后使用合约变异技术进行测试数据的选择,据此可以生成一组达到一定合约变异充分度的有效测试数据,从而提高 Web服务的测试质量和效率.最后实现了一个Web服务的测试数据自动生成工具原型,并通过实验验证了方法的有效性.     

基于变异测试和OWL-S文档的Web服务测试用例生成

由于缺乏必要的细节信息,针对Web服务的测试难度较大。文章的方法是将变异算子作用到OWL-S文档上,以得出不同输出为目标,从而生成测试用例。该方法避免了生成大量的变异体以及判定等价变异体,能降低测试成本。在保证其有效性的同时,还可以根据具体的应用来调整变异算子,保证方法的通用性和可扩展性。     

一种基于本体的测试用例研究与应用

测试用例是软件测试的重要组成,也是软件测试成败的关键性因素之一。测试用例的完备性和复用性研究已经成为当前软件测试领域的两大热点。论文提出一种基于本体论的测试用例分析与刻画方法,它统一了测试用例的内涵,并为测试用例提供了可穷举的外延选择。该方法解决了软件测试用例完备性和复用性问题。