一种基于UML状态图的规约变异测试方法

规约变异测试从软件功能的角度,对规约进行分析,从而揭示规约中存在的问题。本文提出一种基于UML状态图的变异测试方法,针对每种变异算子,分析其是否会引入冲突,进而有效避免不合理的变异操作;分析了每种变异算子产生等价变异体的条件,能够在生成变异体的同时检测并移除等价变异体,进而减少其对测试过程的影响;给出了杀掉每种变异体所需满足的条件,可在此基础上产生杀掉特定变异体所需的测试用例,从而提高测试用例集的质量。在此基础上,根据变异算子的实际功能,整合了功能相同的算子,减少了变异算子的数量,从而进一步降低了变异测试的开销。实验结果表明,本方法能够较好地提高测试用例的质量,进而提升测试的效率。     

消息传递并行程序的弱变异测试及其转化

并行程序执行的不确定性,增加了测试的复杂性和难度.研究消息传递并行程序的变异测试,提出其弱变异测试转化方法,以提高该程序变异测试的效率.首先,根据消息传递并行程序包含语句的类型和语句变异之后导致的变化构建相应的变异条件语句;然后,将构建好的所有变异条件语句插入到原程序中,形成新的被测程序,从而将原程序的弱变异测试问题转化为新程序的分支覆盖问题.这样做的好处是,能够利用已有的分支覆盖方法解决变异测试问题.将该方法应用于8个典型的消息传递并行程序测试中,实验结果表明,该方法不但是可行的,也是必要的.     

一种新的变异测试数据自动生成方法

变异测试是一种行之有效的软件测试方法,通过使用变异算子产生变异体系统地模拟软件中的各种缺陷,然后构造能够杀死这些变异体的测试数据集.自动生成能够杀死变异体的测试数据将提高变异测试的效率和有效性.当前的研究工作只考虑生成杀死单个变异体的测试数据.文中根据杀死同一位置的多个变异体的条件相近的特点,提出一种对杀死这些变异体的条件进行组合,然后生成同时杀死该位置多个变异体的测试数据的方法;给出相应的支持工具,并且通过实验验证方法的有效性.     

基于统计占优分析的变异测试

为数众多的变异体产生的高昂测试代价严重影响了变异测试技术在实际程序中的应用.为了大幅度减少弱变异测试中变异体的数量,提出基于统计占优分析的变异体约简方法.该方法首先利用变异前后的语句构造变异分支,并将所有变异分支集成到原程序中,形成新的被测程序;然后,通过统计测试用例对各个变异分支的覆盖信息,确定变异分支之间的占优关系;最后得到非被占优分支集,其对应的变异体就是约简后的变异体.将该方法用于8个程序的测试,结果表明:该方法能够约简平均90%的变异体,从而显著提高了变异测试的效率.     

An Experimental Evaluation of Data Flow and Mutation Testing

Two experimental comparisons of data flow and mutation testing are presented. These techniques are widely considered to be effective for unit-level software testing, but can only be analytically compared to a limited extent. We compare the techniques by evaluating the effectiveness of test data developed for each. We develop ten independent sets of test data for a number of programs: five to satisfy the mutation criterion and five to satisfy the all-uses data-flow criterion. These test sets are developed using automated tools, in a manner consistent with the way a test engineer might be expected to generate test data in practice. We use these test sets in two separate experiments. First we measure the effectiveness of the test data that was developed for one technique in terms of the other. Second, we investigate the ability of the test sets to find faults. We place a number of faults into each of our subject programs, and measure the number of faults that are detected by the test sets. Our results indicate that while both techniques are effective, mutation-adequate test sets are closer to satisfying the data flow criterion, and detect more faults.     

变异测试:原理、优化和应用

变异测试是一种基于缺陷的软件测试技术,在近四十年得到国内外学者的广泛关注,并取得了一些研究成果。对已有的研究工作进行总结,将其分为变异测试原理、优化和应用三个模块。其中在变异测试原理模块中,给出变异测试的基本假设,对变异测试分析流程进行介绍,并对其中的重要概念依次给出定义,从静态检测和动态检测两个角度对等价变异体检测技术进行总结。在变异测试优化模块中,从变异体选择优化和变异体执行优化两个角度对已有研究工作进行总结。在变异测试应用模块中,选择了测试用例集充分性评估、测试用例生成和回归测试三个应用领域,对研究工作进行分类总结。最后对变异测试的未来研究方向进行了展望。     

一种基于约束的变异测试数据生成方法

作为衡量测试用例集完备性的测试策略,变异测试是一种“面向缺陷”的单元测试技术,主要用来生成完备的测试用例集.其中面向路径测试数据生成技术通过约束系统构造和求解过程实现用例集生成,是一种高效的测试用例生成技术.但目前大部分面向路径测试用例生成技术只考虑了程序语句间的控制依赖,即通过对控制流图的分析来构建约束系统,而忽略了语句间的数据依赖对约束系统的影响.充分考虑两种依赖关系,针对域削减的测试数据生成技术进行了改进,提出了一种考虑数据依赖的域削减方法.实验表明,这种方法在变异测试数据生成的成功率和执行效率上都有较大程度的提高.     

Improving the efficiency of automated protocol implementation using Estelle

Correctness and runtime efficiency are essential properties of software in general and of high-speed protocols in particular. Establishing correctness requires the use of FDTs during protocol design, and to prove the protocol code correct with respect to its formal specification. Another approach to boost confidence in the correctness of the implementation is to generate protocol code automatically from the specification. However, the runtime efficiency of this code is often insufficient. This has turned out to be a major obstacle to the use of FDTs in practice. One of the FDTs currently applied to communication protocols is Estelle. We show how runtime efficiency can be significantly improved by several measures carried out during the design, implementation and runtime of a protocol. Recent results of improvements in the efficiency of Estelle-based protocol implementations are extended and interpreted.     

对逻辑覆盖软件测试准则的公理化评估

由于形式规格说明采用一种精确、一致、容易被机器自动处理的符号系统来描述软件需求,因而形式规格说明为测试用例的自动生成和软件功能的验证提供了基础.在基于形式规格说明的测试过程中逻辑覆盖测试准则是一组常用的测试准则,如何选择和使用其中的每个测试准则是应用这组测试准则时面临的主要问题.因此分析和比较这组测试准则中每个测试准则的性质将为测试工程师选择测试准则提供指导和帮助.对测试充分性准则的公理化评估是一种比较测试准则的方式,这种方式将对理想的测试准则的直觉需求定义为一组公理,然后通过检查测试准则是否满足该组公理来分析和比较相应的测试准则.描述了一组理想的逻辑覆盖测试准则应该具有的性质和用来确定一个测试充分性准则是否完全的生成算法.这组性质被形式化地定义为一组公理.利用这种形式化的定义,用定理的形式精确地给出了这些性质之间的关系.最后通过这组公理系统来评估现有的逻辑覆盖测试准则.评估的结果为测试人员在实际过程中选择逻辑覆盖测试准则提供了指导.     

Contract-based Mutation Testing in the Refinement Calculus

This article discusses mutation testing strategies in the context of refinement. Here, a novel generalization of mutation testing techniques is presented to be applied to contracts ranging from formal specifications to programs. It is demonstrated that refinement and its dual abstraction are the key notions leading to a precise and yet simple theory of mutation testing. The refinement calculus of Back and von Wright is used to express the concepts like contracts, useful mutations, test-cases and test-coverage.     

TestEra: Specification-Based Testing of Java Programs Using SAT

TestEra is a framework for automated specification-based testing of Java programs. TestEra requires as input a Java method (in sourcecode or bytecode), a formal specification of the pre- and post-conditions of that method, and a bound that limits the size of the test cases to be generated. Using the method's pre-condition, TestEra automatically generates all nonisomorphic test inputs up to the given bound. It executes the method on each test input, and uses the method postcondition as an oracle to check the correctness of each output. Specifications are first-order logic formulae. As an enabling technology, TestEra uses the Alloy toolset, which provides an automatic SAT-based tool for analyzing first-order logic formulae. We have used TestEra to check several Java programs including an architecture for dynamic networks, the Alloy-alpha analyzer, a fault-tree analyzer, and methods from the Java Collection Framework.     

Integrated Estelle and ASN.1 specification approach to automatic implementation of application protocol

Estelle currently does not support ASN.1 and encoding and decoding rules. Tools developed for Estelle and ASN.1 have been developed independently of each other. As such, ASN.1 tools do not support Estelle and vice versa. At present, due to these two separate specifications, implementation is time consuming. It is imperative to find a way for ASN.1 and Estelle specifications to be efficiently integrated to facilitate fully automatic implementation. This paper describes how Estelle and ASN.1 can be integrated to give a unified specification which can then be fed into a software environment to produce a truly automatic implementation of an application protocol, including encoding and decoding of protocol data units, when application layer protocols are specified in Estelle and in ASN.1. The integration approach is based on the powerful programming concept, Data Abstraction. The software that implements this integrated specification approach to automatic implementation of application protocol is called EASE (Estelle and ASN.1 Software Environment). To demonstrate the viability of the approach, the unified specification of the ISO ACSE and Presentation protocols are also presented.     

基于XMLAPI的组件扩展接口变异测试方法

在传统的组件接口变异测试方法基础上,提出一种基于XML API的组件扩展接口变异测试方法.首先给出组件扩展接口测试的框架,建立起扩展接口的定义模型.XML API在组件内部建立处理数据集的XML校验器,实现了原组件接口的扩展.借助组件外部的XML Schema变异算子,完成了组件内部数据集的自动验证和组件接口参数的测试.该方法具有多种优点,例如可视的多功能测试接口、可跨平台的通用性的测试语言等.实验表明,该方法可以应用于COM,CORBA,EJB等多种组件的测试环境.     

针对JavaScript浏览器兼容性的变异测试方法

针对JavaScript浏览器兼容性问题缺乏有效的测试方法的问题,基于变异测试技术,对Web应用程序中的JavaScript语言在主流浏览器中的兼容性进行了分析,设计了18个针对浏览器兼容性错误的变异算子并开发了自动化的测试工具Compatibility Mutator。该程序使用抽象语法树（AST）分析JavaScript语法,使用Selenium WebDriver自动调用多个浏览器核心并行执行变异测试。通过对7个广泛使用的JavaScript框架的实验,验证了此方法对每一个框架都能够生成一定数量的变异体,其中对jQuery和YUI框架的变异测试的变异得分分别为43.06%和7.69%,表明了所设计的变异算子能够触发浏览器兼容性方面的错误,并且能够评估测试用例集的完备性。     

基于变异测试和OWL-S文档的Web服务测试用例生成

由于缺乏必要的细节信息,针对Web服务的测试难度较大。文章的方法是将变异算子作用到OWL-S文档上,以得出不同输出为目标,从而生成测试用例。该方法避免了生成大量的变异体以及判定等价变异体,能降低测试成本。在保证其有效性的同时,还可以根据具体的应用来调整变异算子,保证方法的通用性和可扩展性。     

Specification-based Testing for Gui-based Applications

The development of GUI-based applications has raised a lot of new issues, one of them being how to automate effective testing for applications with complicated graphical user interactions. In this paper, we discuss the architectural issues and the implementation concerns of our approach to an automated specification-based testing technique for GUI-based applications. This approach is carried out by enriching existing architecture for automated specification-based testing. An essential part of our work is a visual environment to obtain test specifications. This environment pre-runs the Application Under Test (AUT) under its own control, with two prominent characteristics: First, testers can edit test specifications within the true GUI environment of the AUT. Second, the recorded input and output contain the same references as those in the AUT, so that the test cases generated from the edited specification can be used directly by test oracles during the automated testing procedure.We present our running prototype of a visual specification editor that allows users to graphically manipulate test specifications when these specifications are given in term of Finite State Machines (FSM) and the implementations of the AUT are GUI-based Java applications.     

协议形式描述语言Estelle的交互生成方法

形式描述是协议工程的基础。本文提出一种支持国际标准形式描述语言Estelle 描述生成的交互式生成方法。协议文本可划分为用有限状态自动机(FSM)描述的协议功能部分和用自然语言描述的动作部分。该方法把FSM 描述自动变换成Estelle 描述,把动作描述由自然语言交互地变换成Estelle 描述后,再组合得到完整的Estelle 描述。     

Uncovering ISO ROSE protocol errors using Estelle

One of the main objectives of ISO in developing FDTs is that protocol specified in them can be verified. However, standardized FDTs have been designed largely for specification purpose; success of using them for protocol verification has been rarely reported. We have developed a technique of translating Estelle specifications into Numerical Petri nets, which can then be verified by a proven automated verification tool, PROTEAN. The merits of our approach are that specifications are fully based on standard Estelle, and dynamic behaviours of an Estelle specification can be handled. In this paper, we present a success story of using Estelle and the techniques we have developed to uncover ISO ROSE protocol errors. We find that Estelle is an FDT capable of analysing and verifying real protocols and it is therefore important to the development of ISO protocol standards.     

基于合约变异的构件软件测试技术

介绍基于规约说明的变异算法在测试数据生成中的应用思想,分析不同变异算子与故障类别之间的蕴含关系,把这种算法应用到构件合约变异中,提高了测试数据的有效性,简化了最终测试数据集,克服了传统构件合约变异测试产生大量变异体带来的测试低效性。     

MujavaX: 一个支持非均匀分布的变异生成系统

变异分析是一种广泛用来评估软件测试技术性能的方法.已有的变异分析技术通常将变异算子平均地应用于原始程序.由于现实程序中的故障分布往往具有群束的特征,采用平均分布的变异分析方法不能客观地评估软件测试技术的性能.前期研究工作中提出了非均匀分布的变异分析方法,采用实例研究验证了不同的故障分布对测试技术性能评估的影响.为了增强非均匀分布的变异分析方法的实用性,开发了支持非均匀分布的变异生成系统MujavaX,该系统是对广泛实践的Mujava工具的扩展与改进.采用一个实例系统验证了开发的MujavaX的正确性与可行性,实验结果表明该系统能够生成指定分布的非均匀变体集合.