蜕变测试技术综述

软件测试是一种重要的、不可缺少的软件质量保证技术,用于发现和纠正软件中存在的缺陷和错误,但在很多情况下待测程序的预期输出难以确定。蜕变测试技术通过检查程序的多个执行结果之间的关系来测试程序,可以有效地解决上述问题。经过近十年的研究,蜕变测试技术已经在测试过程的优化、与其他验证或测试方法的结合等方面取得了巨大的进展,并被广泛地应用于各个领域中。对当前蜕变测试技术的研究进行了综述,针对已有方法的不足之处,对未来的研究方向进行了展望,包括蜕变测试充分性研究、实用蜕变关系构造技术、实用原始测试用例选取技术、新型软件中蜕变测试技术的研究、蜕变测试工具的开发等。     

蜕变测试研究进展及其在并行程序测试中的研究展望

在软件测试过程中,待测程序的预期输出是判断软件是否存在缺陷的重要因素.蜕变测试技术是利用被测软件的属性来检查程序输出,从而有效地解决程序预期输出难以构造的问题.近年来,蜕变测试在软件测试领域取得了蓬勃的发展,许多研究人员将蜕变测试技术进行优化,将其运用到各个领域,有效提高了软件质量.从原理、过程及其优化,应用领域3个方面,总结蜕变测试的研究工作,着重分析了近5年的研究进展,进一步展望了蜕变测试用于并行程序时,可能的研究主题.首先,介绍蜕变测试的基本概念和蜕变测试过程;接着,从蜕变关系、测试用例、测试执行过程以及蜕变测试工具4个角度,总结蜕变测试优化技术;然后,汇总了蜕变测试的应用领域;最后,基于已有研究成果,讨论蜕变测试在并行程序测试领域面临的问题,为蜕变技术在并行程序测试领域的研究提供可能的思路.     

基于程序路径分析的有效蜕变测试

蜕变测试对于预期输出难以构造的程序是实用和高效的.作者在系统研究已有蜕变测试方法和路径分析技术的基础上,首先针对使用二元蜕变关系的测试提出了一组蜕变测试准则,以在多个不同的层次上定义蜕变测试用例集的充分性;然后给出了3种能够生成相应测试用例集的算法;最后通过变异分析的方法证实这些算法的有效性.实验结果表明,蜕变关系和测试准则的选取直接影响到测试的效果,另外,使用蜕变关系全路径覆盖可满足性算法(APCEMsT)可以快速准确地发现待测程序中的错误,而生成的测试用例的数量却比传统技术要少.     

蜕变关系构造基本准则与策略研究

蜕变测试可以部分解决软件测试中的Oracle问题,其关键步骤和难点是蜕变关系的构造,它将直接影响测试的效果。通过对典型程序测试的案例对蜕变关系的构造进行分析,归纳总结了若干构造蜕变关系的基本准则,并在案例研究中采用变异分析方法验证了构造准则的合理性。提出了蜕变测试与等价类测试结合运用的测试方法,此方法可用于输入空间易于分类的程序。     

面向服务软件的蜕变测试方法

在面向服务软件的测试过程中,由于在服务发现之前不可知的交互对象和同一个服务可能会有不同实现,往往出现程序执行结果不能提前预知的Oracle问题。为了有效地解决面向服务软件测试中的Oracle问题,基于面向服务架构(SOA)的特点,提出将蜕变测试方法用于面向服务软件的单元测试和集成测试过程中,依据面向服务软件每个服务的自身性质构造蜕变关系,设计蜕变测试类执行测试用例并验证蜕变关系是否保持,如果蜕变关系被违反了,则发现和报告缺陷,从而有效地支持面向服务软件的测试。     

坡度坡向量算程序蜕变测试方法

坡度、坡向量算是地理信息系统的主要功能之一。然而,由于截断误差、舍入误差以及误差传播的影响,导致测试判定难以获取。基于此,提出将蜕变测试应用于坡度、坡向量算程序的测试中,通过分析坡度、坡向量算程序功能的几何属性、数值计算特性以及具体实现算法等提出蜕变关系,并通过分析蜕变关系的适用范围,形成坡度坡向量算程序蜕变测试方法。最后,通过实例研究,验证了提出的蜕变关系在消除不同类型变异方面的有效性。实验结果表明,该方法可有效解决坡度、坡向量算程序的测试判定问题,同时也为解决地理信息系统中其他空间度量程序的测试判定问题提供了借鉴,进一步拓展了蜕变测试技术的应用范围。     

面向对象软件测试的蜕变关系构造方法

针对面向对象软件类级测试中方法序列调用时的Oracle问题,提出了一种基于代数规格的面向对象蜕变关系构造方法,并根据此方法进行蜕变测试。首先基于代数规格给出面向对象测试中蜕变关系的构造准则,然后根据构造准则对GFT(Generating a Finite number of Test cases)算法中范式构造蜕变关系方法进行改进,最后通过一个整数栈类对提出的蜕变关系构造准则进行了验证测试,并与GFT算法中范式构造蜕变关系方法进行了对比,结果表明在相似的变异检测率下,改进后的方法使得构造的蜕变关系冗余度降低了66%,验证了改进的蜕变关系构造方法有更低的蜕变关系冗余度,提高了软件测试效率。     

一种基于蜕变关系的测试与失效测试用例定位模型

针对传统的蜕变测试模型MTM存在的局限性,提出了一种基于蜕变关系的测试模型MRTM。首先通过对比分析,指出了MRTM的适用范围等特点;其次,针对MTM和MRTM都面临的失效测试用例难以确定的难题,提出了一种基于可疑度计算的蜕变测试失效测试用例定位方法FTCL-MT。FTCL-MT作为对已有测试模型的补充,能够在蜕变关系不满足的情况下实现精确定位失效测试用例,从而能够为现有的故障定位技术提供支持。最后,通过实验验证了FTCL-MT方法的有效性。     

面向条件筛选搜索系统的蜕变测试方法

为缓解拥有庞大数据信息的条件筛选搜索系统搜索结果时带来的Oracle问题,提出蜕变测试方法.通过识别程序多次输入输出之间的关系是否违反蜕变关系,可有效缓解Oracle问题.因此,有效识别蜕变关系是完成蜕变测试的前提.根据条件筛选搜索系统的自定义条件,设计两种蜕变关系模式帮助简化蜕变关系的识别,提高蜕变测试的故障检测质量.实验结果表明,利用改进的蜕变关系模式在条件筛选搜索系统中执行蜕变测试提高了14％的精确率,验证了该方法识别蜕变关系的简洁性和有效性.     

针对模型转换程序的蜕变测试方法研究

模型驱动架构中模型转换结果正确与否常常难以判断(即测试Oracle问题),而蜕变测试通过验证多个执行结果之间是否满足蜕变关系可以部分地解决测试Oracle问题。为有效地解决模型转换测试中的Oracle问题,以UML到Java模型转换程序为例,应用蜕变测试,依据转换规则,从增加、删除、修改、替换4个方面设计并构造出一组蜕变关系。最后对待测程序植入在实际测试中常见的两种错误,设计并执行测试用例后验证蜕变关系,发现违反了蜕变关系,暴露出程序缺陷,从而说明了蜕变测试的有效性。     

Tabular-expression-based method for constructing metamorphic relations

Metamorphic testing (MT) is proposed to overcome the oracle problem in software testing, and metamorphic relations (MRs) are the core of MT. There is a lack of guidelines for constructing effective MRs, and it is difficult to reuse MRs mainly because most MRs are closely related to the domain knowledge. In this article, we propose a method for constructing MRs from specifications in tabular expression format. Our method constructs MRs according to the characteristics of tabular expressions, especially the relationships between the header grids and the main grid, namely, our method is domain-independent and the construction process is simplified. In addition, the derived MRs can be applied to specifications with the same tabular expression structure. For specifications with different tabular expression structures, MRs can still be used after slight adjustments. To evaluate the performance of our method in practice, we apply the method to five applications. The experimental results demonstrate that our method is effective for a program with the oracle problem, and that it is applicable to tabular expressions in various formats. Compared with representative testing methods, our method identifies errors that are not detected by the compared methods. Hence, our method and existing methods can complement each other. The MR proposed in this article outperforms MRs constructed based on program properties.     

蜕变测试和断言检查的比较与实验研究

在软件测试中,测试预言是一种用于检查程序在测试中是否正常运行的机制.然而在某些实际情况下,还无法制定测试预言或者难以有效地应用测试预言.针对此类测试预言问题,蜕变测试于近年应运而生,但蜕变测试的效率问题还没有被充分地加以研究.作者用控制实验的方法研究了使用蜕变测试的成本及效率,进而将蜕变测试和常用的断言检查两种方法的错误检测率和时间成本进行了比较和分析.实验结果表明,相比于断言检查方法,蜕变测试具有检测到更多错误的潜力.通过分析蜕变测试的效率和性能,与断言测试相比,蜕变测试的错误检测率更高效而效率有待提高,可适用于较为粗粒度的测试需求.     

机器学习图像分类程序的蜕变测试框架

机器学习在计算机视觉、语音识别和自然语言处理等实际应用中已经取得了显著的成功。图像分类作为计算机视觉的一个主要分支。不久的将来,许多的图像分类程序会以机器学习的方式呈现。然而,由于机器学习图像分类程序的测试面临着测试预言难题,这使得在测试的过程中将需要大量的人力及物力。为了缓解测试预言难题,使用了蜕变测试技术。为了规范测试流程、提高测试效率,提出了一种适用于机器学习图像分类程序的蜕变测试框架。并且通过测试基于SVM和VGGNet图像分类程序,验证了该测试框架的合理性和有效性。     

Finding failures from passed test cases: improving the pattern classification approach to the testing of mesh simplification programs

Mesh simplification programs create three‐dimensional polygonal models similar to an original polygonal model, and yet use fewer polygons. They produce different graphics even though they are based on the same original polygonal model. This results in a test oracle problem. To address the problem, our previous work has developed a technique that uses a reference model of the program under test to train a classifier. Using such an approach may mistakenly mark a failure‐causing test case as passed. It lowers the testing effectiveness of revealing failures. This paper suggests piping the test cases marked as passed by a statistical pattern classification module to an analytical metamorphic testing (MT) module. We evaluate our approach empirically using three subject programs with over 2700 program mutants. The result shows that, using a resembling reference model to train a classifier, the integrated approach can significantly improve the failure detection effectiveness of the pattern classification approach. We also explain how MT in our design trades specificity for sensitivity. Copyright © 2009 John Wiley & Sons, Ltd.     

Coupling-based criteria for integration testing

Integration testing is an important part of the testing process, but few integration testing techniques have been systematically studied or defined. The goal of this research is to develop practical, effective, formalizable, automatable techniques for testing of connections between components during software integration. This paper presents an integration testing technique that is based on couplings between software components. This technique can be used to support integration testing of software components, and satisfies part of the USA's Federal Aviation Authority's requirements for structural coverage analysis of software. The coupling-based testing technique is described, and the coverage criteria for three types of couplings are defined. Techniques and algorithms for developing coverage analysers to measure the extent to which a test set satisfies the criteria are presented, and results from a comparative case study are presented. © 1998 John Wiley & Sons, Ltd.     

The Economics of Unit Testing

Conventional wisdom and anecdote suggests that testing takes between 30 to 50% of a project's effort. However testing is not a monolithic activity as it consists of a number of different phases such as unit testing, integration testing and finally system and acceptance test. Unit testing has received a lot of criticism in terms of the amount of time that it is perceived to take and its perceived costs. However it still remains an important verification activity being an effective means to test individual software components for boundary value behavior and ensure that all code has been exercised adequately. We examine the available data from three safety-related, industrial software projects that have made use of unit testing. Using this information we argue that the perceived costs of unit testing may be exaggerated and that the likely benefits in terms of defect detection are quite high in relation to those costs. We also discuss the different issues that have been found applying the technique at different phases of the development and using different methods to generate those tests. We also compare results we have obtained with empirical results from the literature and highlight some possible weakness of research in this area.