蜕变测试和断言检查的比较与实验研究

在软件测试中,测试预言是一种用于检查程序在测试中是否正常运行的机制.然而在某些实际情况下,还无法制定测试预言或者难以有效地应用测试预言.针对此类测试预言问题,蜕变测试于近年应运而生,但蜕变测试的效率问题还没有被充分地加以研究.作者用控制实验的方法研究了使用蜕变测试的成本及效率,进而将蜕变测试和常用的断言检查两种方法的错误检测率和时间成本进行了比较和分析.实验结果表明,相比于断言检查方法,蜕变测试具有检测到更多错误的潜力.通过分析蜕变测试的效率和性能,与断言测试相比,蜕变测试的错误检测率更高效而效率有待提高,可适用于较为粗粒度的测试需求.

Experimental Study to Compare the Use of Metamorphic Testing and Assertion Checking

A test oracle in software testing is a mechanism for checking whether the program under test behaves correctly for any execution. In some practical situations, oracles can be unavailable or too xpensive to apply. Metamorphic testing (MT) was proposed to alleviate this problem so that software can be delivered under the time-to-market pressure. However, the effectiveness of MT has not been studied adequately. This paper conducts a controlled experiment to investigate the cost effectiveness of using MT. The fault detection capability and time cost of MT are compared with the standard assertion checking method. The results show that MT has potentials to detect more faults than the assertion checking method. The experimental results also show a trade-off between the two testing methods: MT can be less efficient but more effective, and can be defined at a coarser level of granularity than the assertion checking method.