Automatic, evolutionary test data generation for dynamic software testing

This paper proposes a dynamic test data generation framework based on genetic algorithms. The framework houses a Program Analyser and a Test Case Generator, which intercommunicate to automatically generate test cases. The Program Analyser extracts statements and variables, isolates code paths and creates control flow graphs. The Test Case Generator utilises two optimisation algorithms, the Batch-Optimistic (BO) and the Close-Up (CU), and produces a near to optimum set of test cases with respect to the edge/condition coverage criterion. The efficacy of the proposed approach is assessed on a number of programs and the empirical results indicate that its performance is significantly better compared to existing dynamic test data generation methods.     

软件测试用例自动生成算法综述*

按照测试用例自动生成技术的不同,将测试用例自动生成算法分为随机、遗传、蚁群、粒子群四类,对上述各类算法的现状和进展进行介绍、分析和探讨。最后,对软件测试用例自动生成的研究进行了总结。     

Defining a test coverage criterion for model-level testing of FBD programs

ContextThe Programmable Logic Controller (PLC) is being integrated into the automation and control of computer systems in safety–critical domains at an increasing rate. Thoroughly testing such software to ensure safety is crucial. Function Block Diagram (FBD) is a popular data-flow programming language for PLC. Current practice often involves translating an FBD program into an equivalent C program for testing. Little research has been conducted on coverage of direct testing a data-flow program, such as an FBD program, at the model level. There are no commonly accepted structural test coverage criteria for data-flow programs. The objective of this study is to develop effective structural test coverage criterion for testing model-level FBD programs. The proposed testing scheme can be used to detect mutation errors at the logical function level.ObjectiveThe purpose of this study is to design a new test coverage criterion that can directly test FBD programs and effectively detect logical function mutation errors.MethodA complete test set for each function and function block in an FBD program are defined. Moreover, this method augments the data-flow path concept with a sensitivity check to avoid fault masking and effectively detect logical function mutation errors.ResultsPreliminary experiments show that this test coverage criterion is comprehensive and effective for error detection.ConclusionThe proposed coverage criterion is general and can be applied to real cases to improve the quality of data-flow program design.     

面向云测试的并行测试用例自动生成方法

为实现云计算环境下的高效软件测试,提出一种Web 应用系统功能测试的并行测试用例自动生成方法。该方法首先根据场景流图采用深度优先遍历算法生成并行测试路径,录制组合产生并行测试脚本,并进行参数化处理;然后,使用基于搜索的软件测试(SBST)方法自动生成可经过目标路径的有效测试数据集,脚本与数据耦合形成大量可并行部署的自动化测试用例。为验证方法的有效性设计了自动化云测试原型系统。实验结果表明,该并行测试用例自动生成方法可以高效地为云平台提供测试用例输入,提高测试效率。     

基于MDA的软件测试用例生成

将MDA中模型驱动的软件代码自动化生成思想应用于模型驱动的软件测试用例自动化生成。从UML/OCL模型出发,采用缺陷测试理论、变异分析技术,结合约束处理规则,开发一个可以自动生成单元测试用例的框架,提高软件测试的自动化程度,从整体上提高软件的开发效率。     

Web服务自动化测试技术

赋时Petri网为装配序列规划提供了有效的建模方法,但其在求解最优装配序列时受到组合复杂性的严重制约。零压缩二叉决策图(ZBDD)是处理大规模组合集合和0-1稀疏向量的一种有效符号技术,能够有效缓解组合爆炸问题。将赋时Petri网与ZBDD结合起来,给出了一种求解装配序列最优解的有效方法。首先通过转换算法将赋时Petri网转换为等价的普通Petri网,接下来给出普通Petri网可达状态及迁移引发函数的ZBDD表示方法,最后基于ZBDD给出最优装配序列求解算法。实例验证表明,该算法在求解过程中通过隐式符号操作实现了Petri网的可达状态搜索,有效缓解了计算过程中的组合复杂性。     

面向安全苛求软件的测试用例自动生成

测试用例的自动生成是验证安全苛求软件最关键的技术问题,然而目前的研究并没有充分考虑安全苛求软件的安全性需求,为此提出一种应用安全覆盖准则的安全苛求软件的测试用例自动生成策略,将该策略应用于铁路车站计算机连锁软件,并与全节点覆盖准则进行了比较。结果表明该策略对关键变迁有更高的安全性保证。     

Automated test case generation for the stress testing of multimedia systems

With the advancement in network bandwidth and computing power, multimedia systems have become a popular means for information delivery. However, general principles of system testing cannot be directly applied to testing of multimedia systems on account of their stringent temporal and synchronization requirements. In particular, few studies have been made on the stress testing of multimedia systems with respect to their temporal requirements under resource saturation. Stress testing is important because erroneous behavior is most likely to occur under resource saturation. This paper presents an automatable method of test case generation for the stress testing of multimedia systems. It adapts constraint solving techniques to generate test cases that lead to potential resource saturation in a multimedia system. Coverage of the test cases is defined upon the reachability graph of a multimedia system. The proposed stress testing technique is supported by tools and has been successfully applied to a real‐life commercial multimedia system. Although our technique focuses on the stress testing of multimedia systems, the underlying issues and concepts are applicable to other types of real‐time systems. Copyright © 2002 John Wiley & Sons, Ltd.     

基于UML协作图的测试用例生成方法与实现

根据组件软件的特征,提出一种基于UML协作图直接产生测试用例的方法.给出了一系列覆盖准则,并针对协作图中条件、并发和循环消息流提出相应的处理方法.采用深度优先的算法遍历所有测试场景,每个测试场景相应于一个测试用例,并设计和实现了测试用例自动生成工具系统UMLTCG.     

Formalizing interoperability for test case generation purpose

This study deals with interoperability testing of protocol implementations. The objective is to define a method for automatic test derivation based on formal definitions. First, the notion of interoperability criteria is introduced. They formally describes the conditions that two implementations must verify in order to be considered interoperable. Then, based on the equivalence of two of the defined interoperability criteria, we propose a method to derive automatically interoperability test cases that avoids the state-space explosion problem.     

Dynamic method for software test data generation

Test data generation in program testing is the process of identifying a set of test data which satisfies a given testing criterion. Existing pathwise test data generators proceed by selecting program paths that satisfy the selected criterion and then generating program inputs for these paths. One of the problems with this approach is that unfeasible paths are often selected; as a result, significant computational effort can be wasted in analysing those paths. In this paper, an approach to test data generation, referred to as a dynamic approach for test data generation, is presented. In this approach, the path selection stage is eliminated. Test data are derived based on the actual execution of the program under test and function minimization methods. The approach starts by executing a program for an arbitrary program input. During program execution for each executed branch, a search procedure decides whether the execution should continue through the current branch or an alternative branch should be taken. If an undesirable execution flow is observed at the current branch, then a real-valued function is associated with this branch, and function minimization search algorithms are used to locate values of input variables automatically, which will change the flow of execution at this branch.     

应用UML顺序图的联锁测试用例生成方法

计算机联锁软件是安全完善度等级最高的软件, 因此, 在投入使用前对联锁软件的测试是必不可少的。为生成联锁软件测试用例, 结合包含有充分测试信息的UML模型, 提出一种以UML顺序图为主要模型的软件测试用例生成方法。通过提出有向图SG生成算法, 将顺序图转换为SG; 然后基于特定的覆盖准则和采用深度优先搜索算法遍历SG, 得到场景的输入、预期输出、约束条件以及场景环境, 生成测试用例; 最后, 以进路建立过程为例验证了该方法的可行性和正确性。     

Automatic test case generation from Simulink/Stateflow models using model checking

Model‐based test generation techniques based on random input generation and guided simulation do not satisfy the demands of high test coverage and completeness guarantees as required by safety‐critical applications. Recently, test generation techniques based on model checking have been reported to bridge this gap. To evaluate the effectiveness of these techniques, an in‐house tool suite, AutoMOTGen, has been developed for Simulink/Stateflow and applied on real‐life case studies at General Motors. This paper outlines the test generation methodology of AutoMOTGen and gives a comparative study with a commercial, primarily random input‐based, test generation tool on the same set of examples. The results indicate that in terms of coverage, model checking‐based techniques complement the random input‐based techniques. In addition, they provide proofs for unreachability that can aid in debugging the models. Therefore, it is recommended that model checking‐based tools be utilized to complement and enhance the effectiveness of model‐based testing methods in safety‐critical systems engineering. Copyright © 2013 John Wiley & Sons, Ltd.     

A method of test case set generation in the commutativity test of reduce functions

MapReduce framework has become one of the more popular big data processing frameworks. In the MapReduce framework, the test of the commutativity problem of the reduce function may take a lot of time and space. The aim of this study is to reduce the size of the test case set required to detect the commutativity problem of the reduce function when the initial test cases are available. With the initial test case as input, this method will determine the pattern of the function according to the information when the function is running, and select the corresponding test case according to the characteristics of each pattern to generate the subsequent test case set. Experiments on 170 reduce functions can verify that the test case set generated by our method is effective when testing the commutativity of reduce functions. And compared with the existing test case generation strategy, our method can have a smaller test case set to achieve the same accuracy. When detecting the commutativity of reduce functions, the test case set generation strategy we proposed can detect the commutativity of functions with a smaller test scale, and has a high recall.     

A semantic approach for automated test oracle generation

This paper presents the design, implementation, and applications of a software testing tool, TAO, which allows users to specify and generate test cases and oracles in a declarative way. Extended from its previous grammar-based test generation tool, TAO provides a declarative notation for defining denotational semantics on each productive grammar rule, such that when a test case is generated, its expected semantics will be evaluated automatically as well, serving as its test oracle. TAO further provides a simple tagging mechanism to embed oracles into test cases for bridging the automation between test case generation and software testing. Two practical case studies are used to illustrate how automated oracle generation can be effectively integrated with grammar-based test generation in different testing scenarios: locating fault-inducing input patterns on Java applications; and Selenium-based automated web testing.     

Automated Test Case Generation for BDI Agents

We propose a coverage oriented test case generation methodology for BDI multi-agent systems. The coverage criteria involve plans and nodes within plans of multi-agent systems. We organise the criteria into a subsumption hierarchy to show the coverage relationships between the criteria. Then we apply the criteria on multi-agent systems to analyse some empirical data. The data analysed is the effect on the number of test cases generated automatically for each criterion. We use a tool, BDITESTER, to obtain the empirical data and to show that our proposal is pragmatic. Finally, we suggest some guidelines to select a criterion to automatically generate test cases for BDI agents.     

基于OFGA的软件测试用例自动生成

为提高测试用例生成的质量和效率,提出一种基于最优家族遗传算法(OFGA)的软件测试用例自动生成新算法。基于OFGA的测试用例生成算法在执行过程中适当缩小搜索区域,从而在相对更小的区域内快速寻找最优解。因此,OFGA能比较快地加速算法的收敛,提高算法的效率,在测试用例的生成上具有较大的应用潜力。由实验结果可知,新算法比遗传算法(GA)在测试用例自动生成上耗时更少,效果更佳。     

基于动态集合进化算法的弱变异测试用例集生成

为解决基于集合进化算法（SEA）的弱变异测试用例集生成过程中个体规模固定和执行开销大的问题,提出一种基于动态集合进化算法（DSEA）的弱变异测试用例集生成方法。以测试用例集为个体,生成覆盖所有变异分支的弱变异测试用例集。在进化过程中,集合精简算子根据最优个体的最小子集及其未覆盖变异分支数量计算所需测试用例集的最小规模,并基于该最小规模调整种群中所有个体的规模,以生成最小规模的弱变异测试用例集,同时设计了适用于评估以测试用例集为个体的适应度函数。实验结果表明,动态集合进化算法指导弱变异测试用例集生成,获得的测试用例集规模比个体初始规模平均约简了50.15%,执行时间比集合进化的弱变异测试用例集生成最多降低了74.58%。因此,动态集合进化算法为最小规模的弱变异测试用例集生成和提升算法速度提供了一种解决方案。     

SVM应用于测试用例生成的方法

针对在小样本情况下BP神经网络在生成软件测试用例的过程中可能产生的过学习问题及识别正确率较差的缺点,运用支持向量机具有更好的泛化性能的原理,提出了应用支持向量机生成测试用例的方法。对五个软件测试实例针对多组不同数量的训练样本所做的实验表明,在小样本情况下与BP神经网络相比,应用支持向量机得到的测试用例预期结果的正确率提高了10个百分点以上,说明了该方法的有效性。     

DLJ: A dynamic line-justification algorithm for test generation

Lne justification is a basic factor in affecting the efficiency of algorithms for test generation.The existence of reconvergent fanouts in the circuit under test results in backtracks in the process of line justification.In order to reduce the number of backtracks and shorten the processing time between backtracks,we present a new algorithm called DLJ(dynamic line justification)in whic two techniques are employed.1.A cost function called “FOCOST” is proposed as heuristic information to represent the cost of justifying a certain line.Whn the relations among the lines being justified are“and”,the line having the highest FOCOST should be chosen.When the relations are“or”,the line having the lowest FOCOST should be chosen.The computing of the FOCOST of lines is very simple.2.Disjoint justification cubes dynamically generated to perform backtracks make the backtrack number of the algorithm minimal.When the backtrace with cube C1 does not yield a solution,the next cube to be chosen is C‘2=C2-{C1,C2}.Experimental results demonstrate that the combination of the two techniques effectively reduces the backtracks and accelerates the test generation.