面向云测试的并行测试用例自动生成方法

为实现云计算环境下的高效软件测试,提出一种Web 应用系统功能测试的并行测试用例自动生成方法。该方法首先根据场景流图采用深度优先遍历算法生成并行测试路径,录制组合产生并行测试脚本,并进行参数化处理;然后,使用基于搜索的软件测试(SBST)方法自动生成可经过目标路径的有效测试数据集,脚本与数据耦合形成大量可并行部署的自动化测试用例。为验证方法的有效性设计了自动化云测试原型系统。实验结果表明,该并行测试用例自动生成方法可以高效地为云平台提供测试用例输入,提高测试效率。     

活动图模型驱动的Web应用程序测试方法

提出了一种活动图(AD)模型驱动的Web应用程序测试方法,从活动图中生成满足往返路径覆盖准则的测试序列,基于测试输入语法生成驱动测试序列执行的测试数据,将测试数据整合到测试序列中生成测试用例,运行测试用例进行测试并生成测试结果的报告。实现了一个原型工具以支持活动图模型驱动的Web应用程序测试方法,设计并完成了一个实验,验证了该方法的可行性与有效性。     

用于统计测试概率分布生成的自动搜索方法

提出一种基于自动搜索的概率分布生成方法,设计对概率分布的表示形式与评估函数,同时结合模拟退火算法设计基于马尔可夫模型的自动搜索过程。实验结果表明,该方法能够有效地提高自动搜索的准确性,在一定时间内成功找到接近最优的概率分布,生成高效的测试数据,同时达到降低统计测试成本的目的。     

基于活动图模型测试剖面的测试用例生成方法

给出了测试用例的定义和如何根据UML活动图模型来生成测试用例的基本方法，包括基于活动图模型控制流结构的测试场景生成和针对活动的输入量的测试数据生成。根据活动图模型的层次型特点，引入了针对话动的层次化的测试剖面(Test Profiles)概念和输入输出数据描述规范，用以支持用户在活动图上分层次地提供有关测试数据生成的约束条件。同时，给出了基于测试剖面的基本测试数据的生成方法，以及基于测试场景和基本测试数据的组合来生成一组测试用例的方法。     

基于搜索的面向路径字符串测试数据自动生成方法

针对字符串测试数据自动生成,讨论了字符串间的距离,将不满足路径条件的字符串谓词表示成一个实值目标函数;利用快速下降搜索算法实施目标函数极小化,实现了基于搜索的面向路径字符串测试数据自动生成方法;探讨了其测试数据生成效率与初始输入、路径处理顺序之间的关系,并与遗传算法等几种算法进行了比较．实验结果表明：该方法是一种更经济有效的测试数据生成方法．     

UML模型驱动的划分测试用例生成方法研究

测试用例的自动生成是软件测试研究的主要方向之一。针对现有的UML模型驱动测试方法在测试数据生成方面存在低效、无目的性、冗余等问题,提出了基于UML活动图模型驱动的划分测试用例自动生成方法。该方法将测试场景归为五种类型,并为每种类型规划了测试数据取值范围和选择方法。构建了基于该方法的原型工具软件,提供被测软件原模型和数据约束即可生成测试用例。实验结果表明,该方法能够在不降低测试覆盖率的情况下,能生成数量少、针对性强的测试用例集。     

QPSO在结构测试数据生成中的应用研究

针对目前进化算法生成结构测试数据方法存在搜索速度慢、设置参数复杂、易陷入局部最优解等缺陷,提出了一种基于量子粒子群算法的结构测试数据生成方法。该方法采用分支函数叠加法构造适应值函数,将测试数据自动生成问题转化为函数的最优化问题,同时在粒子群算法基础上引入量子理论的思想,提高了算法的收敛性能和全局搜索能力。将其与标准粒子群算法实现结构测试数据自动生成方法进行比较,实验结果表明,该方法能更快生成测试数据。     

一种约束求解的Web应用测试数据生成与筛选方法

传统基于随机算法、约束求解以及面向白盒测试的测试数据生成方法往往忽略了测试数据与数据库系统之间的依赖关系,同时并不适用于数据复用、Web应用测试。提出一种面向业务流程数据约束的测试数据生成与筛选方法,通过在Web业务流程模型中引入对测试数据的半形式化约束,利用数据池存储技术,并提高测试数据的复用性。通过案例表明,新的测试数据生成技术能够以较低的开销生成可用性高的测试数据集。     

基于粒子群算法的改进SCOTEM模型测试数据生成方法

UML已经成为建模语言的事实标准,如何从UML模型生成测试用例为面向对象软件测试带来了新的挑战.为测试用例提供测试数据是其中的关键环节.ShaukatAli等人在UML基础上提出SCOTEM模型,但是该模型的测试数据需要人工生成.针对SCOTEM模型进行了改进使其适于灰盒测试,以OCL约束测试数据的生成和运行结果的验证,并提出了一种为基于粒子群算法的改进SCOTEM模型自动生成测试数据的方法.针对不同的覆盖标准,试验结果表明,该方法能够以更高的效率生成高质量的测试数据.     

GUI测试中多级形态模型的分割与应用

基于模型的GUI测试方法(MBGT)可自动化生成测试用例,在MBGT中引入多级形态模型(Multilevel Morphology Model,MMM)可以实现从不同的形态角度考察系统,可控地提高模型的错误检测效力。但多级形态模型只能整体扩展到高阶,且随着模型的扩展,测试用例的长度与数量急剧增长,极大地影响了测试效率。对此提出一种可进行局部扩展的多级形态模型的分割方法以及相应的测试用例生成策略。该方法通过GUI事件的分类,实现了对基础模型的分割与化简,并采用广度优先搜索(BFS)与中国邮递员问题(CPP)求解算法生成测试用例。该方法使得GUI模型的表达更为清晰直观,在有效区分测试重点的同时,缩小了测试集规模,极大地提高了多级形态模型在GUI测试中的灵活性与测试效率。实验证明,经模型分割后的多级形态模型具有与未分割模型基本等同的错误检测效力,且随着模型级数的提高,模型分割对测试效率的提升增大。     

A search-based framework for automatic testing of MATLAB/Simulink models

Search-based test-data generation has proved successful for code-level testing but almost no search-based work has been carried out at higher levels of abstraction. In this paper the application of such approaches at the higher levels of abstraction offered by MATLAB/Simulink models is investigated and a wide-ranging framework for test-data generation and management is presented. Model-level analogues of code-level structural coverage criteria are presented and search-based approaches to achieving them are described. The paper also describes the first search-based approach to the generation of mutant-killing test data, addressing a fundamental limitation of mutation testing. Some problems remain whatever the level of abstraction considered. In particular, complexity introduced by the presence of persistent state when generating test sequences is as much a challenge at the Simulink model level as it has been found to be at the code level. The framework addresses this problem. Finally, a flexible approach to test sub-set extraction is presented, allowing testing resources to be deployed effectively and efficiently.     

Test-Data Generation Guided by Static Defect Detection

Software testing is an important technique to assure the quality of software systems, especially high-confidence systems. To automate the process of software testing, many automatic test-data generation techniques have been proposed. To generate e?ective test data, we propose a test-data generation technique guided by static defect detection in this paper. Using static defect detection analysis, our approach first identifies a set of suspicious statements which are likely to contain faults, then generates t...     

Modelling dynamic memory management in constraint-based testing

Constraint-based testing (CBT) is the process of generating test cases against a testing objective by using constraint solving techniques. When programs contain dynamic memory allocation and loops, constraint reasoning becomes challenging as new variables and new constraints should be created during the test data generation process. In this paper, we address this problem by proposing a new constraint model of C programs based on operators that model dynamic memory management. These operators apply powerful deduction rules on abstract states of the memory enhancing the constraint reasoning process. This allows to automatically generate test data respecting complex coverage objectives. We illustrate our approach on a well-known difficult example program that contains dynamic memory allocation/deallocation, structures and loops. We describe our implementation and provide preliminary experimental results on this example that show the highly deductive potential of the approach.     

基于布尔表达式约束的测试用例生成技术

布尔表达式约束在软件规格说明和程序中广泛存在，这些约束可作为软件系统的模型，成为测试用例生成依据。本文调研分析基于布尔表达式约束的测试用例生成方法，主要分为基于约束语法的测试和基于约束语义的测试。归纳总结基于约束语法测试的各种故障类型和测试策略，并比较各种测试策略的适用情形和故障检测能力，也对基于约束语义测试的各种约束获取和求解方法进行性能分析，并介绍了典型工具。最后对未来的研究发展进行展望。     

Search-based automated testing of continuous controllers: Framework,tool support,and case studies

ContextTesting and verification of automotive embedded software is a major challenge. Software production in automotive domain comprises three stages: Developing automotive functions as Simulink models, generating code from the models, and deploying the resulting code on hardware devices. Automotive software artifacts are subject to three rounds of testing corresponding to the three production stages: Model-in-the-Loop (MiL), Software-in-the-Loop (SiL) and Hardware-in-the-Loop (HiL) testing.ObjectiveWe study testing of continuous controllers at the Model-in-Loop (MiL) level where both the controller and the environment are represented by models and connected in a closed loop system. These controllers make up a large part of automotive functions, and monitor and control the operating conditions of physical devices.MethodWe identify a set of requirements characterizing the behavior of continuous controllers, and develop a search-based technique based on random search, adaptive random search, hill climbing and simulated annealing algorithms to automatically identify worst-case test scenarios which are utilized to generate test cases for these requirements.ResultsWe evaluated our approach by applying it to an industrial automotive controller (with 443 Simulink blocks) and to a publicly available controller (with 21 Simulink blocks). Our experience shows that automatically generated test cases lead to MiL level simulations indicating potential violations of the system requirements. Further, not only does our approach generate significantly better test cases faster than random test case generation, but it also achieves better results than test scenarios devised by domain experts. Finally, our generated test cases uncover discrepancies between environment models and the real world when they are applied at the Hardware-in-the-Loop (HiL) level.ConclusionWe propose an automated approach to MiL testing of continuous controllers using search. The approach is implemented in a tool and has been successfully applied to a real case study from the automotive domain.     

Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach

Researchers have explored the application of combinatorial interaction testing (CIT) methods to construct samples to drive systematic testing of software system configurations. Applying CIT to highly-configurable software systems is complicated by the fact that, in many such systems, there are constraints between specific configuration parameters that render certain combinations invalid. Many CIT algorithms lack a mechanism to avoid these. In recent work, automated constraint solving methods have been combined with search-based CIT construction methods to address the constraint problem with promising results. However, these techniques can incur a non-trivial overhead. In this paper, we build upon our previous work to develop a family of greedy CIT sample generation algorithms that exploit calculations made by modern boolean satisfiability (SAT) solvers to prune the search space of the CIT problem. We perform a comparative evaluation of the cost-effectiveness of these algorithms on four real-world highly-configurable software systems and on a population of synthetic examples that share the characteristics of those systems. In combination our techniques reduce the cost of CIT in the presence of constraints to 30\% of the cost of widely-used unconstrained CIT methods without sacrificing the quality of the solutions.     

An integrated search-based approach for automatic testing from extended finite state machine (EFSM) models

Context

The extended finite state machine (EFSM) is a modelling approach that has been used to represent a wide range of systems. When testing from an EFSM, it is normal to use a test criterion such as transition coverage. Such test criteria are often expressed in terms of transition paths (TPs) through an EFSM. Despite the popularity of EFSMs, testing from an EFSM is difficult for two main reasons: path feasibility and path input sequence generation. The path feasibility problem concerns generating paths that are feasible whereas the path input sequence generation problem is to find an input sequence that can traverse a feasible path.

Objective

While search-based approaches have been used in test automation, there has been relatively little work that uses them when testing from an EFSM. In this paper, we propose an integrated search-based approach to automate testing from an EFSM.

Method

The approach has two phases, the aim of the first phase being to produce a feasible TP (FTP) while the second phase searches for an input sequence to trigger this TP. The first phase uses a Genetic Algorithm whose fitness function is a TP feasibility metric based on dataflow dependence. The second phase uses a Genetic Algorithm whose fitness function is based on a combination of a branch distance function and approach level.

Results

Experimental results using five EFSMs found the first phase to be effective in generating FTPs with a success rate of approximately 96.6%. Furthermore, the proposed input sequence generator could trigger all the generated feasible TPs (success rate = 100%).

Conclusion

The results derived from the experiment demonstrate that the proposed approach is effective in automating testing from an EFSM.     

An initial industrial evaluation of interactive search-based testing for embedded software

Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. From an industrial perspective, this could enable an increase in product quality without a matching increase in the time and effort required to do so.Search-based software testing, however, is a set of quite complex techniques and approaches that do not immediately translate into a process for use with most companies.For example, even if engineers receive the proper education and training in these new approaches, it can be hard to develop a general fitness function that covers all contingencies. Furthermore, in industrial practice, the knowledge and experience of domain specialists are often key for effective testing and thus for the overall quality of the final software system. But it is not clear how such domain expertise can be utilized in a search-based system.This paper presents an interactive search-based software testing (ISBST) system designed to operate in an industrial setting and with the explicit aim of requiring only limited expertise in software testing. It uses SBST to search for test cases for an industrial software module, while also allowing domain specialists to use their experience and intuition to interactively guide the search.In addition to presenting the system, this paper reports on an evaluation of the system in a company developing a framework for embedded software controllers. A sequence of workshops provided regular feedback and validation for the design and improvement of the ISBST system. Once developed, the ISBST system was evaluated by four electrical and system engineers from the company (the ‘domain specialists’ in this context) used the system to develop test cases for a commonly used controller module. As well as evaluating the utility of the ISBST system, the study generated interaction data that were used in subsequent laboratory experimentation to validate the underlying search-based algorithm in the presence of realistic, but repeatable, interactions.The results validate the importance that automated software testing tools in general, and search-based tools, in particular, can leverage input from domain specialists while generating tests. Furthermore, the evaluation highlighted benefits of using such an approach to explore areas that the current testing practices do not cover or cover insufficiently.     

Code-coverage guided prioritized test generation

Most automatic test generation research focuses on generation of test data from pre-selected program paths or input domains or program specifications. This paper presents a methodology for a full solution to code-coverage-based test case generation, which includes code coverage-based path selection, test data generation and actual test case representation in program’s original languages. We implemented this method in an automatic testing framework, eXVantage. Experimental results and industrial trials show that the framework is able to generate tests to achieve program line coverage from 20% to 98% with reduced overall testing effort. Our major contributions include an innovative coverage-based program prioritization algorithm, a novel path selection algorithm that takes into consideration program priority and functional calling relationship, and a constraint solver for test data generation that derives constraints from bytecode and solves complex constraints involving strings and dynamic objects.     

Dynamic stopping criteria for search-based test data generation for path testing

ContextEvolutionary algorithms have proved to be successful for generating test data for path coverage testing. However in this approach, the set of target paths to be covered may include some that are infeasible. It is impossible to find test data to cover those paths. Rather than searching indefinitely, or until a fixed limit of generations is reached, it would be desirable to stop searching as soon it seems likely that feasible paths have been covered and all remaining un-covered target paths are infeasible.ObjectiveThe objective is to develop criteria to halt the evolutionary test data generation process as soon as it seems not worth continuing, without compromising testing confidence level.MethodDrawing on software reliability growth models as an analogy, this paper proposes and evaluates a method for determining when it is no longer worthwhile to continue searching for test data to cover un-covered target paths. We outline the method, its key parameters, and how it can be used as the basis for different decision rules for early termination of a search. Twenty-one test programs from the SBSE path testing literature are used to evaluate the method.ResultsCompared to searching for a standard number of generations, an average of 30–75% of total computation was avoided in test programs with infeasible paths, and no feasible paths were missed due to early termination. The extra computation in programs with no infeasible paths was negligible.ConclusionsThe method is effective and efficient. It avoids the need to specify a limit on the number of generations for searching. It can help to overcome problems caused by infeasible paths in search-based test data generation for path testing.