Automatic string test data generation for detecting domain errors

Domain testing is designed to detect domain errors that result from a small boundary shift in a path domain. Although many researchers have studied domain testing, automatic domain test data generation for string predicates has seldom been explored. This paper presents a novel approach for the automatic generation of ON–OFF test points for string predicate borders, and describes a corresponding test data generator. Our empirical work is conducted on a set of programs with string predicates, where extensive trials have been done for each string predicate, and the results are analysed using the SPSS tool. Conclusions are drawn that: (i) the approach is promising and effective; (ii) there is a strong linear relationship between the performance of the test generator and the length of target string in the predicate tested; and (iii) initial inputs, no shorter than the target string and with characters generated randomly, may enhance the performance in the test data generation for string predicates. Copyright © 2009 John Wiley & Sons, Ltd.     

Generating consistent test data: Restricting the search space by a generator formula

To address the problem of generating test data for a set of general consistency constraints, we propose a new two-step approach: First the interdepen-dencies between consistency constraints are explored and a generator formula is derived on their basis. During its creation, the user may exert control. In essence, the generator formula contains information to restrict the search for consistent test databases. In the second step, the test database is generated. Here, two different approaches are proposed. The first adapts an already published approach to generating finite models by enhancing it with requirements imposed by test data generation. The second, a new approach, operationalizes the generator formula by translating it into a sequence of operators, and then executes it to construct the test database. For this purpose, we introduce two powerful operators: the generation operator and the test-and-repair operator. This approach also allows for enhancing the generation operators with heuristics for generating facts in a goal-directed fashion. It avoids the generation of test data that may contradict the consistency constraints, and limits the search space for the test data. This article concludes with a careful evaluation and comparison of the performance of the two approaches and their variants by describing a number of benchmarks and their results.     

GA-based multiple paths test data generator

Developers have learned over time that software testing costs a considerable amount of a software project budget. Hence, software quality managers have been looking for solutions to reduce testing costs and time. Considering path coverage as the test adequacy criterion, we propose using genetic algorithms (GA) for automating the generation of test data for white-box testing. There are evidences that GA has been already successful in generating test data. However, existing GA-based test data generators suffer from some problems. This paper presents our approach to overcome one of these problems; that is the inefficiency in covering multiple target paths. We have designed a GA-based test data generator that is, in one run, able to synthesize multiple test data to cover multiple target paths. Moreover, we have implemented a set of variations of the generator. Experimental results show that our test data generator is more efficient and more effective than others.     

An integrated automatic test data generation system

The Godzilla automatic test data generator is an integrated collection of tools that implements a relatively new test data generation method—constraint-based testing—that is based on mutation analysis. Constraint-based testing integrates mutation analysis with several other testing techniques, including statement coverage, branch coverage, domain perturbation, and symbolic evaluation. Because Godzilla uses a rule-based approach to generate test data, it is easily extendible to allow new testing techniques to be integrated into the current system. This article describes the system that has been built to implement constraint-based testing. Godzilla's design emphasizes orthogonality and modularity, allowing relatively easy extensions. Godzilla's internal structure and algorithms are described with emphasis on internal structures of the system and the engineering problems that were solved during the implementation.Parts of this research were supported by Contract F30602-85-C-0255 through Rome Air Development Center while the author was a graduate student at the Georgia Institute of Technology.     

Modelling dynamic memory management in constraint-based testing

Constraint-based testing (CBT) is the process of generating test cases against a testing objective by using constraint solving techniques. When programs contain dynamic memory allocation and loops, constraint reasoning becomes challenging as new variables and new constraints should be created during the test data generation process. In this paper, we address this problem by proposing a new constraint model of C programs based on operators that model dynamic memory management. These operators apply powerful deduction rules on abstract states of the memory enhancing the constraint reasoning process. This allows to automatically generate test data respecting complex coverage objectives. We illustrate our approach on a well-known difficult example program that contains dynamic memory allocation/deallocation, structures and loops. We describe our implementation and provide preliminary experimental results on this example that show the highly deductive potential of the approach.     

基于解空间树的组合测试数据生成

在组合覆盖测试模型的基础上提出：将所有的可用测试数据表示为一棵解空间树,利用回溯法对解空间树进行路径搜索来生成测试数据,然后使用贪心算法补充生成测试数据,以满足两两组合覆盖标准.并且实现了基于该方法的测试数据生成工具,所生成的测试数据集与同类工具相比具有一定的特点和优势.     

基于场景模型的DDS架构一体化舰船任务系统测试

以数据分发服务(data-distribution service, DDS)为基础架构的新型一体化舰船任务系统在研发模式、结构和应用方面具有特殊特点, 使得其测试相当具有挑战. 基于模型的测试(model-based testing, MBT)是工业系统测试的一种有效方法. 然而, 对于类舰船任务系统, 由于其自身的...     

模型驱动下的Web应用系统自动生成

为了提高Web应用系统开发效率,提出了模型驱动下的Web应用系统自动生成方法。这种生成方法是以Me—taEdit＋作为元建模工具,首先创建Web应用系统元模型、定制DSL,进而建立Web应用系统领域模型,然后通过MetaEdit＋提供的生成器定义语言MERL,软件开发人员可以很方便地设计出Web应用系统所需的JSP生成器、Servlet生成器、Jayabeans生成器和数据库生成器,从Web应用系统图形模型直接生成整个Web应用系统。最后通过一个WebShopping实例详细介绍了模型驱动下的Web应用系统生成方法及生成过程。经测试,所生成的Web应用系统可以在Windows操作系统中的Web应用服务器上正确运行。     

A threat model‐based approach to security testing

Software security issues have been a major concern in the cyberspace community, so a great deal of research on security testing has been performed, and various security testing techniques have been developed. Threat modeling provides a systematic way to identify threats that might compromise security, and it has been a well‐accepted practice by the industry, but test case generation from threat models has not been addressed yet. Thus, in this paper, we propose a threat model‐based security testing approach that automatically generates security test sequences from threat trees and transforms them into executable tests. The security testing approach we consider consists of three activities in large: building threat models with threat trees; generating security test sequences from threat trees; and creating executable test cases by considering valid and invalid inputs. To support our approach, we implemented security test generation techniques, and we also conducted an empirical study to assess the effectiveness of our approach. The results of our study show that our threat tree‐based approach is effective in exposing vulnerabilities. Copyright © 2012 John Wiley & Sons, Ltd.     

Generation of Positive and Negative Tests for Parsers

A methodology for automatic positive and negative test set generation for testing parsers is described. Coverage criteria for such test sets based on the model approach to testing are proposed. Methods for the generation of test sets satisfying these criteria are discussed. Results of the application of the proposed methodology for testing parsers for various languages including C and Java are presented.     

Model-based testing for real

Model-based testing relies on abstract behavior models for test case generation. These models are abstractions, i.e., simplifications. For deterministic reactive systems, test cases are sequences of input and expected output. To bridge the different levels of abstraction, input must be concretized before being applied to the system under test. The systems output must then be abstracted before being compared to the output of the model.The concepts are discussed along the lines of a feasibility study, an inhouse smart card case study. We describe the modeling concepts of the CASE tool AutoFocus and an approach to model-based test case generation that is based on symbolic execution with Constraint Logic Programming.Different search strategies and algorithms for test case generation are discussed. Besides validating the model itself, generated test cases were used to verify the actual hardware with respect to these traces.     

测试资源受约束的安全关键软件加速测试方法

基于马尔可夫链使用模型提出了一种针对安全关键软件测试资源受约束的启发式加速测试方法.该方法利用一种新的随机优化技术--交叉熵方法,以软件投放后软件失效风险损失最小为目标,基于失效风险损失通过修正操作剖面,自动生成测试数据集.实验结果表明该方法能有效地降低软件失效风险,提高测试效率,是一种快速有效的加速测试方法.     

A framework for intelligent test data generation

Test data generation using traditional software testing methods generally requires considerable manual effort and generates only a limited number of test cases before the amount of time expanded becomes unacceptably large. A rule-based framework that will automatically generate test data to achieve maximal branch coverage is presented. The design and discovery of rules used to generate meaningful test cases are also described. The rule-based approach allows this framework to be extended to include additional testing requirements and test case generation knowledge.This work was supported in part by George C. Marshall Space Flight Center, NASA/MSFC, AL 35812 (NASA-NCC8-14).     

面向路径的测试数据自动生成工具及其图形界面的Tcl/Tk设计

面向路径的测试数据生成问题是软件测试中一个基本问题。文章介绍了自主开发的面向路径的测试数据自动生成工具。Tcl/Tk是一种图形界面设计工具,其功能强大,可运行于Windows,UNIX等操作系统上,具有良好的可移植性。文章简要介绍Tcl/Tk,并给出用Tcl/Tk设计面向路径的测试数据自动生成工具的图形界面的方法。     

基于禁忌搜索算法的可扩展有限状态机模型测试数据自动生成

可扩展有限状态机(EFSM)模型的测试用例生成包括测试路径生成和测试数据生成两部分。然而,目前针对EFSM模型的测试研究大多集中于测试路径生成。为探索路径上测试数据的自动生成,提出一种面向EFSM路径的测试数据生成方法,利用禁忌搜索(TS)策略实现了EFSM测试数据的自动生成;分析了影响EFSM测试数据生成的关键因素,并与遗传算法(GA)的生成效率进行了比较。实验结果表明：基于TS算法的EFSM模型测试数据自动生成是确实可行的,其测试生成效率相对于GA有很大提高。     

Model-based test generation using extended symbolic grammars

A novel, model-based test case generation approach for validating reactive systems, especially those supporting richly structured data inputs and/or interactions, is presented. Given an executable system model and an extended symbolic grammar specifying plausible system inputs, the approach performs a model-based simulation to (i) ensure the consistency of the model with respect to the specified inputs, and (ii) generate corresponding test cases for validating the system. The model-based simulation produces a state transition diagram (STD) automatically justifying the model runtime behaviors within the test case coverage. The STD can further be transformed to produce an evolved symbolic grammar, which can then be used to incrementally generate a refined set of test cases. As a case study, we present a live sequence chart (LSC) model-based test generator, named LCT in short, for LSC simulation and consistency testing. The evolved symbolic grammar produced by the simulator can either be used to generate practical test cases for software testing, or be further refined by applying our model-based test generation approach again with additional test coverage criteria. We further show that LSCs can also be used to specify and test certain temporal system properties during the model simulation. Their satisfaction, reflected in the STD, can either be served as a directive for selective test generation, or a basis for further temporal property model checking.