Automated software test data generation

An alternative approach to test-data generation based on actual execution of the program under test, function-minimization methods and dynamic data-flow analysis is presented. Test data are developed for the program using actual values of input variables. When the program is executed, the program execution flow is monitored. If during program execution an undesirable execution flow is observed then function-minimization search algorithms are used to automatically locate the values of input variables for which the selected path is traversed. In addition, dynamic data-flow analysis is used to determine those input variables responsible for the undesirable program behavior, significantly increasing the speed of the search process. The approach to generating test data is then extended to programs with dynamic data structures and a search method based on dynamic data-flow analysis and backtracking is presented. In the approach described, values of array indexes and pointers are known at each step of program execution; this information is used to overcome difficulties of array and pointer handling     

A rule-based software test data generator

Rule-based software test data generation is proposed as an alternative to either path/predicate analysis or random data generation. A prototype rule-based test data generator for Ada programs is constructed and compared to a random test data generator. Four Ada procedures are used in the comparison. Approximately 2000 rule-based test cases and 100000 randomly generated test cases are automatically generated and executed. The success of the two methods is compared using standard coverage metrics. Simple statistical tests showing that even the primitive rule-based test data generation prototype is significantly better than random data generation are performed. This result demonstrates that rule-based test data generation is feasible and shows great promise in assisting test engineers, especially when the rule base is developed further     

Generating software test data by evolution

This paper discusses the use of genetic algorithms (GAs) for automatic software test data generation. This research extends previous work on dynamic test data generation where the problem of test data generation is reduced to one of minimizing a function. In our work, the function is minimized by using one of two genetic algorithms in place of the local minimization techniques used in earlier research. We describe the implementation of our GA-based system and examine the effectiveness of this approach on a number of programs, one of which is significantly larger than those for which results have previously been reported in the literature. We also examine the effect of program complexity on the test data generation problem by executing our system on a number of synthetic programs that have varying complexities     

Matrix-based software test data decompression for systems-on-a-chip

This paper describes a new compression/decompression methodology for using an embedded processor to test the other components of a system-on-a-chip (SoC). The deterministic test vectors for each core are compressed using matrix-based operations that significantly reduce the amount of test data that needs to be stored on the tester. The compressed data is transferred from the tester to the processor's on-chip memory. The processor executes a program which decompresses the data and applies it to the scan chains of each core-under-test. The matrix-based operations that are used to decompress the test vectors can be performed very efficiently by the embedded processor thereby allowing the decompression program to be very fast and provide high throughput of the test data to minimize test time. Experimental results demonstrate that the proposed approach provides greater compression than previous methods.     

Studying the fault-detection effectiveness of GUI test cases for rapidly evolving software

Software is increasingly being developed/maintained by multiple, often geographically distributed developers working concurrently. Consequently, rapid-feedback-based quality assurance mechanisms such as daily builds and smoke regression tests, which help to detect and eliminate defects early during software development and maintenance, have become important. This paper addresses a major weakness of current smoke regression testing techniques, i.e., their inability to automatically (re)test graphical user interfaces (GUIs). Several contributions are made to the area of GUI smoke testing. First, the requirements for GUI smoke testing are identified and a GUI smoke test is formally defined as a specialized sequence of events. Second, a GUI smoke regression testing process called daily automated regression tester (DART) that automates GUI smoke testing is presented. Third, the interplay between several characteristics of GUI smoke test suites including their size, fault detection ability, and test oracles is empirically studied. The results show that: 1) the entire smoke testing process is feasible in terms of execution time, storage space, and manual effort, 2) smoke tests cannot cover certain parts of the application code, 3) having comprehensive test oracles may make up for not having long smoke test cases, and 4) using certain oracles can make up for not having large smoke test suites.     

Dynamic method for software test data generation

Test data generation in program testing is the process of identifying a set of test data which satisfies a given testing criterion. Existing pathwise test data generators proceed by selecting program paths that satisfy the selected criterion and then generating program inputs for these paths. One of the problems with this approach is that unfeasible paths are often selected; as a result, significant computational effort can be wasted in analysing those paths. In this paper, an approach to test data generation, referred to as a dynamic approach for test data generation, is presented. In this approach, the path selection stage is eliminated. Test data are derived based on the actual execution of the program under test and function minimization methods. The approach starts by executing a program for an arbitrary program input. During program execution for each executed branch, a search procedure decides whether the execution should continue through the current branch or an alternative branch should be taken. If an undesirable execution flow is observed at the current branch, then a real-valued function is associated with this branch, and function minimization search algorithms are used to locate values of input variables automatically, which will change the flow of execution at this branch.     

An evaluation of methods for prioritizing software requirements

This article describes an evaluation of six different methods for prioritizing software requirements. Based on the quality requirements for a telephony system, the authors individually used all six methods on separate occasions to prioritize the requirements. The methods were then characterized according to a number of criteria from a user's perspective. We found the analytic hierarchy process to be the most promising method, although it may be problematic to scale-up. In an industrial follow-up study we used the analytic hierarchy process to further investigate its applicability. We found that the process is demanding but worth the effort because of its ability to provide reliable results, promote knowledge transfer and create consensus among project members.     

Analysis and refinement of software test data adequacy properties

Test data adequacy criteria are standards that can be applied to decide if enough testing has been performed. Previous research in software testing has suggested 11 fundamental properties which reasonable criteria should satisfy if the criteria make use of the structure of the program being tested. It is shown that there are several dependencies among the 11 properties making them questionable as a set of fundamental properties, and that the statements of the properties can be generalized so that they can be appropriately analyzed with respect to criteria that do not necessarily make use of the program's structure. An analysis that shows the relationships among the properties with respect to different classes of criteria which utilize the program structure and the specification in different ways is discussed. It is shown how the properties differ under the two models in order to maintain consistency that the dependencies are largely a result of five very weak existential properties, and that by modifying three of the properties, these weaknesses can be eliminated. The result is a reduced set of seven properties, each of which is strong from a mathematical perspective     

Search‐based software test data generation: a survey

The use of metaheuristic search techniques for the automatic generation of test data has been a burgeoning interest for many researchers in recent years. Previous attempts to automate the test generation process have been limited, having been constrained by the size and complexity of software, and the basic fact that, in general, test data generation is an undecidable problem. Metaheuristic search techniques offer much promise in regard to these problems. Metaheuristic search techniques are high‐level frameworks, which utilize heuristics to seek solutions for combinatorial problems at a reasonable computational cost. To date, metaheuristic search techniques have been applied to automate test data generation for structural and functional testing; the testing of grey‐box properties, for example safety constraints; and also non‐functional properties, such as worst‐case execution time. This paper surveys some of the work undertaken in this field, discussing possible new future directions of research for each of its different individual areas. Copyright © 2004 John Wiley & Sons, Ltd.     

基于DWSPSO的软件测试数据自动生成

针对软件结构测试数据的自动生成提出了一种动态改变惯性权重的简化粒子群算法(DWSPSO)。该算法舍弃了粒子速度这个参数,并通过粒子群中所有粒子适应度的整体变化跟踪粒子群的状态。在每次迭代时,算法可根据粒子的适应度变化动态改变惯性权重,从而使算法具有动态自适应性全局搜索与局部搜索能力。实验结果表明,该算法在测试数据的自动生成上,优于基本的粒子群算法以及惯性权值线性递减粒子群算法(LDWPSO)。     

面向安全苛求软件的测试用例自动生成*

测试用例的自动生成是验证安全苛求软件最关键的技术问题,然而目前的研究并没有充分考虑安全苛求软件的安全性需求,为此提出一种应用安全覆盖准则的安全苛求软件的测试用例自动生成策略,将该策略应用于铁路车站计算机连锁软件,并与全节点覆盖准则进行了比较。结果表明该策略对关键变迁有更高的安全性保证。     

Framework for cloud-based software test data generation service

This paper presents the framework of cloud-based software test data generation service (CSTS) that caters to cost-effective test data generation service in a cloud environment. In contrast to existing conventional or cloud-based testing frameworks, CSTS has a number of unique benefits. First, CSTS is designed to facilitate test data generation in minimum time and cost. Second, unlike existing frameworks which mandates clients to opt for resources to test their jobs, CSTS guides customer for selecting best cluster configuration in order to minimize the cost. While the existing models do not provide any solution for trust establishment in cloud computing services, CSTS delivers it by implementing security mechanism with the provision of role based access control. The security mechanism proposed in this paper ensures the protection of data and code of different users. Third, CSTS provides a mathematical pricing model to fulfill the expectations of customers and also to maximize the net profit of service providers. Cloud service request model has also been designed that postulates service level agreements between customers and service providers. We have evaluated, compared, and analyzed our framework and have found that it outperforms other existing cloud-based frameworks.     

ADTEST: a test data generation suite for Ada software systems

Presents the design of the software system ADTEST (ADa TESTing), for generating test data for programs developed in Ada83. The key feature of this system is that the problem of test data generation is treated entirely as a numerical optimization problem and, as a consequence, this method does not suffer from the difficulties commonly found in symbolic execution systems, such as those associated with input variable-dependent loops, array references and module calls. Instead, program instrumentation is used to solve a set of path constraints without explicitly knowing their form. The system supports not only the generation of integer and real data types, but also non-numerical discrete types such as characters and enumerated types. The system has been tested on large Ada programs (60,000 lines of code) and found to reduce the effort required to test programs as well as providing an increase in test coverage     

DEMON: mining and monitoring evolving data

Data mining algorithms have been the focus of much research. In practice, the input data to a data mining process resides in a large data warehouse whose data is kept up-to-date through periodic or occasional addition and deletion of blocks of data. Most data mining algorithms have either assumed that the input data is static, or have been designed for arbitrary insertions and deletions of data records. We consider a dynamic environment that evolves through systematic addition or deletion of blocks of data. We introduce a new dimension, called the data span dimension, which allows user-defined selections of a temporal subset of the database. Taking this new degree of freedom into account, we describe efficient model maintenance algorithms for frequent item sets and clusters. We then describe a generic algorithm that takes any traditional incremental model maintenance algorithm and transforms it into an algorithm that allows restrictions on the data span dimension. We also develop an algorithm for automatically discovering a specific class of interesting block selection sequences. In a detailed experimental study, we examine the validity and performance of our ideas on synthetic and real datasets     

恒定应力加速寿命试验及其数据统计分析软件

加速寿命试验是为了缩短试验时间,在不改变产品失效机理的条件下,用加大应力（如温度应力、电压应力等）的方法对产品进行的寿命试验。通过加大应力水平,加快产品失效,获得失效数据,运用加速寿命模型外推产品在正常应力水平下的可靠性特性。本文主要介绍恒定应力加速寿命试验的组织实施方法,以及其试验数据统计分析软件。     

Collaborative technologies for evolving software systems

The article discusses four major collaboration theme areas: collaborative information engineering; collaborative process and workflow; collaboration support (awareness); and integration of collaborative systems. Various projects incorporating these technologies are outlined. All of the projects described are developing component technologies intended for mixing and matching with each other and with technologies under investigation by other EDCS projects not particularly concerned with collaborative work, as well as with COTS software. Only GroupSpaces intends to assemble a full-blown, directly usable software development environment framework, and that comprises components, developed under EDCS and available off-the-shelf     

Mutation-oriented test data augmentation for GUI software fault localization

ContextFault localization lies at the heart of program debugging and often proceeds by contrasting the statistics of program constructs executed by passing and failing test cases. A vital issue here is how to obtain these “suitable” test cases. Techniques presented in the literature mostly assume the existence of a large test suite a priori. However, developers often encounter situations where a failure occurs, but where no or no appropriate test suite is available for use to localize the fault.ObjectiveThis paper aims to alleviate this key limitation of traditional fault localization techniques for GUI software particularly, namely, it aims at enabling cost-effective fault localization process for GUI software in the described scenario.MethodTo address this scenario, we propose a mutation-oriented test data augmentation technique, which actually is directed by the “similarity” criterion in GUI software’s test case context towards the generation of test suite with excellent fault localization capabilities. More specifically, the technique mainly uses four proposed novel mutation operators to iteratively mutate some failing GUI test cases’ event sequences to derive new test cases potentially useful to localize the specific encountered fault. We then compare the fault localization performance of the test suite generated using this technique with that of an original provided large event-pair adequate test suite on some GUI applications.ResultsThe results indicate that the proposed technique is capable of generating a test suite that has comparable, if not better, fault localization effectiveness to the event-pair adequate test suite, but it is much smaller and it is generated immediately once a failure is encountered by developers.ConclusionIt is concluded that the proposed technique can truly enable quick-start cost-effective fault localization process under the investigated all-too-common scenario, greatly alleviating one key limitation of traditional fault localization techniques and prompting the test–diagnose–repair cycle.     

基于多态蚁群算法的测试用例自动生成*

提出了一种基于多态蚁群算法的测试数据自动生成方法。该方法使用二进制编码将输入数据转换为位串;然后在蚁群算法的基础上将蚁群分为三类,据其信息素的不同采用不同的移动准则,重点对侦察蚁和搜索蚁进行功能分析。将局部搜索与全局搜索结合起来,结合路径的相似度,缩小搜索空间;根据适应度函数确定最好路径,既解决局部最优化问题,又提高收敛效率。与基本蚁群算法对比,其结果显示该方法效率优于基本蚁群算法。     

Automatic, evolutionary test data generation for dynamic software testing

This paper proposes a dynamic test data generation framework based on genetic algorithms. The framework houses a Program Analyser and a Test Case Generator, which intercommunicate to automatically generate test cases. The Program Analyser extracts statements and variables, isolates code paths and creates control flow graphs. The Test Case Generator utilises two optimisation algorithms, the Batch-Optimistic (BO) and the Close-Up (CU), and produces a near to optimum set of test cases with respect to the edge/condition coverage criterion. The efficacy of the proposed approach is assessed on a number of programs and the empirical results indicate that its performance is significantly better compared to existing dynamic test data generation methods.