共查询到20条相似文献，搜索用时 15 毫秒
1.
Many modern automated test generators are based on either metaheuristic search techniques or use constraint solvers. Both approaches have their advantages, but they also have specific drawbacks: Search‐based methods may get stuck in local optima and degrade when the search landscape offers no guidance; constraint‐based approaches, on the other hand, can only handle certain domains efficiently. This paper describes a method that integrates both techniques and delivers the best of both worlds. On a high‐level view, the proposed method uses a genetic algorithm to generate tests, but the twist is that during evolution, a constraint solver is used to ensure that mutated offspring efficiently explores different control flow. Experiments on 20 case study programmes show that on average the combination improves branch coverage by 28% over search‐based techniques while reducing the number of tests by 55%, and improves coverage by 13% over constraint‐based techniques while reducing the number of tests by 73%. Copyright © 2013 John Wiley & Sons, Ltd. 相似文献
2.
Identifying a finite test set that adequately captures the essential behaviour of a program such that all faults are identified is a well‐established problem. This is traditionally addressed with syntactic adequacy metrics (e.g. branch coverage), but these can be impractical and may be misleading even if they are satisfied. One intuitive notion of adequacy, which has been discussed in theoretical terms over the past three decades, is the idea of behavioural coverage: If it is possible to infer an accurate model of a system from its test executions, then the test set can be deemed to be adequate. Despite its intuitive basis, it has remained almost entirely in the theoretical domain because inferred models have been expected to be exact (generally an infeasible task) and have not allowed for any pragmatic interim measures of adequacy to guide test set generation. This paper presents a practical approach to incorporate behavioural coverage. Our BESTEST approach (1) enables the use of machine learning algorithms to augment standard syntactic testing approaches and (2) shows how search‐based testing techniques can be applied to generate test sets with respect to this criterion. An empirical study on a selection of Java units demonstrates that test sets with higher behavioural coverage significantly outperform current baseline test criteria in terms of detected faults. © 2015 The Authors. Software Testing, Verification and Reliability published by John Wiley & Sons, Ltd. 相似文献
3.
Alireza Salahirad Hussein Almulla Gregory Gay 《Software Testing, Verification and Reliability》2019,29(4-5)
Search‐based unit test generation, if effective at fault detection, can lower the cost of testing. Such techniques rely on fitness functions to guide the search. Ultimately, such functions represent test goals that approximate—but do not ensure—fault detection. The need to rely on approximations leads to two questions—can fitness functions produce effective tests and, if so, which should be used to generate tests? To answer these questions, we have assessed the fault‐detection capabilities of unit test suites generated to satisfy eight white‐box fitness functions on 597 real faults from the Defects4J database. Our analysis has found that the strongest indicators of effectiveness are a high level of code coverage over the targeted class and high satisfaction of a criterion's obligations. Consequently, the branch coverage fitness function is the most effective. Our findings indicate that fitness functions that thoroughly explore system structure should be used as primary generation objectives—supported by secondary fitness functions that explore orthogonal, supporting scenarios. Our results also provide further evidence that future approaches to test generation should focus on attaining higher coverage of private code and better initialization and manipulation of class dependencies. 相似文献
4.
Jos Miguel Rojas Gordon Fraser Andrea Arcuri 《Software Testing, Verification and Reliability》2016,26(5):366-401
Search‐based techniques have been applied successfully to the task of generating unit tests for object‐oriented software. However, as for any meta‐heuristic search, the efficiency heavily depends on many factors; seeding, which refers to the use of previous related knowledge to help solve the testing problem at hand, is one such factor that may strongly influence this efficiency. This paper investigates different seeding strategies for unit test generation, in particular seeding of numerical and string constants derived statically and dynamically, seeding of type information and seeding of previously generated tests. To understand the effects of these seeding strategies, the results of a large empirical analysis carried out on a large collection of open‐source projects from the SF110 corpus and the Apache Commons repository are reported. These experiments show with strong statistical confidence that, even for a testing tool already able to achieve high coverage, the use of appropriate seeding strategies can further improve performance. © 2016 The Authors. Software Testing, Verification and Reliability Published by John Wiley & Sons Ltd. 相似文献
5.
Phil McMinn 《Software Testing, Verification and Reliability》2004,14(2):105-156
The use of metaheuristic search techniques for the automatic generation of test data has been a burgeoning interest for many researchers in recent years. Previous attempts to automate the test generation process have been limited, having been constrained by the size and complexity of software, and the basic fact that, in general, test data generation is an undecidable problem. Metaheuristic search techniques offer much promise in regard to these problems. Metaheuristic search techniques are high‐level frameworks, which utilize heuristics to seek solutions for combinatorial problems at a reasonable computational cost. To date, metaheuristic search techniques have been applied to automate test data generation for structural and functional testing; the testing of grey‐box properties, for example safety constraints; and also non‐functional properties, such as worst‐case execution time. This paper surveys some of the work undertaken in this field, discussing possible new future directions of research for each of its different individual areas. Copyright © 2004 John Wiley & Sons, Ltd. 相似文献
6.
An important aim in software testing is constructing a test suite with high structural code coverage, that is, ensuring that most if not all of the code under test have been executed by the test cases comprising the test suite. Several search‐based techniques have proved successful at automatically generating tests that achieve high coverage. However, despite the well‐established arguments behind using evolutionary search algorithms (eg, genetic algorithms) in preference to random search, it remains an open question whether the benefits can actually be observed in practice when generating unit test suites for object‐oriented classes. In this paper, we report an empirical study on the effects of using evolutionary algorithms (including a genetic algorithm and chemical reaction optimization) to generate test suites, compared with generating test suites incrementally with random search. We apply the EVO SUITE unit test suite generator to 1000 classes randomly selected from the SF110 corpus of open‐source projects. Surprisingly, the results show that the difference is much smaller than one might expect: While evolutionary search covers more branches of the type where standard fitness functions provide guidance, we observed that, in practice, the vast majority of branches do not provide any guidance to the search. These results suggest that, although evolutionary algorithms are more effective at covering complex branches, a random search may suffice to achieve high coverage of most object‐oriented classes. 相似文献
7.
Andrea Arcuri 《Software Testing, Verification and Reliability》2013,23(2):119-147
The use of search algorithms for test data generation has seen many successful results. For structural criteria like branch coverage, heuristics have been designed to help the search. The most common heuristic is the use of approach level (usually represented with an integer) to reward test cases whose executions get close (in the control flow graph) to the target branch. To solve the constraints of the predicates in the control flow graph, the branch distance is commonly employed. These two measures are linearly combined. Since the approach level is more important, the branch distance is normalized, often in the range [0, 1]. In this paper, different types of normalizing functions are analyzed. The analyses show that the one that is usually employed in the literature has several flaws. The paper presents a different normalizing function that is very simple and does not suffer from these limitations. Empirical and analytical analyses are carried out to compare these two functions. In particular, their effect is studied on commonly used search algorithms, such as Hill Climbing, Simulated Annealing and Genetic Algorithms. Copyright © 2011 John Wiley & Sons, Ltd. 相似文献
8.
Grammar‐based test generation provides a systematic approach to producing test cases from a given context‐free grammar. Unfortunately, naive grammar‐based test generation is problematic because of the fact that exhaustive random test case production is often explosive, and grammar‐based test generation with explicit annotation controls often causes unbalanced testing coverage. In this paper, we present an automatic grammar‐based test generation approach, which takes a symbolic grammar as input, requires zero control input from users, and produces well‐distributed test cases. Our approach utilizes a novel dynamic stochastic model where each variable is associated with a tuple of probability distributions, which are dynamically adjusted along the derivation. We further present a coverage tree illustrating the distribution of generated test cases and their detailed derivations. More importantly, the coverage tree supports various implicit derivation control mechanisms. We implemented this approach in a Java‐based system, named Gena. Each test case generated by Gena automatically comes with a set of structural features, which can play an important and effective role on automated failure causes localization. Experimental results demonstrate the effectiveness of our approach, the well‐balanced distribution of generated test cases over grammatical structures, and a case study on grammar‐based failure causes localization. Copyright © 2014 John Wiley & Sons, Ltd. 相似文献
9.
Automatic test data generation is a very popular domain in the field of search‐based software engineering. Traditionally, the main goal has been to maximize coverage. However, other objectives can be defined, such as the oracle cost, which is the cost of executing the entire test suite and the cost of checking the system behavior. Indeed, in very large software systems, the cost spent to test the system can be an issue, and then it makes sense by considering two conflicting objectives: maximizing the coverage and minimizing the oracle cost. This is what we did in this paper. We mainly compared two approaches to deal with the multi‐objective test data generation problem: a direct multi‐objective approach and a combination of a mono‐objective algorithm together with multi‐objective test case selection optimization. Concretely, in this work, we used four state‐of‐the‐art multi‐objective algorithms and two mono‐objective evolutionary algorithms followed by a multi‐objective test case selection based on Pareto efficiency. The experimental analysis compares these techniques on two different benchmarks. The first one is composed of 800 Java programs created through a program generator. The second benchmark is composed of 13 real programs extracted from the literature. In the direct multi‐objective approach, the results indicate that the oracle cost can be properly optimized; however, the full branch coverage of the system poses a great challenge. Regarding the mono‐objective algorithms, although they need a second phase of test case selection for reducing the oracle cost, they are very effective in maximizing the branch coverage. Copyright © 2011 John Wiley & Sons, Ltd. 相似文献
10.
Roy P. Pargas Mary Jean Harrold Robert R. Peck 《Software Testing, Verification and Reliability》1999,9(4):263-282
This paper presents a technique that uses a genetic algorithm for automatic test‐data generation. A genetic algorithm is a heuristic that mimics the evolution of natural species in searching for the optimal solution to a problem. In the test‐data generation application, the solution sought by the genetic algorithm is test data that causes execution of a given statement, branch, path, or definition–use pair in the program under test. The test‐data‐generation technique was implemented in a tool called TGen, in which parallel processing was used to improve the performance of the search. To experiment with TGen, a random test‐data generator called Random was also implemented. Both Tgen and Random were used to experiment with the generation of test‐data for statement and branch coverage of six programs. Copyright © 1999 John Wiley & Sons, Ltd. 相似文献
11.
Among the tasks in software testing, test data generation is particularly difficult and costly. In recent years, several approaches that use metaheuristic search techniques to automatically obtain the test inputs have been proposed. Although work in this field is very active, little attention has been paid to the selection of an appropriate search space. The present work describes an alternative to this issue. More precisely, two approaches which employ an Estimation of Distribution Algorithm as the metaheuristic technique are explained. In both cases, different regions are considered in the search for the test inputs. Moreover, to depart from a region near to the one containing the optimum, the definition of the initial search space incorporates static information extracted from the source code of the software under test. If this information is not enough to complete the definition, then a grid search method is used. According to the results of the experiments conducted, it is concluded that this is a promising option that can be used to enhance the test data generation process. 相似文献
12.
提出一种使用元启发搜索技术产生测试数据的方法。这种方法能够针对测试中低概率执行部分产生测试数据，提高测试覆盖率，并通过实验举例说明了该方法的可行性。 相似文献
13.
Ricardo Ferreira Vilela João Choma Neto Victor Hugo Santiago Costa Pinto Paulo Sérgio Lopes de Souza Simone do Rocio Senger de Souza 《Concurrency and Computation》2023,35(2):e7489
Concurrent programming is increasingly present in modern applications. Although it provides higher performance and better use of available resources, the mechanisms of interaction between processes/threads result in a greater challenge for software testing activity. The nondeterminism present in those applications is one of the main issues during the test activity since the same test input can produce different possible execution paths, which may or not contain defects. The test data automatic generation can alleviate this problem, ensuring higher speed and reliability in software testing activity. This paper explores the automatic test data generation for concurrent programs through Genetic Algorithm, a bioinspired optimization technique, and proposes a test data generation approach for concurrent programs, called BioConcST, and a new operator for the selection of test subjects, called FuzzyST, which uses fuzzy logic. The approaches were evaluated in an experimental study towards their validation. The results showed that BioConcST is more promising than the other approaches at all analyzed levels. FuzzyST, together with Elitism and Tournament operators, provided the best results; however, it proved more suitable for concurrent programs of higher complexity. 相似文献
14.
Previous research using genetic algorithms to automate the generation of data for path testing has utilized several different fitness functions, assessing their usefulness by comparing them to random generation. This paper describes two sets of experiments that assess the performance of several fitness functions, relative to one another and to random generation. The results demonstrate that some fitness functions provide better results than others, generating fewer test cases to exercise a given program path. In these studies, the branch predicate and inverse path probability approaches were the best performers, suggesting that a two‐step process combining these two methods may be the most efficient and effective approach to path testing. Copyright © 2005 John Wiley & Sons, Ltd. 相似文献
15.
Sebastian Müller;Jan Arne Sparka;Martin Kuban;Claudia Draxl;Lars Grunske; 《Software》2024,54(2):208-224
Computational materials science (CMS) focuses on in silico experiments to compute the properties of known and novel materials, where many software packages are used in the community. The NOMAD Laboratory (Draxl C, Scheffler) offers to store the input and output files in its FAIR data repository. Since the file formats of these software packages are non-standardized, parsers are used to provide the results in a normalized format. 相似文献
16.
Nasir Mehmood Minhas Sohaib Masood Kai Petersen Aamer Nadeem 《Journal of Software: Evolution and Process》2020,32(6)
Model‐based test case generation techniques provide a mechanism to derive tests systematically. This study provides a systematic mapping of test case generation techniques based on UML interaction diagrams. The study compares the test case generation techniques regarding their capabilities and limitations, and it also assesses the reporting quality of the primary studies. We can conclude that the studies presenting test case generation techniques using UML interaction diagrams were not following the guidelines for research methods (eg, case studies or experiments). Solutions were not empirically evaluated in industrial contexts. Our study revealed that better tool support is needed to introduce the UML interaction diagram–based test case generation techniques in the industry. 相似文献
17.
Pouria Derakhshanfar Xavier Devroey Gilles Perrouin Andy Zaidman Arie van Deursen 《Software Testing, Verification and Reliability》2020,30(3)
Search‐based crash reproduction approaches assist developers during debugging by generating a test case, which reproduces a crash given its stack trace. One of the fundamental steps of this approach is creating objects needed to trigger the crash. One way to overcome this limitation is seeding: using information about the application during the search process. With seeding, the existing usages of classes can be used in the search process to produce realistic sequences of method calls, which create the required objects. In this study, we introduce behavioural model seeding: a new seeding method that learns class usages from both the system under test and existing test cases. Learned usages are then synthesized in a behavioural model (state machine). Then, this model serves to guide the evolutionary process. To assess behavioural model seeding, we evaluate it against test seeding (the state‐of‐the‐art technique for seeding realistic objects) and no seeding (without seeding any class usage). For this evaluation, we use a benchmark of 122 hard‐to‐reproduce crashes stemming from six open‐source projects. Our results indicate that behavioural model seeding outperforms both test seeding and no seeding by a minimum of 6% without any notable negative impact on efficiency. 相似文献
18.
Oracles used for testing graphical user interface (GUI) programmes are required to take into consideration complicating factors such as variations in screen resolution or colour scheme when comparing observed GUI elements with expected GUI elements. Researchers proposed fuzzy comparison rules and computationally expensive image processing techniques to tame the comparison process because otherwise the naïve matching comparison would be too constraining and consequently impractical. Alternatively, this paper proposes GUICop, a novel approach with a supporting toolset that takes (1) a GUI programme and (2) user‐defined GUI specifications characterizing the rendering behaviour of the GUI elements and checks whether the execution traces of the programme satisfy the specifications. GUICop comprises the following: (1) a GUI Specification Language; (2) a Driver; (3) Instrumented GUI Libraries; 4) a Solver; and (5) a Code Weaver. The user defines the specifications of the subject GUI programme using the GUI Specification Language. The Driver traverses the GUI structure of the programme and generates events that drive its execution. The Instrumented GUI Libraries capture the GUI execution trace, ie, information about the positions and visibility of the GUI elements. And the Solver, enabled by code injected by the Code Weaver, checks whether the traces satisfy the specifications. GUICop was successfully evaluated using 4 open source GUI applications that included 8 defects, namely, Jajuk, Gason, JEdit, and TerpPaint. 相似文献
19.
Felix Redmill 《Software Testing, Verification and Reliability》2004,14(1):3-15
If software cannot be tested exhaustively, it must be tested selectively. But, on what should selection be based in order to maximize test effectiveness? It seems sensible to concentrate on the parts of the software where the risks are greatest, but what risks should be sought and how can they be identified and analysed? ‘Risk‐based testing’ is a term in current use, for example in an accredited test‐practitioner's course syllabus, but there is no broadly accepted definition of the phrase and no literature or body of knowledge to underpin the subject implied by it. Moreover, there has so far been no suggestion that it requires an understanding of the subject of risk. This paper examines what is implied by risk‐based testing, shows that its practice requires an understanding of risk, and points to the need for research into the topic and the development of a body of knowledge to underpin it. Copyright © 2004 John Wiley & Sons, Ltd. 相似文献
20.
Debasish Kundu Monalisa Sarma Debasis Samanta Rajib Mall 《Software Testing, Verification and Reliability》2009,19(4):297-333
This paper presents an approach to generate test cases from UML 2.0 sequence diagrams and subsequently prioritize those test cases using model information encapsulated in the sequence diagrams. The test cases generated according to the proposed approach satisfy the scenario coverage criterion and are suitable for system‐level testing. For prioritizing test cases, three different prioritization metrics are proposed. The values of these prioritization metrics can be analytically computed from the model information only. This paper also presents an approach to generate test data using a concept called rule‐based matrix. The prioritization metrics are used to control the number of test data without compromising the test adequacy. The effectiveness of the proposed approach has been verified using two industrial designs. Copyright © 2009 John Wiley & Sons, Ltd. 相似文献