Employing user profiles to test a new version of a GUI component in its context of use

Advancement in reusable component technology has had a significant impact on the development of complex graphical user interfaces (GUIs), which are front-ends to most of today’s software. Software developers can, with very little effort, integrate components into their software’s GUI. Problems, however, arise when new versions of GUI components replace their predecessors in an implementation. Often, the inclusion of a new version of a component breaks some part of the software, i.e., tasks that end-users were able to perform before modifications were made can no longer be performed. Software developers (who also happen to be component users) are unable to perform adequate regression testing in this context because of several factors, including lack of source code, environmental side-effects on GUI rendering, event-driven nature of GUIs, and large number of possible permutations of events. This paper presents a new technique that end-users’ quality assurance (QA) teams can employ to test the new version of a component in its application context by using the existing version as a baseline. The technique combines lightweight event-level dynamic profiling to collect user profiles in a transparent manner, GUI reverse engineering to extract the structure of the component’s GUI, test case execution to replay the collected profiles on the new version, and GUI oracles that collect properties from the existing version. Empirical studies demonstrate the practicality, usefulness, and limitations of the technique. Atif M. Memon is an Assistant Professor at the Department of Computer Science, University of Maryland. He received his BS, MS, and Ph.D. in Computer Science in 1991, 1995, and 2001 respectively. He was awarded a Gold Medal in BS. He was awarded Fellowships from the Andrew Mellon Foundation for his Ph.D. research. He received the NSF CAREER award in 2005. His research interests include program testing, software engineering, artificial intelligence, plan generation, reverse engineering, and program structures. He is a member of the ACM and the IEEE Computer Society.     

A language-independent approach to black-box testing using Erlang as test specification language

Integration of reused, well-designed components and subsystems is a common practice in software development. Hence, testing integration interfaces is a key activity, and a whole range of technical challenges arise from the complexity and versatility of such components.     

基于概率统计的测试用例生成技术研究

软件可靠性测试是软件工程中的一个新的研究领域。为了缩短软件测试周期,测试用例的生成是关键,提出将测试用例的生成问题转化为一个数学问题。通过实例证明,这种测试技术具有一定实用性和有效性。     

Distributing test cases more evenly in adaptive random testing

Adaptive random testing (ART) has recently been proposed to enhance the failure-detection capability of random testing. In ART, test cases are not only randomly generated, but also evenly spread over the input domain. Various ART algorithms have been developed to evenly spread test cases in different ways. Previous studies have shown that some ART algorithms prefer to select test cases from the edge part of the input domain rather than from the centre part, that is, inputs do not have equal chance to be selected as test cases. Since we do not know where the failure-causing inputs are prior to testing, it is not desirable for inputs to have different chances of being selected as test cases. Therefore, in this paper, we investigate how to enhance some ART algorithms by offsetting the edge preference, and propose a new family of ART algorithms. A series of simulations have been conducted and it is shown that these new algorithms not only select test cases more evenly, but also have better failure detection capabilities.     

基于系统功能测试的软件自动化测试可行性分析

本文基于软件自动化测试的特点和项目的实际需要,比较传统的手工测试和自动测试两种方法,介绍了如何进行软件自动化测试可行性分析。重点基于项目的系统功能测试,讨论软件自动化测试的收益问题,最终得出影响软件自动测试可行性程度的因素。     

Qualifying input test data for model transformations

Model transformation is a core mechanism for model-driven engineering (MDE). Writing complex model transformations is error-prone, and efficient testing techniques are required as for any complex program development. Testing a model transformation is typically performed by checking the results of the transformation applied to a set of input models. While it is fairly easy to provide some input models, it is difficult to qualify the relevance of these models for testing. In this paper, we propose a set of rules and a framework to assess the quality of given input models for testing a given transformation. Furthermore, the framework identifies missing model elements in input models and assists the user in improving these models.

GA-based multiple paths test data generator

Developers have learned over time that software testing costs a considerable amount of a software project budget. Hence, software quality managers have been looking for solutions to reduce testing costs and time. Considering path coverage as the test adequacy criterion, we propose using genetic algorithms (GA) for automating the generation of test data for white-box testing. There are evidences that GA has been already successful in generating test data. However, existing GA-based test data generators suffer from some problems. This paper presents our approach to overcome one of these problems; that is the inefficiency in covering multiple target paths. We have designed a GA-based test data generator that is, in one run, able to synthesize multiple test data to cover multiple target paths. Moreover, we have implemented a set of variations of the generator. Experimental results show that our test data generator is more efficient and more effective than others.     

A semantic approach for automated test oracle generation

This paper presents the design, implementation, and applications of a software testing tool, TAO, which allows users to specify and generate test cases and oracles in a declarative way. Extended from its previous grammar-based test generation tool, TAO provides a declarative notation for defining denotational semantics on each productive grammar rule, such that when a test case is generated, its expected semantics will be evaluated automatically as well, serving as its test oracle. TAO further provides a simple tagging mechanism to embed oracles into test cases for bridging the automation between test case generation and software testing. Two practical case studies are used to illustrate how automated oracle generation can be effectively integrated with grammar-based test generation in different testing scenarios: locating fault-inducing input patterns on Java applications; and Selenium-based automated web testing.     

成对测试中的测试生成策略研究

测试用例生成是软件测试的关键,成对测试是根据特定的测试原则研究测试用例的产生方法。基于这一原则分别构造了IPO-H算法和IPO-V算法的测试以产生整个测试用例,具有产生的测试用例少,时间消耗小等优点。并与另一个成对测试的测试生成工具AETG(高效自动测试生成器)进行了比较、分析,证明了改进的IPO策略便于构造自动测试工具。     

Fault-based test suite prioritization for specification-based testing

Context

Existing test suite prioritization techniques usually rely on code coverage information or historical execution data that serve as indicators for estimating the fault-detecting ability of test cases. Such indicators are primarily empirical in nature and not theoretically driven; hence, they do not necessarily provide sound estimates. Also, these techniques are not applicable when the source code is not available or when the software is tested for the first time.

Objective

We propose and develop the novel notion of fault-based prioritization of test cases which directly utilizes the theoretical knowledge of their fault-detecting ability and the relationships among the test cases and the faults in the prescribed fault model, based on which the test cases are generated.

Method

We demonstrate our approach of fault-based prioritization by applying it to the testing of the implementation of logical expressions against their specifications. We then validate our proposal by an empirical study that evaluates the effectiveness of prioritization techniques using two different metrics.

Results

A theoretically guided fault-based prioritization technique generally outperforms other techniques under study, as assessed by two different metrics. Our empirical results also show that the technique helps to reveal all target faults by executing only about 72% of the prioritized test suite, thereby reducing the effort required in testing.

Conclusions

The fault-based prioritization approach is not only applicable to the instance empirically validated in this paper, but should also be adaptable to other fault-based testing strategies. We also envisage new research directions to be opened up by our work.     

On the estimation of adequate test set size using fault failure rates

Test set size in terms of the number of test cases is an important consideration when testing software systems. Using too few test cases might result in poor fault detection and using too many might be very expensive and suffer from redundancy. We define the failure rate of a program as the fraction of test cases in an available test pool that result in execution failure on that program. This paper investigates the relationship between failure rates and the number of test cases required to detect the faults. Our experiments based on 11 sets of C programs suggest that an accurate estimation of failure rates of potential fault(s) in a program can provide a reliable estimate of adequate test set size with respect to fault detection and should therefore be one of the factors kept in mind during test set construction. Furthermore, the model proposed herein is fairly robust to incorrect estimations in failure rates and can still provide good predictive quality. Experiments are also performed to observe the relationship between multiple faults present in the same program using the concept of a failure rate. When predicting the effectiveness against a program with multiple faults, results indicate that not knowing the number of faults in the program is not a significant concern, as the predictive quality is typically not affected adversely.     

Analysing the effectiveness of rule-coverage as a reduction criterion for test suites of grammar-based software

The term grammar-based software describes software whose input can be specified by a context-free grammar. This grammar may occur explicitly in the software, in the form of an input specification to a parser generator, or implicitly, in the form of a hand-written parser. Grammar-based software includes not only programming language compilers, but also tools for program analysis, reverse engineering, software metrics and documentation generation. Hence, ensuring their completeness and correctness is a vital prerequisite for their use. In this paper we propose a strategy for the construction of test suites for grammar based software, and illustrate this strategy using the ISO C^+ + grammar. We use the concept of grammar-rule coverage as a pivot for the reduction of an implementation-based test suite, and demonstrate a significant decrease in the size of this suite. The effectiveness of this reduced test suite is compared to the original test suite with respect to code coverage and more importantly, fault detection. This work greatly expands upon previous work in this area and utilises large scale mutation testing to compare the effectiveness of grammar-rule coverage to that of statement coverage as a reduction criterion for test suites of grammar-based software. This work finds that when grammar rule coverage is used as the sole criterion for reducing test suites of grammar based software, the fault detection capability of that reduced test suite is greatly diminished when compared to other coverage criteria such as statement coverage.

Automatic generation of basis test paths using variable length genetic algorithm

Path testing is the strongest coverage criterion in white box testing. Finding target paths is a key challenge in path testing. Genetic algorithms have been successfully used in many software testing activities such as generating test data, selecting test cases and test cases prioritization. In this paper, we introduce a new genetic algorithm for generating test paths. In this algorithm the length of the chromosome varies from iteration to another according to the change in the length of the path. Based on the proposed algorithm, we present a new technique for automatically generating a set of basis test paths which can be used as testing paths in any path testing method. The proposed technique uses a method to verify the independency of the generated paths to be included in the basis set of paths. In addition, this technique employs a method for checking the feasibility of the generated paths. We introduce new definitions for the key concepts of genetic algorithm such as chromosome representation, crossover, mutation, and fitness function to be compatible with path generation. In addition, we present a case study to show the efficiency of our technique. We conducted a set of experiments to evaluate the effectiveness of the proposed path generation technique. The results showed that the proposed technique causes substantial reduction in path generation effort, and that the proposed GA algorithm is effective in test path generation.     

N因素S水平的配对测试集生成算法研究

在分析配对测试及其有效性的基础上,给出了一种新的关于n因素s(s≥2)水平的配对测试集生成算法。实验表明该算法与以往一些算法相比具有一定的优越性,最后将该算法运用于一个具体实例,并较详细地介绍了该实例的测试集生成过程。     

Web服务软件测试技术进展

Web服务软件由于其强大的功能和良好的特性得到了越来越广泛的应用,但针对Web服务测试方法和技术的研究还处于初级阶段.通过对Web服务软件的特点的分析,归纳了近年来一些典型的Web服务测试方法和技术,并对当前较为有代表性的研究工作进行了总结,探讨了若干研究方向.     

随机测试用例的优化技术研究

提出了一种基于粗糙集的不完备测试数据填补方法。该方法首先利用粗糙集中下近似集的性质对随机生成的测试数据进行填补,然后根据属性数据的取值概率函数求出的结果进行二次填补,从而完成对不完备测试数据的完备化处理,生成最优测试用例。采用本方法可以较好地反映待测系统所蕴含的规则,且可以避免测试数据的冲突。     

测试用例的设计和复用技术

软件测试是企业保证软件产品质量的一个重要手段,其中测试用例的设计是软件测试的关键,它一般包括功能测试用例的设计,结构测试用例设计以及系统方面的测试用例设计等.结合实际经验,系统地阐述了如何有效地进行测试用例的设计以及复用.并给出两个案例进行分析,探讨测试用例设计中的一些注意事项.     

支持共享和复用的测试用例库系统的设计

测试用例的共享和复用对于提高测试工作的效率极为重要,本文提出了一个支持共享和复用的测试用例库系统,文章首先时测试用例的管理属性和测试用例体进行建模,然后描述了系统的版本管理、导出导入和统计等多项功能。     

测试自动机的测试驱动器设计与实现技术

介绍了测试自动机的测试驱动器设计与实现技术,给出了测试驱动器接口的测试用例、测试输出的形式化定义,提出了一种通用的“客户／服务”测试驱动模型及其实现技术。