基于用户行为和会话的Web应用测试方法

Web应用程序难以创建有效的测试用例,使其既能满足测试的需求,又能测试Web应用程序的多用户交互行为。针对上述问题,采用从Web日志数据中得到域数据创建测试用例的方法,从Web日志中获取用户行为数据和用户会话数据。提出一种基于用户行为和用户会话的测试新方法,通过实验验证该方法在功能覆盖和故障检测方面的有效性。     

Web系统测试用例自动执行系统

由于Web应用系统的迅速发展,如何进行有效的Web测试对于保证Web系统的可靠性越来越重要。设计的Web功能测试用例自动执行系统,改进了传统的录制/回放方式,可以模拟用户操作,对Web系统进行功能性测试。提出一种基于XML的测试用例的结构描述,称之为X-Case,对于经常更新的Web系统,修改X-Case结构内的相关数据就可以进行回归测试,不需要重新生成大量的测试用例,节约Web测试的成本。该系统支持部分测试数据自动生成,对动态和静态网页均能执行自动化测试。     

基于页面对象的Web应用测试用例生成方法

针对Web应用测试用例生成过程中导航图规模过大以及存在冗余测试路径的问题,提出了一种基于Selenium页面对象设计模式和图遍历算法的Web应用测试用例生成方法。首先,通过将原始页面对象进行分类,以导航页面对象类为节点、导航方法为迁移边构建页面对象导航图;其次,结合图的最短路径算法提出了一种页面对象图算法（POGA）来遍历导航图以生成测试路径集;最后,提取测试路径,利用Faker生成模拟数据,进一步生成可以直接执行的测试用例。实验结果表明：与爬取Web应用生成导航图的规模相比,该方法约简率约为89%;与基于状态迁移生成Web应用测试用例的方法相比,该方法减小了冗余和不可行路径的数量,并进一步提高了页面对象的复用率和测试用例的可维护性。     

Web应用前后端融合的遗传算法并行化测试用例生成

Web应用测试用例生成并行化，是提升Web应用测试生成效率的一个有效手段.Web应用的前后端分离、事件驱动等特性，导致传统的并行化技术难以直接应用于Web应用的测试用例自动生成中.因此，如何针对Web应用进行并行化测试用例生成，是一项具有挑战性的工作.将种群并行化计算引入到基于遗传算法的Web应用前后端融合的测试用例生成中，通过线程池及调度逻辑、多浏览器进程管理及后端覆盖路径的获取，实现种群个体在多浏览器上的并行化执行及基于后端路径覆盖的适应度值并行化计算，以更高效地生成Web应用的测试用例.实验结果表明：相较于Web应用的GA串行化测试用例生成方法，所提的并行化测试生成方法能更充分地利用系统资源，极大地提升Web应用测试用例的生成效率.     

基于用户会话的Web应用性能测试方法的研究

近年来,Web应用成为互联网时代基础设施中最重要的组成部分。随着互联网技术的不断进步以及使用用户数量的不断增长,Web应用也越发变得复杂。为了保证Web应用性能测试的有效性和真实性,提出一种利用服务器日志中的用户会话信息、结合经过序列匹配算法改进的层次凝聚算法来自动产生Web应用性能测试脚本的方法。提出的方法通过从服务器日志中提取出用户会话信息,分析用户真实的访问模式,通过聚类算法将同类型的用户访问模式进行聚集,自动产生相应的测试请求序列,生成测试集。相比于传统的基于录制/回访的测试方法,基于用户会话的性能测试方法通过对用户会话进行聚类分析,还原实际的用户访问模式,减少了设计测试用例时的人工参与,保证了测试结果的有效性和真实性。     

回归测试中的测试用例优先排序技术述评

测试用例优先排序(test case prioritization,简称TCP)问题是回归测试研究中的一个热点.通过设定特定排序准则,对测试用例进行排序以优化其执行次序,旨在最大化排序目标,例如最大化测试用例集的早期缺陷检测速率.TCP问题尤其适用于因测试预算不足以致不能执行完所有测试用例的测试场景.首先对TCP问题进行描述,并依次从源代码、需求和模型这3个角度出发对已有的TCP技术进行分类;然后对一类特殊的TCP问题(即测试资源感知的TCP问题)的已有研究成果进行总结;随后依次总结实证研究中常用的评测指标、评测数据集和缺陷类型对实证研究结论的影响;接着依次介绍TCP技术在一些特定测试领域中的应用,包括组合测试、事件驱动型应用测试、Web服务测试和缺陷定位等;最后对下一步工作进行展望.     

基于历史数据和多目标优化的测试用例排序方法

针对如何提高测试用例序列的揭错效率和回归测试效益问题,提出一种基于历史数据和多目标优化的测试用例排序方法。首先,根据测试用例的文本主题相似性和代码覆盖相似性对测试用例集进行聚类,并根据历史执行信息对测试用例间的执行失败关系进行关联规则挖掘,从而为后续过程做准备;然后,利用多目标优化算法对每个类簇内的测试用例进行排序,在此之后生成最终排序序列,使相似的测试用例分隔开;最后,利用测试用例间的关联规则,动态调整测试用例执行次序,从而使可能失败的测试用例优先执行,以进一步提高缺陷检测效率。与随机排序方法、基于聚类的排序方法、基于主题模型的排序方法、基于关联规则和多目标优化的排序方法相比,所提方法的平均故障检测率（APFD）平均值分别提高了12.59%、5.98%、3.01%和2.95%,基于成本的平均故障检测率（APFDc）平均值分别提高了17.17%、5.04%、5.08%和8.21%。实验结果表明,所提方法能有效提高回归测试效益。     

基于聚类算法的测试用例排序

在软件开发过程中,回归测试是一项重要而复杂耗时的工作。测试用例排序技术作为提高测试效率的一种有效手段,是近年的研究热点之一。测试用例排序技术根据测试目标对测试用例进行排序,使得最优的测试用例能够优先执行。同时,各种软件信息的底层结构及关联信息有助于提高测试效率。不同的代码单元在测试过程中拥有不同的测试需求。因此,利用聚类算法能够有效地分析测试用例间的结构信息,从而指导测试用例的排序过程。实验结果表明,新的测试用例排序策略有效地提高了测试用例排序的效果,能够更早地检测到软件错误。     

基于蚁群算法的测试用例优先排序

测试用例优先排序技术通过优化测试用例的执行次序来提高软件测试的效率,是增强型软件测试和回归测试的重要研究课题。针对基于需求的测试用例优先排序问题,提出了一种基于蚁群算法的求解方法,采用不同的测试用例间距离及用例序列评价策略,给出了该方法的2种不同实现方式。首先,针对黑盒测试特点,设计了基于需求的一般性测试用例序列评价指标;其次,提出测试用例吸引度概念,基于测试用例吸引度定义了测试用例间的距离;然后,给出了信息素更新策略、最优解集更新策略、局部最优解突变策略等主要设计策略,分别实现了该方法基于距离和基于指标的2种实现方式。实验结果表明,该方法具有很好的全局寻优能力,整体效果上优于粒子群算法、遗传算法和随机测试。     

基于数据流的测试用例自动生成研究

目前的数据流测试技术存在缺乏具体的数据驱动方法,测试用例的生成过程没有与测试需求结合起来等问题。基于此,文中提出一种自动生成测试用例的模型（TRGA）,利用控制流图（CFG）计算生成测试用例所需的变量的定义使用对,使用数据类型结构图（DTG）来作为创建测试对象的数据驱动,利用遗传算法的搜索能力来生成数据,并提出了一种新的适应度计算方法。实验结果表明,该模型能够在减少搜索时间,降低生成测试用例规模的同时达到较高的测试覆盖率。     

Applying Concept Analysis to User-Session-Based Testing of Web Applications

The continuous use of the Web for daily operations by businesses, consumers, and the government has created a great demand for reliable Web applications. One promising approach to testing the functionality of Web applications leverages the user-session data collected by Web servers. User-session-based testing automatically generates test cases based on real user profiles. The key contribution of this paper is the application of concept analysis for clustering user sessions and a set of heuristics for test case selection. Existing incremental concept analysis algorithms are exploited to avoid collecting and maintaining large user-session data sets and to thus provide scalability. We have completely automated the process from user session collection and test suite reduction through test case replay. Our incremental test suite update algorithm, coupled with our experimental study, indicates that concept analysis provides a promising means for incrementally updating reduced test suites in response to newly captured user sessions with little loss in fault detection capability and program coverage.     

Regression testing minimization,selection and prioritization: a survey

Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research. Copyright © 2010 John Wiley & Sons, Ltd.     

Improving the effectiveness of test suite reduction for user-session-based testing of web applications

ContextTest suite reduction is the problem of creating and executing a set of test cases that are smaller in size but equivalent in effectiveness to an original test suite. However, reduced suites can still be large and executing all the tests in a reduced test suite can be time consuming.ObjectiveWe propose ordering the tests in a reduced suite to increase its rate of fault detection. The ordered reduced test suite can be executed in time constrained situations, where, even if test execution is stopped early, the best test cases from the reduced suite will already be executed.MethodIn this paper, we present several approaches to order reduced test suites using experimentally verified prioritization criteria for the domain of web applications. We conduct an empirical study with three subject applications and user-session-based test cases to demonstrate how ordered reduced test suites often make a practical contribution. To enable comparison between test suites of different sizes, we develop Mod_APFD_C, a modification of the traditional prioritization effectiveness measure.ResultsWe find that by ordering the reduced suites, we create test suites that are more effective than unordered reduced suites. In each of our subject applications, there is at least one ordered reduced suite that outperforms the best unordered reduced suite and the best prioritized original suite.ConclusionsOur results show that when a tester does not have enough time to execute the entire reduced suite, executing an ordered reduced suite often improves the rate of fault detection. By coupling the underlying system’s characteristics with observations from our study on the criteria that produce the best ordered reduced suites, a tester can order their reduced test suites to obtain increased testing effectiveness.     

Directed test suite augmentation: an empirical investigation

Test suite augmentation techniques are used in regression testing to identify code elements in a modified program that are not adequately tested and to generate test cases to cover those elements. A defining feature of test suite augmentation techniques is the potential for reusing existing regression test suites. Our preliminary work suggests that several factors influence the efficiency and effectiveness of augmentation techniques that perform such reuse. These include the order in which target code elements are considered while generating test cases, the manner in which existing regression test cases and newly generated test cases are used, and the algorithm used to generate test cases. In this work, we present the results of two empirical studies examining these factors, considering two test case generation algorithms (concolic and genetic). The results of our studies show that the primary factor affecting augmentation using these approaches is the test case generation algorithm utilized; this affects both cost and effectiveness. The manner in which existing and newly generated test cases are utilized also has a substantial effect on efficiency and in some cases a substantial effect on effectiveness. The order in which target code elements are considered turns out to have relatively few effects when using concolic test case generation but in some cases influences the efficiency of genetic test case generation. The results of our first study, on four relatively small programs using a large number of test suites, are supported by our second study of a much larger program available in multiple versions. Together, the studies reveal a potential opportunity for creating a more cost‐effective hybrid augmentation approach leveraging both concolic and genetic test case generation techniques, while appropriately utilizing our understanding of the factors that affect them. Copyright © 2014 John Wiley & Sons, Ltd.     

测试用例排序的研究

回归测试是一个成本很高的测试过程。为了减少回归测试的成本,可以使用测试用例排序技术。测试用例排序是指按照事先确定的目标重新安排测试用例集中测试用例的执行次序,使得具有高优先级的测试用例比低优先级的测试用例在测试过程中更早执行。本文描述了测试用例排序问题;给出了两个一般测试用例排序算法,即总计排序算法和 附加排序算法;根据不同的覆盖准则（如语句、分支和定义-使用等）,可以从这两个一般算法得到对应的排序算法;最后,讨论了测试用例排序算法的有效性。     

基于提交排序和预测模型的测试套件选择方法

为在持续集成（CI）环境下减少回归测试集、提升回归测试的效率,提出一种适用于CI环境的回归测试套件选择方法。首先,根据每个提交的测试套件历史失败率和执行率信息,进行提交排序;然后,采用机器学习方法,对提交涉及的测试套件进行失败率预测,并选择具有较高失败率的测试套件。该方法综合使用提交排序技术和测试套件选择技术,从而保证既提高故障检测率又能在一定程度上降低测试成本。在Google的开源数据集上进行的实验结果表明：与同样采用提交排序的方法和采用测试套件选择的方法相比,所提方法的开销感知平均故障检测率APFDc提高了1%~27%;在相同的测试时间成本下,所提方法的测试召回提高了33.33~38.16个百分点,变更召回提高了15.67~24.52个百分点,测试套件选择率降低了约6个百分点。     

FSM-based testing from user defined faults adapted to incremental and mutation testing

We study the problem of deriving a test suite with guaranteed fault coverage from a given finite state machine specification with respect to some given user defined faults. We consider the case when an implementation under test can have more states than its specification while user defined faults are implemented in an arbitrary way. We show that our approach can be used for FSM-based incremental and mutation testing and correspondingly we investigate cases that can be used for reducing length of obtained test suites. In some cases, worst-case length of obtained test suite becomes polynomial. Experiments show significant gains is using our approach in comparison to testing the whole specification.     

基于概念分析的用户会话约减技术研究

基于用户会话的测试技术依赖于收集的用户会话数据集,数据集规模越大,测试效力就越强,但用于收集、储存和分析数据的成本也会迅速增加。该文提出一种在Web应用测试中对用户会话数据集进行约减的方法。运用概念分析技术聚类用户会话,从中选取数据,再利用增量式的概念分析算法更新数据集。设计一个试验原型框架用于评估新方法的实际效果。     

Static test case prioritization using topic models

Software development teams use test suites to test changes to their source code. In many situations, the test suites are so large that executing every test for every source code change is infeasible, due to time and resource constraints. Development teams need to prioritize their test suite so that as many distinct faults as possible are detected early in the execution of the test suite. We consider the problem of static black-box test case prioritization (TCP), where test suites are prioritized without the availability of the source code of the system under test (SUT). We propose a new static black-box TCP technique that represents test cases using a previously unused data source in the test suite: the linguistic data of the test cases, i.e., their identifier names, comments, and string literals. Our technique applies a text analysis algorithm called topic modeling to the linguistic data to approximate the functionality of each test case, allowing our technique to give high priority to test cases that test different functionalities of the SUT. We compare our proposed technique with existing static black-box TCP techniques in a case study of multiple real-world open source systems: several versions of Apache Ant and Apache Derby. We find that our static black-box TCP technique outperforms existing static black-box TCP techniques, and has comparable or better performance than two existing execution-based TCP techniques. Static black-box TCP methods are widely applicable because the only input they require is the source code of the test cases themselves. This contrasts with other TCP techniques which require access to the SUT runtime behavior, to the SUT specification models, or to the SUT source code.