测试成本约束下的测试用例选择寻优

在软件测试实践中,由于受到测试投入时间、投入资金和人力成本的约束,软件测试不可能以无限制投入来获取理想的软件质量。针对测试成本约束下条件下在测试用例集中如何选择测试用例以获取最优软件测试覆盖这样一个具有理论价值和现实意义的问题,提出了综合考虑测试成本、测试用例权重和测试用例相距度三要素的测试用例选择寻优新思路,给出了测试成本约束下测试用例选择寻优问题的形式化描述,提出了基于遗传算法和测试成本约束的测试用例选择寻优算法的两种基本算法,即测试用例权重优先的测试用例选择寻优算法以及集群测试用例优先的测试用例选择寻优算法,并以一个模拟测试用例集合对算法有效性进行了检验,实验结果表明两种测试用例选择寻优算法在测试覆盖效果上均显著优于随机选择算法。     

一种基于测试需求约简的测试用例集优化方法

测试用例集优化的目标是用尽可能少的测试用例充分满足给定的测试目标.针对给定的测试目标,获得精简的测试需求集有助于提高测试用例集优化的效率和效果.从测试需求约简的角度考虑测试用例集优化,首先给出可以精确描述测试需求间相互关系的测试需求约简模型;基于此模型,提出一种测试需求约简方法,可以获得精简测试需求集,作为测试用例集生成和约简的基础,从而实现测试用例集优化.实验结果表明,测试需求约简有助于获得规模较小的测试用例集,实现系统、科学、有效的测试.     

基于模糊逻辑的测试用例揭错能力分析

利用模糊逻辑，分析了各测试用例中交元的测试特性，得到如下结果：交元在不同测试用例中所发挥的测试作用是相同的。据此给出了生成充分考虑参数取值组合作用、且个数少的测试用例（测试用例基）的一种方法，建立了以测试用例基表示所有测试用例的表达式。利用模糊逻辑证明了用测试用例基执行测试，不仅能确保软件质量满足用户需求，且提高了测试用例的揭错能力。由于测试用例基具有数量少、测试效率高的特点，该文的研究有利于改善测试工作，降低测试成本。     

基于变异分析的测试用例约简方法

针对回归测试过程中由于测试需求的变更导致测试用例规模不断扩大、测试成本不断增加的问题,提出一种基于变异分析的测试用例约简方法（RTM）。首先,以测试用例能否检测到指定变异体为依据,对测试用例进行划分并创建二进制数值形式的变异体事务集矩阵;然后,应用改进的关联挖掘算法获取测试用例间的关联关系;最后,根据这些关联关系有效约简测试用例。6个经典程序仿真实验结果表明,RTM能够使约简后的测试用例约简率达到37%,与传统贪心算法和启发式算法相比,测试用例约简率提高了6%,且在提高测试用例约简率的同时,保证了测试覆盖率,单个测试用例的测试覆盖率平均提高了11%。所提方法能够利用尽可能少的测试用例满足更多的测试需求,有效提高了测试效率,降低了测试成本。     

软件测试入门浅谈

文中结合自己软件测试入门阶段的体会介绍了识别测试需求、设计测试用例和执行测试用例,其中识别测试需求重点介绍主动获取需求、确认需求的优先级两方面,设计测试用例着重介绍了测试用例的基本格式、加强测试用例的评审和定义测试用例的执行顺序,执行测试用例主要介绍搭建软件测试环境,执行测试用例和更新测试用例几个方面。     

软件回归测试中的自动测试生成方法

软件回归测试一般使用现有的测试用例集进行测试,如何有效利用这些用例成为回归测试的关键。研究自动对现有测试用例集的扩展,包括自动改变测试用例的执行顺序、自动进行测试用例数目的增减等,提出基于现有测试用例随机生成新的测试用例集的方法。实验证明,该方法生成的测试文件能在原有环境下成功执行,能实现自动测试范围的扩大或缩小,以及测试时间的增加或减少,为软件回归测试提供了有用的自动化工具。     

最小测试用例集生成方法改进及应用

软件测试是保证软件质量和可靠性的重要手段,如何对软件进行全面且高效的测试一直是备受关注的问题。分析了白盒测试与黑盒测试的优缺点;具体分析了最小测试用例生成算法,接着对生成最小测试用例集的方法提出改进：首先消除掉测试需求中存在的冗余,再对由该测试需求生成的测试用例集使用简化算法,得到一组无冗余的测试用例集。这种先对测试需求进行精简的方法,使得测试用例集中测试用例的数量大为减少,提高了简化算法的使用效率。将上述最小测试用例集生成方法运用到Apla到delphi生成器系统的测试中,提高了测试效率。     

最小测试用例集生成方法改进及应用

软件测试是保证软件质量和可靠性的重要手段,如何对软件进行全面且高效的测试一直是备受关注的问题.分析了白盒测试与黑盒测试的优缺点;具体分析了最小测试用例生成算法,接着对生成最小测试用例集的方法提出改进:首先消除掉测试需求中存在的冗余,再对由该测试需求生成的测试用例集使用简化算法,得到一组无冗余的测试用例集.这种先对测试需求进行精简的方法,使得测试用例集中测试用例的数量大为减少,提高了简化算法的使用效率.将上述最小测试用例集生成方法运用到Apla到delphi生成器系统的测试中,提高了测试效率.     

用于GUI软件的测试工具

为提高GUI软件测试的效率、降低软件成本,提出一种针对GUI软件的测试工具.该工具由测试配置设定模块、测试用例设计模块和测试用例执行模块组成.首先根据测试需求进行测试配置设定生成INI文件,然后根据INI文件内容将测试对象窗体中需要测试的控件及属性设定到测试用例设计界面中,在界面中进行测试用例设计,并将设计数据保存到XML文件中,测试用例执行模块通过读取XML文件中的数据完成测试用例的自动执行,同时进行测试结果判断,最后生成测试报告.测试人员只需输入数据,不用编写测试脚就能实现测试用例的自动执行,能减轻测试人员负担,提高软件测试效率.     

基于众测操作记录的GUI软件测试用例生成

众包测试是一种新兴的软件测试模式,具有测试周期短、测试成本低的优点.但是在众包测试中也存在测试人员水平不一、无法形成规范化测试用例的问题.文章提出一种基于操作记录的GUI软件测试用例生成方法,该方法将测试人员操作记录转化为规范化测试用例.实验结果表明,相对于手工测试生成的测试用例,该方法生成的测试用例在缺陷检测方面有显...     

Using coverage to automate and improve test purpose based testing

Test purposes have been presented as a solution to avoid the state space explosion when selecting test cases from formal models. Although such techniques work very well with regard to the speed of the test derivation, they leave the tester with one important task that influences the quality of the overall testing process: test purposes have to be formulated manually. In this paper, we present an approach that assists a test engineer with test purpose design in two ways: it allows automatic generation of coverage based test suites and can be used to automatically exercise those aspects of the system that are missed by hand-crafted test purposes. We consider coverage of Lotos specifications, and show how labeled transition systems derived from such specifications have to be extended in order to allow the application of logical coverage criteria to Lotos specifications. We then show how existing tools can be used to efficiently derive test cases and suggest how to use the coverage information to minimize test suites while generating them.     

SQL标准符合性测试的框架

文章阐述了基于美国国家标准与技术研究所(NIST)的SQL标准符合性测试用例集(SQLTestSuiteVersion6)设计开发的一个调用式SQL标准符合性测试平台框架,并针对实施中的测试用例的生成和管理、由用例之间相关性引起的用例结果错误的处理、测试结果的分析和分析结果的统计等问题进行了剖析。     

A framework to test interval arithmetic libraries and their IEEE 1788-2015 compliance

As developers of libraries implementing interval arithmetic, we faced the same difficulties when it comes to testing our libraries. What must be tested? How can we devise relevant test cases for unit testing? How can we ensure a high (and possibly 100%) test coverage? Before considering these questions, we briefly recall the main features of interval arithmetic and of the IEEE 1788-2015 standard for interval arithmetic. After listing the different aspects that, in our opinion, must be tested, we contribute a first step towards offering a test suite for an interval arithmetic library. First we define a format that enables the exchange of test cases, so that they can be read and tried easily. Then we offer a first set of test cases, for a selected set of mathematical functions. Next, we examine how the Julia interval arithmetic library, IntervalArithmetic.jl, actually performs to these tests. As this is an ongoing work, we list extra tests that we deem important to perform.     

Distributing test cases more evenly in adaptive random testing

Adaptive random testing (ART) has recently been proposed to enhance the failure-detection capability of random testing. In ART, test cases are not only randomly generated, but also evenly spread over the input domain. Various ART algorithms have been developed to evenly spread test cases in different ways. Previous studies have shown that some ART algorithms prefer to select test cases from the edge part of the input domain rather than from the centre part, that is, inputs do not have equal chance to be selected as test cases. Since we do not know where the failure-causing inputs are prior to testing, it is not desirable for inputs to have different chances of being selected as test cases. Therefore, in this paper, we investigate how to enhance some ART algorithms by offsetting the edge preference, and propose a new family of ART algorithms. A series of simulations have been conducted and it is shown that these new algorithms not only select test cases more evenly, but also have better failure detection capabilities.     

ODBC标准符合性测试框架

阐述了国产DBMS测评体系中ODBC标准符合性测试的测试框架,并针对实施过程中测试用例的产生、测试用例的独立性和可扩展性及测试结果的易分析性进行了剖析。     

Similarity-based prioritization of test case automation

The importance of efficient software testing procedures is driven by an ever increasing system complexity as well as global competition. In the particular case of manual test cases at the system integration level, where thousands of test cases may be executed before release, time must be well spent in order to test the system as completely and as efficiently as possible. Automating a subset of the manual test cases, i.e, translating the manual instructions to automatically executable code, is one way of decreasing the test effort. It is further common that test cases exhibit similarities, which can be exploited through reuse when automating a test suite. In this paper, we investigate the potential for reducing test effort by ordering the test cases before such automation, given that we can reuse already automated parts of test cases. In our analysis, we investigate several approaches for prioritization in a case study at a large Swedish vehicular manufacturer. The study analyzes the effects with respect to test effort, on four projects with a total of 3919 integration test cases constituting 35,180 test steps, written in natural language. The results show that for the four projects considered, the difference in expected manual effort between the best and the worst order found is on average 12 percentage points. The results also show that our proposed prioritization method is nearly as good as more resource demanding meta-heuristic approaches at a fraction of the computational time. Based on our results, we conclude that the order of automation is important when the set of test cases contain similar steps (instructions) that cannot be removed, but are possible to reuse. More precisely, the order is important with respect to how quickly the manual test execution effort decreases for a set of test cases that are being automated.     

Automating Test Case Selection in Model-Based Software Product Line Development

We address the problem of how to select test cases for products in a controlled model-based software product line development process. CVL, the common variability language, gives a framework for materialisation of product models from a given base model, variability model and resolution model. From such product models, software products can be derived. In practise, test case development for the product line often is independent from the product development. Therefore, the problem arises which test cases can be applied to which products. In particular, the question is whether a test case for one speci c product can be also used for a "similar" product. In this paper, we show how the expected outcome of a test case to a product in a model-based software product line development can be determined. That is, we give a procedure for assigning the outcome of a given test case on an arbitrary member of a software product line. We recall the relevant de nitions for software product line engineering, describe our approach, and demonstrate it with the example of a product line of super-automatic espresso machines.     

基于用户会话的Web应用性能测试方法的研究

近年来,Web应用成为互联网时代基础设施中最重要的组成部分。随着互联网技术的不断进步以及使用用户数量的不断增长,Web应用也越发变得复杂。为了保证Web应用性能测试的有效性和真实性,提出一种利用服务器日志中的用户会话信息、结合经过序列匹配算法改进的层次凝聚算法来自动产生Web应用性能测试脚本的方法。提出的方法通过从服务器日志中提取出用户会话信息,分析用户真实的访问模式,通过聚类算法将同类型的用户访问模式进行聚集,自动产生相应的测试请求序列,生成测试集。相比于传统的基于录制/回访的测试方法,基于用户会话的性能测试方法通过对用户会话进行聚类分析,还原实际的用户访问模式,减少了设计测试用例时的人工参与,保证了测试结果的有效性和真实性。     

Should software testers use mutation analysis to augment a test set?

Mutation testing has historically been used to assess the fault-finding effectiveness of a test suite or other verification technique. Mutation analysis, rather, entails augmenting a test suite to detect all killable mutants. Concerns about the time efficiency of mutation analysis may prohibit its widespread, practical use. The goal of our research is to assess the effectiveness of the mutation analysis process when used by software testers to augment a test suite to obtain higher statement coverage scores. We conducted two empirical studies and have shown that mutation analysis can be used by software testers to effectively produce new test cases and to improve statement coverage scores in a feasible amount of time. Additionally, we find that our user study participants view mutation analysis as an effective but relatively expensive technique for writing new test cases. Finally, we have shown that the choice of mutation tool and operator set can play an important role in determining how efficient mutation analysis is for producing new test cases.