A practical model‐based statistical approach for generating functional test cases: application in the automotive industry

With the growing complexity of industrial software applications, industrials are looking for efficient and practical methods to validate the software. This paper develops a model‐based statistical testing approach that automatically generates online and offline test cases for embedded software. It discusses an integrated framework that combines solutions for three major software testing research questions: (i) how to select test inputs; (ii) how to predict the expected results of a test; and (iii) when to stop testing software. The automatic selection of test inputs is based on a stochastic test model that accounts for the main particularity of embedded software: time sensitivity. Software test practitioners may design one or more test models when they generate random, user‐oriented, or fault‐oriented test inputs. A formal framework integrating existing and appropriate specification techniques was developed for the design of automated test oracles (executable software specifications) and the formal measurement of functional coverage. The decision to stop testing software is based on both test coverage objectives and cost constraints. This approach was tested on two representative case studies from the automotive industry. The experiment was performed at unit testing level in a simulated environment on a host personal computer (automatic test execution). The two software functionalities tested had previously been unit tested and validated using the test design approach conventionally used in the industry. Applying the proposed model‐based statistical testing approach to these two case studies, we obtained significant improvements in performing functional unit testing in a real and complex industrial context: more bugs were detected earlier and in a shorter time. Copyright © 2012 John Wiley & Sons, Ltd.     

嵌入式软件仿真测试脚本语言的设计与实现

软件测试中,脚本技术的引入是实现软件测试自动化技术的有效手段。软件测试脚本化可以减少测试人员的工作量,提高软件测试的可维护性。另外,脚本语言具有较好的移植性,有利于实现跨平台,可提高脚本代码的可重用性和测试的可重复性。结合嵌入式软件测试的特点,提出了针对嵌入式软件仿真测试的通用测试脚本语言ESSTSL,并对该语言的设计以及在测试环境中的实现机制进行了阐述,并将其应用于实际的软件测试中。     

嵌入式软件语句覆盖率测试插桩技术

针对基于宿主机的嵌入式软件测试,提出一种单元测试中通用的语句覆盖率测试方法,通过插桩技术,采用向源代码插桩实现语句覆盖率测试。设计了测试代码的实现算法,通过测试代码可以自动完成向被测代码插桩。这些方法被成功地应用到笔者所在项目组开发的嵌入式软件仿真测试平台ARMtest上。利用这些方法,在嵌入式硬件系统未完成开发之前,可通过宿主机环境和仿真环境及时发现嵌入式软件开发初期的一些不足并加以完善。     

基于ARINC429接口板的嵌入式底层软件测试平台设计

为满足底层软件的使用需求及减少其存在覆盖率低、消耗成本大、丢包率高、数据采集频率小等问题,提出并设计了基于ARINC429接口板的嵌入式底层软件测试平台。根据ARINC429接口板的结构框图能够设计出嵌入式底层软件测试平台,针对用户安全登录问题,设计了平台登录验证功能;通过对嵌入式底层软件运行信息显示,能够实时对软件运行状态进行监测,进而为数据采集与查询提供支持;设计工具链作为代码编译开发和结构测试工具;通过数据采集、处理、传输来完善软件测试平台的覆盖率。通过实验对比分析可知,该测试平台能够提高测试覆盖率,降低丢包率,促使数据采集频率大大增强,为软件测试平台的安全有效运行提供了保障。     

嵌入式API测试套生成方法和技术

随着嵌入式计算机系统应用的不断扩展,嵌入式系统的可靠性引起了学术界和工业界的广泛关注,也提出了很多增进可靠性的方法和技术.然而,现有的方法和技术在测试套生成方面论述不多,所以在处理大批量嵌入式系统测试工作中遇到了挑战.讨论抽象测试套生成方法和适配技术,提出了LTS(labeled transition system)到BT(behavior tree)的转换算法,从而使TTCN(test and testing control notation)测试套可以通过转换嵌入式软件的LTS描述产生.还介绍了基于上述转换算法的嵌入式软件测试工具包,以及一个嵌入式物联网识读器测试案例研究.     

浮力调节软件的单元测试技术研究

为了提高嵌入式软件的单元测试效率,同时能达到很好的测试效果,针对嵌入式软件制定单元测试充分性准则和单元测试策略,并使用测试工具对嵌入式软件进行单元测试具有实际意义;以浮力调节软件为例,通过研究基于控制流的单元测试充分性准则,结合浮力调节软件的单元测试需求,提出了针对浮力调节软件的单元测试充分性准则;同时,考虑到圈复杂度和函数节点数对函数正确实现的影响,制定了基于优先级的单元测试策略;利用自动化单元测试工具Testbed搭建了浮力调节软件动态测试环境,通过代码覆盖率分析,帮助创建测试用例以达到单元测试充分性要求,从而实现了浮力调节软件单元测试自动化;在自动化测试工具的帮助下,结合单元测试充分性准则和单元测试策略,最终实现严格而高效的单元测试。     

All‐uses testing of shared memory parallel programs

Parallelism has become a way of life for many scientific programmers. A significant challenge in bringing the power of parallel machines to these programmers is providing them with a suite of software tools similar to the tools that sequential programmers currently utilize. Unfortunately, writing correct parallel programs remains a challenging task.In particular, automatic or semi‐automatic testing tools for parallel programs are lacking. This paper takes a first step in developing an approach to providing all‐uses coverage for parallel programs. A testing framework and theoretical foundations for structural testing are presented, including test data adequacy criteria and hierarchy, formulation and illustration of all‐uses testing problems, classification of all‐uses test cases for parallel programs, and both theoretical and empirical results with regard to what can be achieved with all‐uses coverage for parallel programs. Copyright © 2003 John Wiley & Sons, Ltd.     

基于仿真的嵌入式软件白盒覆盖测试系统设计

随着软件系统变得越来越复杂和庞大,如何对它进行快速有效的测试已经成为一大热点。该系统实现了基于仿真的嵌入式软件覆盖测试系统。系统通过对源程序进行词法语法分析、自动插装、测试用例的生成和加入、动态仿真运行及覆盖率报告分析等得到测试结果,有效地解决了传统的基于目标测试带来的一些问题,基本满足了国内用户对嵌入式软件测试的需求。     

面向对象测试代码复用算法与工具

本文主要研究面向对象软件系统的自动化测试代码的复用问题。复用测试代码是基于建立的方法测试用例。当创建了一个方法的测试用例之后,可以发现这个用例可在其他相 关方法的测试过程中起到多种作用。这个生成的测试用例可以作为另一个方法测试输入的对象参数,或者作为一个主测试对象以调用其方法。我们设计了一个可自动生成复用用测试代码的算法,并应用于一个微软平台上的对象测试自动化工具的研发过程中。     

一种软件自动测试系统

针对航天测控任务频度高,测控任务软件系统测试重复性高、人工测试工作量大、测试效率和可靠性低的特点,通过分析系统测试条件和需求,设计了一种基于黑盒测试的软件自动测试系统.文章介绍了自动测试系统的系统结构、主要功能模块、测试平台部署及测试流程等,并详细阐述了测试用例管理、测试数据自动生成、测试过程控制、测试结果评估和测试报告自动生成等关键技术;该系统能够有效的减少人工重复工作量,增强系统测试的自动化,提高系统测试效率和质量.     

工业TD-SCDMA系统的软件测试策略研究

对工业TD-SCDMA系统的软件测试策略进行研究,能够有效解决传统系统频率和扰码规划存在扰码覆盖率低、性能差、精准度低等问题。针对不同频率复用方式特征,对基站布局进行设计,获取频率配置结果进行优化,得出频率重新规划结果;采用Galib遗传算法,对簇复用码和互斥性码等系统扰码规划进行改进,完成工业TD-SCDMA系统的软件测试策略研究。实验结果表明,该软件测试策略能够有效提高TD-SCDMA系统的扰码覆盖率、性能和精准度,充分满足TD-SCDMA系统规划要求。     

通用嵌入式系统软件测试平台的设计

仿真测试是嵌入式软件系统测试阶段的一种有效测试方法,探讨了通过在PC机上仿真模拟ARM嵌入式系统,对嵌入式系统软件进行仿真测试的通用测试平台的设计。该平台可以在不做大幅度修改的情况下对不同的嵌入式系统软件进行各种测试。重点介绍了仿真模拟器、测试管理器和测试平台专用工具链的设计,提出了一种测试管理器的实现模型,并进行了详细的描述。     

基于覆盖测试的嵌入式软件自动裁剪

针对软件移植嵌入式平台时的裁剪问题,提出一个通用的、自动化的裁剪方案。该方案基于覆盖测试思想,利用源代码静态分析技术,在每个函数的首尾插入桩代码,通过执行已插桩的程序,动态地获得函数的覆盖信息。根据函数的覆盖信息,能对软件进行自动化的裁剪。该裁剪方案可以将软件体积减少30%左右。     

基于目标码的测试覆盖不可达分析方法

基于目标码的测试覆盖率分析是软件测试过程的必要关键步骤,不可达分析能够保证测试的完整性和充分性.给出嵌入式软件基于覆盖率测试的分析过程,在嵌入式虚拟测试平台的基础上,对程序目标代码插桩,采用语句和分支覆盖率分析准则,将黑盒测试和白盒测试相结合,分析不可达分支语句引入机理,并通过具体的反汇编代码实例分析来验证这种测试方法的可行性和有效性.专门针对覆盖率不可达的分析可以有效验证软件功能,发现软件缺陷,进一步提升软件质量.     

Estimating software testing complexity

ContextComplexity measures provide us some information about software artifacts. A measure of the difficulty of testing a piece of code could be very useful to take control about the test phase.ObjectiveThe aim in this paper is the definition of a new measure of the difficulty for a computer to generate test cases, we call it Branch Coverage Expectation (BCE). We also analyze the most common complexity measures and the most important features of a program. With this analysis we are trying to discover whether there exists a relationship between them and the code coverage of an automatically generated test suite.MethodThe definition of this measure is based on a Markov model of the program. This model is used not only to compute the BCE, but also to provide an estimation of the number of test cases needed to reach a given coverage level in the program. In order to check our proposal, we perform a theoretical validation and we carry out an empirical validation study using 2600 test programs.ResultsThe results show that the previously existing measures are not so useful to estimate the difficulty of testing a program, because they are not highly correlated with the code coverage. Our proposed measure is much more correlated with the code coverage than the existing complexity measures.ConclusionThe high correlation of our measure with the code coverage suggests that the BCE measure is a very promising way of measuring the difficulty to automatically test a program. Our proposed measure is useful for predicting the behavior of an automatic test case generator.     

嵌入式软件覆盖测试工具的研究

随着嵌入式系统的复杂性不断提高,嵌入式软件测试变得越来越重要;通过对ARM系统软件进行深入的研究,提出了一种应用于ARM嵌入式软件系统的覆盖测试工具的设计原理、系统模型及组成模块,分别就各个模块的具体设计进行了详细说明,着重探讨了词法语法分析和插装方式,提出了一种新的词法语法分析和插装方式;该工具能够实现ARM系列嵌入式软件的自动测试,测试了代码执行的覆盖情况,检查出代码死区,从而提高了测试效率,使嵌入式软件质量得到保证.     

Using combinatorial testing to build navigation graphs for dynamic web applications

Modelling a software system is often a challenging prerequisite to automatic test case generation. Modelling the navigation structure of a dynamic web application is particularly challenging because of the presence of a large number of pages that are created dynamically and the difficulty of reaching a dynamic page unless a set of appropriate input values are provided for the parameters. To address the first challenge, some form of abstraction is required to enable scalable modelling. For the second challenge, techniques are required to select appropriate input values for parameters and systematically combine them to reach new pages. This paper presents a combinatorial approach in building a navigation graph for dynamic web applications. The navigation graph can then be used to automatically generate test sequences for testing web applications. The novelty of our approach is twofold. First, we use an abstraction scheme to control the page explosion problem, where pages that are likely to have the same navigation behaviour are grouped together and are represented as a single node in the navigation graph. Second, assuming that values of individual parameters are supplied manually or generated from other techniques, we combine parameter values such that well‐defined combinatorial coverage of input parameter values is achieved. Using combinatorial coverage can significantly reduce the number of requests that have to be submitted while still achieving effective coverage of the navigation structure. We implement our combinatorial approach in a tool, Tansuo, and apply the tool on seven open‐source web applications. We evaluate the effectiveness of Tansuo's exploration process guided by t‐way coverage, for t = 1,2,3, with respect to code coverage, and find that the navigation structure exploration by Tansuo, in general, results in high code coverage (more than 80% statement coverage for most of our subject applications when dead code is removed). We compare Tansuo's effectiveness with two other navigation graph tools and find that Tansuo is more effective. Our empirical results indicate that using pairwise coverage in Tansuo results in the efficient generation of navigation graphs and effective exploration of dynamic web applications. Copyright © 2016 John Wiley & Sons, Ltd.     

面向RISC-V的汇编程序语义等价性自动化测试系统

本文设计并实现了一套面向RISC-V的汇编程序语义等价性自动化测试系统.在面向RISC-V开发软件时,尤其是基于扩展指令(例如向量指令)编写高效的程序时,很难避免以手写汇编的方式编写代码.例如,为标准的C函数库编写相应的向量版函数.与编译器自动生成的代码不同,手写的汇编代码虽然可以最大限度地提高程序的效率,但因绕过了编译时对程序的约束(如类型检查、寄存器分配等)而对开发者提出了更高的要求.能否对新版本与标准版本的汇编程序进行快速地、自动化的语义等价性测试,将大大影响代码的正确性和软件开发和调试的效率.已有面向RISC-V的测试框架缺乏对语义等价性测试的支持,也未考虑程序执行带来的副作用.本研究基于模拟器的动态测试环境,设计并实现了一套面向RISC-V的汇编程序语义等价性自动化测试系统.系统通过跟踪机器状态,捕获程序执行的副作用,并结合用户定义的测试目标生成测试报告.实验表明,本系统相比已有的测试系统,能够有效地对RISC-V汇编程序的语义等价性进行测试.     

对逻辑覆盖软件测试准则的公理化评估

由于形式规格说明采用一种精确、一致、容易被机器自动处理的符号系统来描述软件需求,因而形式规格说明为测试用例的自动生成和软件功能的验证提供了基础.在基于形式规格说明的测试过程中逻辑覆盖测试准则是一组常用的测试准则,如何选择和使用其中的每个测试准则是应用这组测试准则时面临的主要问题.因此分析和比较这组测试准则中每个测试准则的性质将为测试工程师选择测试准则提供指导和帮助.对测试充分性准则的公理化评估是一种比较测试准则的方式,这种方式将对理想的测试准则的直觉需求定义为一组公理,然后通过检查测试准则是否满足该组公理来分析和比较相应的测试准则.描述了一组理想的逻辑覆盖测试准则应该具有的性质和用来确定一个测试充分性准则是否完全的生成算法.这组性质被形式化地定义为一组公理.利用这种形式化的定义,用定理的形式精确地给出了这些性质之间的关系.最后通过这组公理系统来评估现有的逻辑覆盖测试准则.评估的结果为测试人员在实际过程中选择逻辑覆盖测试准则提供了指导.     

基于MBD模型自动生成测试用例的软件测试方法

在安全关键软件领域应用基于模型的开发技术时,存在着软件测试充分性的问题。提出了一种基于MBD模型自动生成测试用例、对航空发动机控制软件进行测试的新方法。通过使用MathWorks提供的自动生成测试用例工具箱(SLDV),基于FADEC控制软件的Simulink模型,自动生成满足控制软件模型MC/DC覆盖率100%的测试用例,并在Cppunit测试框架下执行测试用例;通过对比模型输出和测试输出来发现被测对象的问题,确保软件代码与模型的一致性。项目实践表明,该测试方法是软件集成测试的有益补充,提升了软件测试的充分性。