一种基于模型的测试充分性评估方法

测试充分性评估通常采用覆盖率的方法来评估测试对软件特征的覆盖充分程度。如今,传统的充分性评估方法难以满足复杂软件的测试评估需求。首先,代码覆盖准则难以准确验证软件需求;其次,软件测试还需考虑软件不同特征对系统测试充分性的不同影响。对此,提出一种基于接口的建模方法和基于该模型的综合覆盖充分性评估方法。该方法根据软件接口说明,对系统功能进行特征抽取、建模,并对接口模型的测试用例进行不同层级的充分性评估,对评佑结果进行归一化处理,得到系统的综合测试充分性。通过案例表明,这种评佑方法能够反映功能的测试充分性,对测试用例的设计和优化有一定指导意义。     

基于UML状态图的软件测试充分性准则研究

文中详细讨论了一组基于UML状态图的软件测试充分性准则。在软件测试时,还没有测试充分性准则明确定义循环应该被执行多少次,而循环只被执行一次是很不充分的。针对这种情况,在现有测试准则的基础上提出了两种扩展的基于UML状态图测试的充分性准则：ZOT-R循环覆盖准则和全ZOT-R路径覆盖准则,丰富并完善了现有的测试准则体系。并且通过结合一个具体实例给出了这些覆盖准则在具体测试实践中的应用方法,探讨了这些覆盖准则之间的包含关系。     

测试性试验中样本集的测试覆盖充分性研究

测试性试验结果的准确性不仅与样本量有关,还与试验中能够激活的测试数量有关.在建立故障模式测试模型的基础上,提出了样本集的测试覆盖充分性度量和准则.给出了模型中要素的确定方法,以及该度量和准则的应用流程.最后通过实例应用说明了测试覆盖充分性度量和准则的可行性.     

软件测试充分性研究

软件测试充分性准则是决定一个软件系统能否被充分测试的规则,而测试充分性的关键问题是其检测故障的能力。该文首先给出测试充分性准则的形式化描述,然后讨论了软件测试充分性的性质,最后给出了两种理想的测试数据生成方案,并提出了测试充分性度量方法。     

软件可靠性测试充分性研究

在阐述了软件测试充分性的定义和软件测试充分性准则的定义、分类和作用的基础上，形式化分析了软件正确性测试充分性和软件可靠性测试充分性的目的和实质，重点研究了软件可靠性测试的充分性，将软件可靠性测试充分性准则分为软件可靠性增长测试充分性准则和软件可靠性验证测试充分性准则，给出将测试充分性准则用于可靠性测试流程的模型。     

几种范畴划分测试充分性准则的比较

选择测试充分性准则对软件的测试是至关重要的。本文对范畴划分(Category—Partition)测试中的All—Combination、Each—Choice—Used和base—Choice—Coverage这三个充分性准则在揭错能力、软件可靠性、测试开销这几个方面进行分析和比较，并给出一个量化的比较结果，帮助测试员选择测试充分性准则。最后对基于票据计算的形式规格说明的例子进行了分析。     

常用的基于控制流的测试充分性准则

本文介绍了几种在软件测试中常用的基于控制流的测试充分性评价准则。这些测试充分性准则适用于用过程式语言、面向对象语言编写的程序。测试人员可用这些准则判断究竟测试了多少代码,还有多少代码未测试到。     

线性独立路径覆盖率的软件测试充分性判别方法

软件测试充分性判别准则是决定一个软件系统是否已经被充分测试的停止准则,而充分性判别准则的关键是它的揭错能力。对充分性判别准则进行了形式化描述,并且讨论了充分性判别准则的性质及准则之间的比较方法。为了给保障软件测试充分性提供理论依据,提出了一个软件测试充分性的度量准则。     

基于可靠性增长模型的软件可靠性增长测试充分性准则

结合软件可靠性增长模型,扩充了基于可信度度量的软件可靠性增长测试充分性问题度量准则,提出了关于可靠性增长测试充分性问题新的度量准则：可靠性测试只有同时满足可靠性增量指标要求、可信度要求和每单位测试资源发现的故障数要求方可终止。然后,以一个真实数据集为例,应用所提出度量准则求出了测试应该停止的时间。实验证明,这样的度量准则,使得终止条件更加严格,可靠性测试更加充分,为可靠性测试充分性问题的解决提供了新的思路。     

基于测试充分性准则的教育软件内容测试

教育软件内容测试是对教育软件根本性目标进行测试的需要,也是教育信息技术和教育软件工程研究的重点内容之一。基于测试充分性准则提出了教育软件内容测试,通过测试充分性准则来指导确定测试数据集,保障遵循学习特征的教育软件所包含的教育思想内容的正确性测试。     

Automatic Generation of Unit Tests for Correlated Variables in Parallel Programs

A notorious class of concurrency bugs are race condition related to correlated variables, which make up about 30 % of all non-deadlock concurrency bugs. A solution to prevent this problem is the automatic generation of parallel unit tests. This paper presents an approach to generate parallel unit tests for variable correlations in multithreaded code. We introduce a hybrid approach for identifying correlated variables. Furthermore, we estimate the number of potentially violated correlations for methods executed in parallel. In this way, we are capable of creating unit tests that are suited for race detectors considering correlated variables. We were able to identify more than 85 % of all race conditions on correlated variables in eight applications after applying our parallel unit tests. At the same time, we reduced the number of unnecessary generated unit tests. In comparison to a test generator unaware of variable correlations, redundant unit tests are reduced by up to 50 %, while maintaining the same precision and accuracy in terms of the number of detected races.     

一种基于变异分析的BPEL程序故障定位技术

不同于传统C,C++或Java程序,BPEL(Business Process Execution Language)程序由一组活动及其之间的交互组成,同时引入了并发、序列化、XML表示等新特征,这些新特点使得定位BPEL程序的故障具有一定的挑战性.针对现有故障定位技术在有效性方面的不足,提出一种基于变异分析的BPEL...     

A test-suite reduction approach to improving fault-localization effectiveness

In order to improve the effectiveness of fault localization, researchers are interested in test-suite reduction to provide suitable test-suite inputs. Different test-suite reduction approaches have been proposed. However, the results are usually not ideal. Reducing the test-suite improperly or excessively can even negatively affect fault-localization effectiveness. In this paper, we propose a two-step test-suite reduction approach to remove the test cases which have little or no effect on fault localization, and improve the distribution evenness of concrete execution paths of test cases. This approach consists of coverage matrix based reduction and path vector based reduction, so it analyzes not only the test cases coverage but also the concrete path information. We design and implement experiments to verify the effect of our approach. The experimental results show that our reduced test-suite can improve fault-localization effectiveness. On average, our approach can reduce the size of a test-suite in 47.87% (for Siemens programs) and 23.03% (for space program). At the same time, on average our approach can improve the fault-localization effectiveness, 2.12 on Siemens programs and 0.13 on space program by Tarantula approach.     

A formal semantics for debugging synchronous message passing-based concurrent programs

In this paper, we propose a semantic framework to debug synchronous message passing-based con- current programs, which are increasingly useful as parallel computing and distributed systems become more and more pervasive. We first design a concurrent programming language model to uniformly represent exist- ing concurrent programming languages. Compared to sequential programming languages, this model contains communication statements, i.e., sending and receiving statements, and a concurrent structure to represent com- munication and concurrency. We then propose a debugging process consisting of a tracing and a locating procedure. The tracing procedure re-executes a program with a failed test case and uses specially designed data structures to collect useful execution information for locating bugs. We provide for the tracing procedure a struc- tural operational semantics to represent synchronous communication and concurrency. The locating procedure backward locates the ill-designed statement by using information obtained in the tracing procedure, generates a fix equation, and tries to fix the bug by solving the fix equation. We also propose a structural operational semantics for the locating procedure. We supply two examples to test our proposed operational semantics.     

基于路径分析和信息熵的错误定位方法

软件错误定位是一项耗时又费力的工作,因此如何提高软件错误定位的自动化程度一直以来都是软件工程领域研究的热点.现有的基于频谱的错误定位方法很少利用程序的上下文信息,而程序的上下文信息对错误定位至关重要.针对这一问题,提出了一种基于路径分析和信息熵的错误定位方法FLPI.该方法在基于频谱信息技术的基础上,通过对所有执行路径...     

A Hardware Memory Race Recorder for Deterministic Replay

As hardware vendors transition to multicore chips, software vendors face increased software reliability challenges. To effectively debug software in this new world, developers must be able to replay executions that exhibit a bug so that they can zero in on concurrency bugs - especially intermittent ones. Such deterministic replay also aids fault detection and recovery, intrusion detection, and the like. Deterministic replay requires both a recorder and a replayer. The recorder logs information during multithreaded program execution that is sufficient to enable deterministic replay. In the postmortem analysis, the replayer uses the logged information, together with the program binary, to faithfully replay the original execution. The replay will always exercise the same bugs and produce the same outputs. Clearly, an investment in modest chip resources has great potential to ease the challenges of debugging the multithreaded software that both users and vendors will continue to demand     

On the adoption of MC/DC and control-flow adequacy for a tight integration of program testing and statistical fault localization

ContextTesting and debugging consume a significant portion of software development effort. Both processes are usually conducted independently despite their close relationship with each other. Test adequacy is vital for developers to assure that sufficient testing effort has been made, while finding all the faults in a program as soon as possible is equally important. A tight integration between testing and debugging activities is essential.ObjectiveThe paper aims at finding whether three factors, namely, the adequacy criterion to gauge a test suite, the size of a prioritized test suite, and the percentage of such a test suite used in fault localization, have significant impacts on integrating test case prioritization techniques with statistical fault localization techniques.MethodWe conduct a controlled experiment to investigate the effectiveness of applying adequate test suites to locate faults in a benchmark suite of seven Siemens programs and four real-life UNIX utility programs using three adequacy criteria, 16 test case prioritization techniques, and four statistical fault localization techniques. We measure the proportion of code needed to be examined in order to locate a fault as the effectiveness of statistical fault localization techniques. We also investigate the integration of test case prioritization and statistical fault localization with postmortem analysis.ResultThe main result shows that on average, it is more effective for a statistical fault localization technique to utilize the execution results of a MC/DC-adequate test suite than those of a branch-adequate test suite, and is in turn more effective to utilize the execution results of a branch-adequate test suite than those of a statement-adequate test suite. On the other hand, we find that none of the fault localization techniques studied can be sufficiently effective in suggesting fault-relevant statements that can fit easily into one debug window of a typical IDE.ConclusionWe find that the adequacy criterion and the percentage of a prioritized test suite utilized are major factors affecting the effectiveness of statistical fault localization techniques. In our experiment, the adoption of a stronger adequacy criterion can lead to more effective integration of testing and debugging.     

Spectral‐based fault localization using hyperbolic function

Debugging is crucial for producing reliable software. One of the effective bug localization techniques is spectral‐based fault localization. It tries to locate a buggy statement by applying an evaluation metric to program spectra and ranking program components on the basis of the score it computes. Here, we propose a restricted class of “hyperbolic” metrics, with a small number of numeric parameters. This class of functions is based on past theoretical and empirical results. We show that optimization methods such as genetic programming and simulated annealing can reliably discover effective metrics over a wide range of data sets of program spectra. We evaluate the performance for both real programs and model programs with single bugs, multiple bugs, “deterministic” bugs, and nondeterministic bugs and find that the proposed class of metrics performs as well as or better than the previous best‐performing metrics over a broad range of data.     

Graphs for Mining-Based Defect Localization in Multithreaded Programs

Trends in modern multicore architecture design requires software developers to develop and debug multithreaded programs. Consequently, software developers must face new challenges because of bug patterns occurring at runtime and due to the non-deterministic behavior of multi-threaded program executions. This calls for new defect-localization techniques. There has been much work in the field of defect localization for sequential programs on the one side and on the localization of specific multithreading bugs on the other side, but we are not aware of any general technique for multithreaded programs. This paper proposes such an approach. It generalizes data mining-based defect-localization techniques for sequential programs. The techniques work by analyzing call graphs. More specifically, we propose new graph representations of multithreaded program executions as well as two mining-based localization approaches based on these representations. Our evaluation shows that our technique yields good results and is able to find defects that other approaches cannot localize.     

运用变异测试的并行程序测试用例最小化算法

在并行程序测试中,测试输入和线程交互时序是影响并行错误检测的两个关键因素。以缩减并行错误检测的输入空间为目标,给出一种基于变异测试的测试用例最小化算法。首先对并行程序进行研究,选取与并行错误密切相关的9个变异算子,并以此为基础为待测程序生成多种变异体;采用JPF作为线程调度工具来执行测试用例,根据变异评分与平均时间成本对测试用例进行排序,在优化后的测试用例集中选取检测能力不重复的测试用例,从而得到面向并行错误检测的最小测试用例集。实验结果证明,该方法能有效减小测试用例集的规模,并大幅缩短运行时间,从而提高了并行程序的测试效率。