一种基于变异分析的BPEL程序故障定位技术

不同于传统C,C++或Java程序,BPEL(Business Process Execution Language)程序由一组活动及其之间的交互组成,同时引入了并发、序列化、XML表示等新特征,这些新特点使得定位BPEL程序的故障具有一定的挑战性.针对现有故障定位技术在有效性方面的不足,提出一种基于变异分析的BPEL...     

软件错误定位研究综述

错误定位是软件调试中的一个热点问题,旨在高效地检测出软件错误。首先根据研究方法的不同,将已有错误定位方法从轻量级和重量级两个角度进行分类并进行比较。轻量级错误定位技术不涉及程序依赖关系分析,在程序执行覆盖信息的基础上用统计学或数据挖掘等方法找出可疑错误代码的集合来定位错误;重量级错误定位技术涉及程序依赖关系分析,主要分析数据依赖、控制依赖关系或使用程序切片等来识别可疑代码。然后,总结了常用的评测数据集和评测标准。最后,对错误定位的未来研究趋势进行了展望。     

A test-suite reduction approach to improving fault-localization effectiveness

In order to improve the effectiveness of fault localization, researchers are interested in test-suite reduction to provide suitable test-suite inputs. Different test-suite reduction approaches have been proposed. However, the results are usually not ideal. Reducing the test-suite improperly or excessively can even negatively affect fault-localization effectiveness. In this paper, we propose a two-step test-suite reduction approach to remove the test cases which have little or no effect on fault localization, and improve the distribution evenness of concrete execution paths of test cases. This approach consists of coverage matrix based reduction and path vector based reduction, so it analyzes not only the test cases coverage but also the concrete path information. We design and implement experiments to verify the effect of our approach. The experimental results show that our reduced test-suite can improve fault-localization effectiveness. On average, our approach can reduce the size of a test-suite in 47.87% (for Siemens programs) and 23.03% (for space program). At the same time, on average our approach can improve the fault-localization effectiveness, 2.12 on Siemens programs and 0.13 on space program by Tarantula approach.     

mcfTRaptor: Toward unobtrusive on-the-fly control-flow tracing in multicores

Software testing and debugging has become the most critical aspect of the development of modern embedded systems, mainly driven by growing software and hardware complexity, increasing integration, and tightening time-to-market deadlines. Software developers increasingly rely on on-chip trace and debug infrastructure to locate software bugs faster. However, the existing infrastructure offers limited visibility or relies on hefty on-chip buffers and wide trace ports that significantly increase system cost. This paper introduces a new technique called mcfTRaptor for capturing and compressing functional and time-stamped control-flow traces on-the-fly in modern multicore systems. It relies on private on-chip predictor structures and corresponding software modules in the debugger to significantly reduce the number of events that needs to be streamed out of the target platform. Our experimental evaluation explores the effectiveness of mcfTRaptor as a function of the number of cores, encoding mechanisms, and predictor configurations. When compared to the Nexus-like control-flow tracing, mcfTRaptor reduces the trace port bandwidth in the range from 14 to 23.8 times for functional traces and 10.8–18.6 times for time-stamped traces.     

基于执行轨迹的软件缺陷定位方法研究

软件中隐含的缺陷数目与可靠性直接相关,软件缺陷定位是移除软件缺陷的关键,缺陷定位的及时性和有效性直接影响软件的可用性。基于执行轨迹的软件缺陷定位能够很好地与自动化测试相结合,有较强的现实意义。讨论了基于执行轨迹的软件缺陷定位方法通用框架FLOC,详细介绍了该框架的各个阶段,包括执行轨迹的组织、执行轨迹的选择、怀疑率的计算、定位报告的评价。分析了现有的基于执行轨迹的软件缺陷定位方法,并按照框架的结构比较了这些方法的特点,提出了改进的思路。最后对缺陷定位的发展提出展望。     

针对基于变异错误定位的一种动态变异执行策略

在软件调试过程中,如何快速、精确地定位程序中的错误代码是软件开发人员普遍关注的问题。基于变异的错误定位方法是一种通过分析被测程序与程序变异体之间的行为相似性来估计语句出错概率、进行错误定位的方法。该方法有较高的错误定位精确度,但由于需对大量程序变异体执行测试用例集,因此其变异执行开销较大。为此提出了一种动态变异执行策略,它通过搜集测试用例执行信息,动态地调整变异体及测试用例的执行顺序,以减少其变异执行开销。实验结果表明,在6个程序包的127个错误版本上,应用提出的动态变异执行策略可在保证错误定位精确度的前提下,减少23%~78%的变异执行开销,显著提高了基于变异的错误定位方法的效率。     

An empirical study on the use of mutant traces for diagnosis of faults in deployed systems

Debugging deployed systems is an arduous and time consuming task. It is often difficult to generate traces from deployed systems due to the disturbance and overhead that trace collection may cause on a system in operation. Many organizations also do not keep historical traces of failures. On the other hand earlier techniques focusing on fault diagnosis in deployed systems require a collection of passing–failing traces, in-house reproduction of faults or a historical collection of failed traces. In this paper, we investigate an alternative solution. We investigate how artificial faults, generated using software mutation in test environment, can be used to diagnose actual faults in deployed software systems. The use of traces of artificial faults can provide relief when it is not feasible to collect different kinds of traces from deployed systems. Using artificial and actual faults we also investigate the similarity of function call traces of different faults in functions. To achieve our goal, we use decision trees to build a model of traces generated from mutants and test it on faulty traces generated from actual programs. The application of our approach to various real world programs shows that mutants can indeed be used to diagnose faulty functions in the original code with approximately 60–100% accuracy on reviewing 10% or less of the code; whereas, contemporary techniques using pass–fail traces show poor results in the context of software maintenance. Our results also show that different faults in closely related functions occur with similar function call traces. The use of mutation in fault diagnosis shows promising results but the experiments also show the challenges related to using mutants.     

A mutation carol: Past, present and future

Context

The field of mutation analysis has been growing, both in the number of published papers and the number of active researchers. This special issue provides a sampling of recent advances and ideas. But do all the new researchers know where we started?

Objective

To imagine where we are going, we must first know where we are. To understand where we are, we must know where we have been. This paper reviews past mutation analysis research, considers the present, then imagines possible future directions.

Method

A retrospective study of past trends lets us the ability to see the current state of mutation research in a clear context, allowing us to imagine and then create future vectors.

Results

The value of mutation has greatly expanded since the early view of mutation as an expensive way to unit test subroutines. Our understanding of what mutation is and how it can help has become much deeper and broader.

Conclusion

Mutation analysis has been around for 35 years, but we are just now beginning to see its full potential. The papers in this issue and future mutation workshops will eventually allow us to realize this potential.     

面向对象程序蜕变关系构造方法

针对面向对象软件类级测试中蜕变关系构造不充分的问题,提出一种由错误类型指导面向对象程序蜕变关系构造的方法。首先分析类方法中包含的基本操作,根据错误发生位置和作用效果将错误分成3类;其次 按照执行效果的不同 将类中方法分成两类;再针对错误类型对每个方法分别构造蜕变关系;最后通过Rectangle类的实验对比该方法与其他方法。实验证明提出的由错误类型指导蜕变关系构造的方法的检错率有所提高,并且有助于错误定位。     

A family of code coverage-based heuristics for effective fault localization

Locating faults in a program can be very time-consuming and arduous, and therefore, there is an increased demand for automated techniques that can assist in the fault localization process. In this paper a code coverage-based method with a family of heuristics is proposed in order to prioritize suspicious code according to its likelihood of containing program bugs. Highly suspicious code (i.e., code that is more likely to contain a bug) should be examined before code that is relatively less suspicious; and in this manner programmers can identify and repair faulty code more efficiently and effectively. We also address two important issues: first, how can each additional failed test case aid in locating program faults; and second, how can each additional successful test case help in locating program faults. We propose that with respect to a piece of code, the contribution of the first failed test case that executes it in computing its likelihood of containing a bug is larger than or equal to that of the second failed test case that executes it, which in turn is larger than or equal to that of the third failed test case that executes it, and so on. This principle is also applied to the contribution provided by successful test cases that execute the piece of code. A tool, χDebug, was implemented to automate the computation of the suspiciousness of the code and the subsequent prioritization of suspicious code for locating program faults. To validate our method case studies were performed on six sets of programs: Siemens suite, Unix suite, space, grep, gzip, and make. Data collected from the studies are supportive of the above claim and also suggest Heuristics III(a), (b) and (c) of our method can effectively reduce the effort spent on fault localization.     

基于测试充分性准则的非死锁并发缺陷定位方法

并发程序的非确定性使得其调试工作异常困难。基于程序谱的软件缺陷定位方法虽然能够缓解该情况,但其定位结果依赖于调试信息。针对在此过程中难以获得调试信息及如何选择利用调试信息的问题,提出了一种基于测试充分性准则的缺陷定位方法,该方法包括3个部分:预测满足测试充分性准则的条件;制定相应的测试方案;将收集到的调试信息用于缺陷定位分析。依据此方法,用C#语言实现了缺陷定位工具——ConFinder。在含有实际并发缺陷的程序上进行实验,结果表明该方法可以有效找出引起程序失效的原因并且所得结果具有很好的稳定性。     

一种基于程序切片技术的软件测试方法

1 引言软件测试是人们发现、纠正、预防软件错误以及完善软件功能的重要手段。软件测试的目的就是为了发现程序中的错误。对于传统程序设计语言书写的软件,软件测试人员普遍接受三个级别的测试:单元测试、集成测试和系统测试。无论在哪个级别上进行测试,其测试过程均为输入测试数据、处理和验证输出结果三个步骤。目前面向对象软件开发技术发展迅速,但面向对象软件测试技术的研究还相对薄弱。例如,对面向对象的程序测试应当分为多少级尚未达成共识。基于结构的传统集成策略并不适于面向对象的程序。这是因为面向对象的程序的执行实际上是执行一个由消息连接起来的方法序列,而这个方法序列往往是由外部事件驱动的,在面向对象语言中,虽然信息隐藏和封装使得类具有较好的独立性,有利于提高软件的易测试性和保证软件的质量,但是,这些机制与继承机制和     

Error explanation with distance metrics

In the event that a system does not satisfy a specification, a model checker will typically automatically produce a counterexample trace that shows a particular instance of the undesirable behavior. Unfortunately, the important steps that follow the discovery of a counterexample are generally not automated. The user must first decide if the counterexample shows genuinely erroneous behavior or is an artifact of improper specification or abstraction. In the event that the error is real, there remains the difficult task of understanding the error well enough to isolate and modify the faulty aspects of the system. This paper describes a (semi-)automated approach for assisting users in understanding and isolating errors in ANSI C programs. The approach, derived from Lewis’ counterfactual approach to causality, is based on distance metrics for program executions. Experimental results show that the power of the model checking engine can be used to provide assistance in understanding errors and to isolate faulty portions of the source code.     

面向语句的MBFL变异体约减策略

在软件调试过程中如何高效、精确地定位程序中的错误代码是软件开发人员普遍关注的问题。MBFL是一种基于变异分析的错误定位技术,它在获得较高错误定位精度的同时会生成大量变异体,并在变异体上执行测试用例集,开销庞大。为了减少MBFL的变异执行开销,提出面向语句的变异体约减策略,通过分析测试用例的执行信息, 按一定比例 对每条由失败测试用例覆盖的语句生成的变异体集合进行约减。实验结果表明,在7个程序包的112个错误版本上,应用面向语句的变异体约减策略的MBFL,在保持较高错误定位精度的同时,能够有效减少73.51%~79.98%的变异执行开销。     

Bayesian updating of optimal release time for software systems

In this paper, a Bayesian method dealing with software reliability growth with consideration of the learning effect is proposed to determine an optimal release time for software systems with regard to the testing cost and experts’ prior judgments. Such an approach is able to devise an appropriate software-debugging scheme which has the best arrangement of available resources and personnel with a minimal software testing cost when lacking sufficient information for decision making. Past research on software reliability emphasized the estimation of the number of cumulative software errors or the software reliability with respect to a specific time period, yet it neglected the determination of software release time with consideration of the software testing cost, meaning that existing approaches are not entirely practical. Accordingly, the proposed method is concerned with the evaluation of the software testing cost incurred during the testing period based on experts’ prior judgments and the software testing data collected within a given duration, and is thus characterized by its practicality as well as meaningfulness with consideration of the learning effect. Finally, a numerical example is given to verify the effectiveness of the proposed approach, and sensitivity and risk analyses are performed on this example.

软件测试中故障模型的建立

在软件测试中,会碰到各种类型的软件故障,而且这些故障的灵活性很强。这篇文章旨在建立不同类型的故障模型,以便于管理和标准化这些故障。该文建立了6种故障模型,可以覆盖相当数量的软件故障。而且在文章的附录部分给出了试验数据,有效地证明了这些故障模型的建立是非常有意义的。     

基于程序插装的动态测试技术实现

动态测试是检测程序的动态行为的一种有效手段,本文介绍了Safepro白盒软件测试工具系列中基于程序插装的动脉测试技术实现,具体讨论了动态测试的模型、数据流模型和动态跟踪数据的编码和解码技术,插装库设计与插装策略等内容。     

结合语句执行补集的程序错误定位

基于频谱的错误定位方法一般利用覆盖信息为每条语句度量出错的可能,即可疑度,通过逐条检查按可疑度值降序排列的语句序列来确定错误语句.针对已有的方法大多只考虑覆盖信息中语句执行信息的问题,分析了语句执行补集对错误定位的积极影响,进一步提出了在语句执行信息基础上结合语句执行补集的错误定位方法.实验结果表明,与其他方法相比,所...     

环境错误注入测试机制

环境错误注入是一种动态的测试软件脆弱性的技术,建立在对系统脆弱性分类模型的基础上,通过选择一个恰当的错误模型,测试系统有意识地触发软件内部存在的安全漏洞。该测试方法对于通常使用的静态检测方法,是一种重要的补充。文章论述了环境错误注入系统的一种实现机制。     

数组越界的故障模型及其检测方法研究

数组越界是C程序中的常见故障,该类故障可能造成系统的崩溃。首先针对常见的数组越界故障进行了分析,提出了检测数组越界的判定准则,建立了故障模型。根据该故障模型,采用程序控制流图和路径条件,并结合静态分析思想,给出了可有效地检测出程序中存在的数组越界故障的方法。最后通过实例分析了该方法的应用过程。