Efficient multi-objective higher order mutation testing with genetic programming

It is said 90% of faults that survive manufacturer’s testing procedures are complex. That is, the corresponding bug fix contains multiple changes. Higher order mutation testing is used to study defect interactions and their impact on software testing for fault finding. We adopt a multi-objective Pareto optimal approach using Monte Carlo sampling, genetic algorithms and genetic programming to search for higher order mutants which are both hard-to-kill and realistic. The space of complex faults (higher order mutants) is much larger than that of traditional first order mutations which correspond to simple faults, nevertheless search based approaches make this scalable. The problems of non-determinism and efficiency are overcome. Easy to detect faults may become harder to detect when they interact and impossible to detect single faults may be brought to light when code contains two such faults. We use strong typing and BNF grammars in search based mutation testing to find examples of both in ancient heavily optimised every day C code.     

A family of code coverage-based heuristics for effective fault localization

Locating faults in a program can be very time-consuming and arduous, and therefore, there is an increased demand for automated techniques that can assist in the fault localization process. In this paper a code coverage-based method with a family of heuristics is proposed in order to prioritize suspicious code according to its likelihood of containing program bugs. Highly suspicious code (i.e., code that is more likely to contain a bug) should be examined before code that is relatively less suspicious; and in this manner programmers can identify and repair faulty code more efficiently and effectively. We also address two important issues: first, how can each additional failed test case aid in locating program faults; and second, how can each additional successful test case help in locating program faults. We propose that with respect to a piece of code, the contribution of the first failed test case that executes it in computing its likelihood of containing a bug is larger than or equal to that of the second failed test case that executes it, which in turn is larger than or equal to that of the third failed test case that executes it, and so on. This principle is also applied to the contribution provided by successful test cases that execute the piece of code. A tool, χDebug, was implemented to automate the computation of the suspiciousness of the code and the subsequent prioritization of suspicious code for locating program faults. To validate our method case studies were performed on six sets of programs: Siemens suite, Unix suite, space, grep, gzip, and make. Data collected from the studies are supportive of the above claim and also suggest Heuristics III(a), (b) and (c) of our method can effectively reduce the effort spent on fault localization.     

基于代价敏感间隔分布优化的软件缺陷定位

在大型软件项目的开发与维护中,从大量的代码文件中定位软件缺陷费时、费力,有效地进行软件缺陷自动定位,将能极大地降低开发成本.软件缺陷报告通常包含了大量未发觉的软件缺陷的信息,精确地寻找与缺陷报告相关联的代码文件,对于降低维护成本具有重要意义.目前,已有一些基于深度神经网络的缺陷定位技术相对于传统方法,其效果有所提升,但相关工作大多关注网络结构的设计,缺乏对训练过程中损失函数的研究,而损失函数对于预测任务的性能会有极大的影响.在此背景下,提出了代价敏感的间隔分布优化（cost-sensitive margin distribution optimization,简称CSMDO）损失函数,并将代价敏感的间隔分布优化层应用到深度卷积神经网络中,能够良好地处理软件缺陷数据的不平衡性,进一步提高缺陷定位的准确度.     

Recognizing authors: an examination of the consistent programmer hypothesis

Software developers have individual styles of programming. This paper empirically examines the validity of the consistent programmer hypothesis: that a facet or set of facets exist that can be used to recognize the author of a given program based on programming style. The paper further postulates that the programming style means that different test strategies work better for some programmers (or programming styles) than for others. For example, all‐edges adequate tests may detect faults for programs written by Programmer A better than for those written by Programmer B. This has several useful applications: to help detect plagiarism/copyright violation of source code, to help improve the practical application of software testing, and to help pursue specific rogue programmers of malicious code and source code viruses. This paper investigates this concept by experimentally examining whether particular facets of the program can be used to identify programmers and whether testing strategies can be reasonably associated with specific programmers. Copyright © 2009 John Wiley & Sons, Ltd.     

方法级别的细粒度软件缺陷定位方法

当软件缺陷报告在跟踪系统中被指派给开发人员进行缺陷修复之后,缺陷修复人员就需要根据提交的缺陷报告来进行软件缺陷定位,并做出相应的代码变更,以修复该软件缺陷.在缺陷修复的整个过程中,软件缺陷定位占用了开发人员大量的时间.提出了一种方法级别的细粒度软件缺陷定位方法MethodLocator,以提高软件修复人员的工作效率.MethodLocator首先对缺陷报告和源代码方法体利用词向量（word2vec）和TF-IDF结合的方法进行向量表示;然后,根据源代码文件中方法体之间的相似度对方法体进行扩充;最后,通过对扩充后的方法体和缺陷报告计算其余弦距离并排序,来定位为修复软件缺陷所需做出变更的方法.在4个开源软件项目ArgoUML、Ant、Maven和Kylin上的实验结果表明,MethodLocator方法优于现有的缺陷定位方法,它能够有效地将软件缺陷定位到源代码的方法级别上.     

数据流相关软件故障的静态检测

建立了数据流相关的软件故障模型,对应用程序中由于变量的定值与引用操作及动态内存访问过程中导致故障发生的情况进行了分析．该类故障的检测需要数据流分析的支持．基于程序控制结构的精确数据流分析方法,充分考虑了不同程序路径上变量的定值情况,在静态分析过程中模拟了动态执行过程中到达各引用点的定值信息．根据引用-定值链建立的故障诊断规则,为有效地进行故障检测提供了依据．     

基于信息检索的缺陷定位:问题、进展与挑战

缺陷的存在,会影响软件系统的正常使用甚至带来重大危害.为了帮助开发者尽快找到并修复这些缺陷,研究者提出了基于信息检索的缺陷定位方法.这类方法将缺陷定位视为一个检索任务,它为每个缺陷报告生成一份按照程序实体与缺陷相关度降序排序的列表.开发者可以根据列表顺序来审查代码,从而降低审查成本并加速缺陷定位的进程.近年来,该领域的研究工作十分活跃,在改良定位方法和完善评价体系方面取得了较大进展.与此同时,为了能够在实践中更好地应用这类方法,该领域的研究工作仍面临着一些亟待解决的挑战.对近年来国内外学者在该领域的研究成果进行系统性的总结：首先,描述了基于信息检索的缺陷定位方法的研究问题;然后,分别从模型改良和模型评估两方面陈述了相关的研究进展,并对具体的理论和技术途径进行梳理;接着,简要介绍了缺陷定位的其他相关技术;最后,总结了目前该领域研究过程中面临的挑战并给出建议的研究方向.     

Introduction of static quality analysis in small- and medium-sized software enterprises: experiences from technology transfer

Today, small- and medium-sized enterprises (SMEs) in the software industry face major challenges. Their resource constraints require high efficiency in development. Furthermore, quality assurance (QA) measures need to be taken to mitigate the risk of additional, expensive effort for bug fixes or compensations. Automated static analysis (ASA) can reduce this risk because it promises low application effort. SMEs seem to take little advantage of this opportunity. Instead, they still mainly rely on the dynamic analysis approach of software testing. In this article, we report on our experiences from a technology transfer project. Our aim was to evaluate the results static analysis can provide for SMEs as well as the problems that occur when introducing and using static analysis in SMEs. We analysed five software projects from five collaborating SMEs using three different ASA techniques: code clone detection, bug pattern detection and architecture conformance analysis. Following the analysis, we applied a quality model to aggregate and evaluate the results. Our study shows that the effort required to introduce ASA techniques in SMEs is small (mostly below one person-hour each). Furthermore, we encountered only few technical problems. By means of the analyses, we could detect multiple defects in production code. The participating companies perceived the analysis results to be a helpful addition to their current QA and will include the analyses in their QA process. With the help of the Quamoco quality model, we could efficiently aggregate and rate static analysis results. However, we also encountered a partial mismatch with the opinions of the SMEs. We conclude that ASA and quality models can be a valuable and affordable addition to the QA process of SMEs.     

基于Mozilla的安全性漏洞再修复经验研究

相较于其他类型的漏洞,安全性漏洞更容易发生再修复,这使得安全性漏洞需要更多的开发资源,从而增加了这些安全性漏洞修复的成本。因此,减少安全性漏洞再修复的发生的重要性不言而喻。对安全性漏洞再修复的经验研究有助于减少再修复的发生。首先,通过对Mozilla工程中一些发生再修复的安全性漏洞的安全性漏洞类型、发生再修复的原因、再修复的次数、修改的提交数、修改的文件数、修改的代码行数的增减、初始修复和再修复的对比等数据进行分析,发现了安全性漏洞发生再修复是普遍存在的,且与漏洞发生原因的识别的复杂程度和漏洞修复的复杂程度这两个因素有关;其次,初始修复涉及的文件、代码的集中程度是影响再修复的原因之一,而使用更复杂、更有效的修复过程可有效避免再修复的发生;最后,总结了几种安全性漏洞发生再修复的原因,使开发人员有效地识别不同类型的安全性漏洞再修复。     

基于变量访问序模式的中断数据竞争检测方法

在航天嵌入式软件等中断驱动型软件中,中断数据竞争问题十分突出.然而中断在并发语义、同步机制、调度机制等方面与线程（任务）有诸多不同,具有Ad-hoc特征,难以统一刻画,因此主流的数据竞争检测方法并不适用.以航天嵌入式软件数据竞争案例库为基础进行了系统分析,提出刻画有害中断数据竞争的7种缺陷模式.针对其中最常见且最难解决的单变量访问序模式,基于抽象解释提出一种支持过程间分析、中断并发分析的高效检测方法.设计并实现了相应的检测工具SpaceDRC.实验表明,SpaceDRC能够在145毫秒内检测出约21400行程序中的真实数据竞争.SpaceDRC已经在多个航天重点型号中进行了应用,使得中断数据竞争专项分析的效率提高了至少5倍,并且降低了问题遗漏率.     

代码变更中抽取类重构模式的识别

在现代软件开发和维护中,重构是提高软件可维护性和软件质量的常用手段.而大量重构模式掺杂在日常的bug修复、功能增加等代码变更中,使得变更理解变得非常复杂.识别重构模式可以将重构与其它类型的代码变更隔离,利于变更理解.目前在识别重构模式的相关研究中,并没有结合变更类型和相似性比较的识别重构模式的方法及工具.为此,提出了一种基于细粒度变更类型和文本相似性比较识别重构模式的方法.将该方法应用于抽取类重构模式,并在4个开源项目中进行了实验,其平均准确率在82.6%左右.     

Collaborative eye tracking based code review through real-time shared gaze visualization

Code review is intended to find bugs in early development phases, improving code quality for later integration and testing. However, due to the lack of experience with algorithm design, or software development, individual novice programmers face challenges while reviewing code. In this paper, we utilize collaborative eye tracking to record the gaze data from multiple reviewers, and share the gaze visualization among them during the code review process. The visualizations, such as borders highlighting current reviewed code lines, transition lines connecting related reviewed code lines, reveal the visual attention about program functions that can facilitate understanding and bug tracing. This can help novice reviewers to make sense to confirm the potential bugs or avoid repeated reviewing of code, and potentially even help to improve reviewing skills. We built a prototype system, and conducted a user study with paired reviewers. The results showed that the shared real-time visualization allowed the reviewers to find bugs more efficiently.     

Improved code defect detection with fault links

Fault links represent relationships between the types of code faults, or defects, and the types of components in which faults are detected. For example, our prior work validated that a fault link exists between Controller components and Control/Logic faults (such as unreachable code). Fault link information can guide code reviews, walkthroughs, testing, maintenance, and can advise fault seeding. In this paper, we use fault links to augment code reviews. Two experiments were undertaken to evaluate the usefulness of fault links, one with 26 Computer Science students and another with 24 software engineering professionals. The first experiment showed that fault link information assisted in finding more total defects and more ‘hard to detect’ defects, in the same amount of time, in a Java component of an online course management application. The experiment was repeated with professionals, adding a second Java component from the same application. For the second experiment, more total defects were found by the participants using fault link information for one of the two components and more hard to detect defects were found, in the same amount of time, in both Java components. The group using fault link information for code walkthroughs found, on average, 1.7–2 times more faults and 2–3 times more hard faults than the control group. Copyright © 2010 John Wiley & Sons, Ltd.     

操作系统内核并发错误检测研究进展

并发错误是程序设计语言和软件工程领域的研究热点之一.近年来,针对应用程序并发错误检测的研究已取得了很大进展.但是由于操作系统内核的并发和同步机制复杂、代码规模庞大,与应用程序级并发错误检测相比,操作系统内核的并发错误检测研究仍面临巨大的挑战.对此,国内外学者提出了各种用于操作系统内核并发错误检测的方法.首先介绍了并发错...     

基于缺陷修复历史的两阶段缺陷定位方法

缺陷定位是软件缺陷修复的关键步骤。随着计算机软件的日趋复杂和网络的迅速发展,如何快速高效的定位缺陷相关代码成为了一个急待解决的问题。在研究现有基于信息检索技术的缺陷定位方法的基础上,综合考虑缺陷修复历史信息,提出了基于缺陷修复历史的两阶段缺陷定位方法。该方法不再单一依赖文本相似度,从缺陷修复的局部性现象入手,更多的考虑了缺陷修复的历史记录、变更信息及代码特征等因素,结合信息检索和缺陷预测方法来提高缺陷定位的精度。最后本文以两个开源项目为例,验证了方法的可行性和有效性。     

一种基于Linux平台的嵌入式软件测试工具

详述了软件测试工具K7在Linux环境下对嵌入式软件进行代码测试的过程。结果表明,测试嵌入式Linux程序时,使用该工具能使被测程序从目标平台脱离,在宿主机RedHat9环境下完成代码静态分析,从而有效降低了嵌入式软件测试的复杂度,能够帮助程序员迅速查找软件的设计缺陷,提高软件可靠性。     

面向细粒度源代码变更的缺陷预测方法

软件在其生命周期中不断地发生变更,以适应需求和环境的变化。为了及时预测每次变更是否引入了缺陷,研究者们提出了面向软件源代码变更的缺陷预测方法。然而现有方法存在以下3点不足：(1)仅实现了较粗粒度(事务级和源文件级变更)的预测;(2)仅采用向量空间模型表征变更,没有充分挖掘蕴藏在软件库中的程序结构、自然语言语义以及历史等信息;(3)仅探讨较短时间范围内的预测,未考虑在长时间软件演化过程中由于新需求或人员重组等外界因素所带来的概念漂移问题。针对现有的不足,提出一种面向源代码变更的缺陷预测方法。该方法将细粒度(语句级)变更作为预测对象,从而有效降低了质量保证成本;采用程序静态分析和自然语言语义主题推断相结合的技术深入挖掘软件库,从变更的上下文、内容、时间以及人员4个方面构建特征集,从而揭示了变更易于引入缺陷的因素;采用特征熵差值矩阵分析了软件演化过程中概念漂移问题的特点,并通过一种伴随概念回顾的动态窗口学习机制实现了长时间的稳定预测。通过6个著名开源软件验证了该方法的有效性。     

Automatic mining of source code repositories to improve bug finding techniques

We describe a method to use the source code change history of a software project to drive and help to refine the search for bugs. Based on the data retrieved from the source code repository, we implement a static source code checker that searches for a commonly fixed bug and uses information automatically mined from the source code repository to refine its results. By applying our tool, we have identified a total of 178 warnings that are likely bugs in the Apache Web server source code and a total of 546 warnings that are likely bugs in Wine, an open-source implementation of the Windows API. We show that our technique is more effective than the same static analysis that does not use historical data from the source code repository.     

Semantic metrics for software testability

Software faults that infrequently affect output cause problems in most software and are dangerous in safety-critical systems. When a software fault causes frequent software failures, testing is likely to reveal the fault before the software is released; when the fault “hides” from testing, the hidden fault can cause disaster after the software is installed. During the design of safety-critical software, we can isolate certain subfunctions of the software that tend to hide faults. A simple metric, derivable from semantic information found in software specifications, indicates software subfunctions that tend to hide faults. The metric is the domain/range ratio (DRR): the ratio of the cardinality of the possible inputs to the cardinality of the possible outputs. By isolating modules that implement a high DRR function during design, we can produce programs that are less likely to hide faults during testing. The DRR is available early in the software lifecycle; when code has been produced, the potential for hidden faults can be further explored using empirical methods. Using the DRR during design and empirical methods during execution, we can better plan and implement strategies for enhancing testability. For certain specifications, testability considerations can help produce modules that require less additional testing when assumptions change about the distribution of inputs. Such modules are good candidates for software reuse.     

Testing scientific software: A systematic literature review

ContextScientific software plays an important role in critical decision making, for example making weather predictions based on climate models, and computation of evidence for research publications. Recently, scientists have had to retract publications due to errors caused by software faults. Systematic testing can identify such faults in code.ObjectiveThis study aims to identify specific challenges, proposed solutions, and unsolved problems faced when testing scientific software.MethodWe conducted a systematic literature survey to identify and analyze relevant literature. We identified 62 studies that provided relevant information about testing scientific software.ResultsWe found that challenges faced when testing scientific software fall into two main categories: (1) testing challenges that occur due to characteristics of scientific software such as oracle problems and (2) testing challenges that occur due to cultural differences between scientists and the software engineering community such as viewing the code and the model that it implements as inseparable entities. In addition, we identified methods to potentially overcome these challenges and their limitations. Finally we describe unsolved challenges and how software engineering researchers and practitioners can help to overcome them.ConclusionsScientific software presents special challenges for testing. Specifically, cultural differences between scientist developers and software engineers, along with the characteristics of the scientific software make testing more difficult. Existing techniques such as code clone detection can help to improve the testing process. Software engineers should consider special challenges posed by scientific software such as oracle problems when developing testing techniques.