Understanding the triaging and fixing processes of long lived bugs

ContextBug fixing is an integral part of software development and maintenance. A large number of bugs often indicate poor software quality, since buggy behavior not only causes failures that may be costly but also has a detrimental effect on the user’s overall experience with the software product. The impact of long lived bugs can be even more critical since experiencing the same bug version after version can be particularly frustrating for user. While there are many studies that investigate factors affecting bug fixing time for entire bug repositories, to the best of our knowledge, none of these studies investigates the extent and reasons of long lived bugs.ObjectiveIn this paper, we investigate the triaging and fixing processes of long lived bugs so that we can identify the reasons for delay and improve the overall bug fixing process.MethodologyWe mine the bug repositories of popular open source projects, and analyze long lived bugs from five different perspectives: their proportion, severity, assignment, reasons, as well as the nature of fixes.ResultsOur study on seven open-source projects shows that there are a considerable number of long lived bugs in each system and over 90% of them adversely affect the user’s experience. The reasons for these long lived bugs are diverse including long assignment time, not understanding their importance in advance, etc. However, many bug-fixes were delayed without any specific reasons. Furthermore, 40% of long lived bugs need only small fixes.ConclusionOur overall results suggest that a significant number of long lived bugs may be minimized through careful triaging and prioritization if developers could predict their severity, change effort, and change impact in advance. We believe our results will help both developers and researchers better to understand factors behind delays, improve the overall bug fixing process, and investigate analytical approaches for prioritizing bugs based on bug severity as well as expected bug fixing effort.     

重复软件缺陷报告检测方法综述

软件缺陷在软件的开发和维护过程中是不可避免的,软件缺陷报告是软件维护过程中重要的缺陷描述文档,高质量的软件缺陷报告可以有效提高软件缺陷修复的效率.然而,由于存在许多开发人员、测试人员和用户与缺陷跟踪系统交互并提交软件缺陷报告,同一个软件缺陷可能被不同的人员报告,导致了大量重复的软件缺陷报告.重复的软件缺陷报告势必加重人工检测重复缺陷报告的工作量,并造成人力物力的浪费,降低了软件缺陷修复的效率.以系统文献调研的方式,对近年来国内外学者在重复软件缺陷报告检测领域的研究工作进行了系统的分析.主要从研究方法、数据集的选取、性能评价等方面具体分析总结,并提出该领域在后续研究中存在的问题、挑战以及建议.     

Developer Activity Motivated Bug Triaging: Via Convolutional Neural Network

Neural Processing Letters - As bugs become prevalent in software development, bug triaging has become one of the most important activities in software maintenance. To decrease the time cost in...     

基于循环神经网络的缺陷报告分派方法

随着开源软件项目规模的不断增大,人工为缺陷报告分派合适的开发人员（缺陷分派）变得越来越困难.而不合适的缺陷分派往往会严重影响缺陷修复的效率,为此迫切需要一种缺陷分派辅助技术帮助项目管理者更好地完成缺陷分派任务.当前,大部分研究工作都基于缺陷报告文本以及相关元数据信息分析来刻画开发者的特征,忽略了对开发者活跃度的考虑,使得对具有相似特征的开发者进行缺陷报告分派预测时表现较差.本文提出了一个基于循环神经网络的深度学习模型DeepTriage,一方面利用双向循环网络加池化方法提取缺陷报告的文本特征,一方面利用单向循环网络提取特定时刻的开发者活跃度特征,并融合两者,利用已修复的缺陷报告进行监督学习.在Eclipse等四个不同的开源项目数据集上的实验结果表明,DeepTriage较同类工作在缺陷分派预测准确率上有显著提升.     

Studying re-opened bugs in open source software

Bug fixing accounts for a large amount of the software maintenance resources. Generally, bugs are reported, fixed, verified and closed. However, in some cases bugs have to be re-opened. Re-opened bugs increase maintenance costs, degrade the overall user-perceived quality of the software and lead to unnecessary rework by busy practitioners. In this paper, we study and predict re-opened bugs through a case study on three large open source projects—namely Eclipse, Apache and OpenOffice. We structure our study along four dimensions: (1) the work habits dimension (e.g., the weekday on which the bug was initially closed), (2) the bug report dimension (e.g., the component in which the bug was found) (3) the bug fix dimension (e.g., the amount of time it took to perform the initial fix) and (4) the team dimension (e.g., the experience of the bug fixer). We build decision trees using the aforementioned factors that aim to predict re-opened bugs. We perform top node analysis to determine which factors are the most important indicators of whether or not a bug will be re-opened. Our study shows that the comment text and last status of the bug when it is initially closed are the most important factors related to whether or not a bug will be re-opened. Using a combination of these dimensions, we can build explainable prediction models that can achieve a precision between 52.1–78.6 % and a recall in the range of 70.5–94.1 % when predicting whether a bug will be re-opened. We find that the factors that best indicate which bugs might be re-opened vary based on the project. The comment text is the most important factor for the Eclipse and OpenOffice projects, while the last status is the most important one for Apache. These factors should be closely examined in order to reduce maintenance cost due to re-opened bugs.     

Firefox缺陷跟踪系统中的用户反馈

缺陷追踪是软件项目管理的一个重要环节，是保证现代大规模开源软件开发顺利进行并持续提高软件质量的必要手段.目前，大部分开源软件都使用开放的缺陷跟踪系统进行软件缺陷的管理.它允许用户向开发者提交系统故障（即defect类型缺陷）以及系统改进建议（即enhancement类型缺陷），但是这些用户的反馈所起的作用尚未得到充分研究.针对这一问题，对Firefox的缺陷跟踪系统进行实证研究，收集了2018年和2019年提交的19 474份Firefox Desktop以及3 057份Firefox for Android缺陷报告.在此基础上，对比分析了普通用户和核心开发者提交的缺陷在数量、严重性、组件分布、修复率、修复速度以及修复者上的差别，并调查了缺陷报告的撰写质量与缺陷处理结果和修复时间的关系.主要发现包括：（1）当前缺陷追踪系统中普通用户人数众多，但参与程度较浅，86%的用户只提交过一个缺陷，其中，高严重等级的缺陷不超过3%；（2）普通用户提交的缺陷主要分布在和用户交互相关的UI组件上（例如地址栏、音频/视频等），然而还有43%的缺陷由于缺乏充分描述信息而难以准确地定位到具体的关联组件；（3）在缺陷处理结果上，由于查重系统以及缺陷填报系统在设计上过于简单，致使普通用户提交的大量缺陷被处理为“无用”缺陷，缺陷修复率低于10%；（4）在缺陷修复流程上，由于普通用户难以准确、充分地描述缺陷，导致系统对其重视程度不足，普通用户提交缺陷的处理流程也比核心开发者提交的复杂，平均需要多花至少8天的时间进行修复.上述研究结果揭示了当前缺陷追踪系统在用户参与激励机制、缺陷自动查重以及缺陷报告填写智能辅助等方面的不足，能够为缺陷跟踪系统开发者和管理者改进系统、提高普通用户对开源软件的贡献提供参考.     

A contextual approach towards more accurate duplicate bug report detection and ranking

The issue-tracking systems used by software projects contain issues, bugs, or tickets written by a wide variety of bug reporters, with different levels of training and knowledge about the system under development. Typically, reporters lack the skills and/or time to search the issue-tracking system for similar issues already reported. As a result, many reports end up referring to the same issue, which effectively makes the bug-report triaging process time consuming and error prone. Many researchers have approached the bug-deduplication problem using off-the-shelf information-retrieval (IR) tools. In this work, we extend the state of the art by investigating how contextual information about software-quality attributes, software-architecture terms, and system-development topics can be exploited to improve bug deduplication. We demonstrate the effectiveness of our contextual bug-deduplication method at ranking duplicates on the bug repositories of the Android, Eclipse, Mozilla, and OpenOffice software systems. Based on this experience, we conclude that taking into account domain-specific context can improve IR methods for bug deduplication.     

Bug Triaging Based on Tossing Sequence Modeling

Bug triaging, which routes the bug reports to potential fixers, is an integral step in software development and maintenance. To make bug triaging more efficient, many researchers propose to adopt machine learning and information retrieval techniques to identify some suitable fixers for a given bug report. However, none of the existing proposals simultaneously take into account the following three aspects that matter for the efficiency of bug triaging:1) the textual content in the bug reports, 2) the metadata in the bug reports, and 3) the tossing sequence of the bug reports. To simultaneously make use of the above three aspects, we propose iTriage which first adopts a sequence-to-sequence model to jointly learn the features of textual content and tossing sequence, and then uses a classification model to integrate the features from textual content, metadata, and tossing sequence. Evaluation results on three different open-source projects show that the proposed approach has significantly improved the accuracy of bug triaging compared with the state-of-the-art approaches.     

Bug Prioritization to Facilitate Bug Report Triage

The large number of new bug reports received in bug repositories of software systems makes their management a challenging task.Handling these reports manually is time consuming,and often results in delaying the resolution of important bugs.To address this issue,a recommender may be developed which automatically prioritizes the new bug reports.In this paper,we propose and evaluate a classification based approach to build such a recommender.We use the Na¨ ve Bayes and Support Vector Machine (SVM) classifiers,and present a comparison to evaluate which classifier performs better in terms of accuracy.Since a bug report contains both categorical and text features,another evaluation we perform is to determine the combination of features that better determines the priority of a bug.To evaluate the bug priority recommender,we use precision and recall measures and also propose two new measures,Nearest False Negatives (NFN) and Nearest False Positives (NFP),which provide insight into the results produced by precision and recall.Our findings are that the results of SVM are better than the Na¨ ve Bayes algorithm for text features,whereas for categorical features,Na¨ ve Bayes performance is better than SVM.The highest accuracy is achieved with SVM when categorical and text features are combined for training.     

面向软件安全性缺陷的开发者推荐方法

软件开发与维护过程中常会出现一些安全性缺陷,这些安全性缺陷会给软件和用户带来很大的风险.安全性缺陷在修复过程中,其修复级别和质量要求往往高于一般性的缺陷,因此,推荐出富有安全性经验的开发者及时有效地修复这些安全性缺陷非常重要.现有的开发者推荐技术在推荐开发者时仅仅考虑了开发者的历史开发内容,很少考虑到开发人员的安全性缺陷修复经验和修复质量等因素,所以这些技术不适用于安全性缺陷的开发者推荐.本文针对安全性缺陷的修复提出了一种有效的软件开发者推荐方法SecDR.SecDR在推荐开发者时不仅考虑了开发者的历史开发内容（与安全性相关）,还分析了开发者的修复质量和历史修复缺陷的复杂度等因素.此外,SecDR还实现了开发者的多经验级别推荐：推荐初级开发者修复简单的安全性缺陷,高级开发者修复复杂的安全性缺陷.本文在三个开源项目（Mozilla,Libgdx,ElasticSearch）上分别对SecDR推荐开发者进行有效性验证.通过对比实验证明,SecDR针对安全性缺陷推荐开发者相比于其他方法（如：DR_PSF）的推荐精度平均高出19%~42%.另外,实验对比了SecDR与实际开发人员的分配情况,结果显示SecDR可以更好地规避不合理的软件开发者的推荐.     

Automated bug assignment: Ensemble-based machine learning in large scale industrial contexts

Bug report assignment is an important part of software maintenance. In particular, incorrect assignments of bug reports to development teams can be very expensive in large software development projects. Several studies propose automating bug assignment techniques using machine learning in open source software contexts, but no study exists for large-scale proprietary projects in industry. The goal of this study is to evaluate automated bug assignment techniques that are based on machine learning classification. In particular, we study the state-of-the-art ensemble learner Stacked Generalization (SG) that combines several classifiers. We collect more than 50,000 bug reports from five development projects from two companies in different domains. We implement automated bug assignment and evaluate the performance in a set of controlled experiments. We show that SG scales to large scale industrial application and that it outperforms the use of individual classifiers for bug assignment, reaching prediction accuracies from 50 % to 89 % when large training sets are used. In addition, we show how old training data can decrease the prediction accuracy of bug assignment. We advice industry to use SG for bug assignment in proprietary contexts, using at least 2,000 bug reports for training. Finally, we highlight the importance of not solely relying on results from cross-validation when evaluating automated bug assignment.     

BUTTER：一种基于主题模型和异构网络的缺陷分发方法

当软件缺陷被提交到缺陷跟踪系统并经过确认之后,它会被分发给开发人员进行缺陷修复．这个过程就叫做缺陷分发．随着被提交到系统的缺陷报告日益增多,手工分发缺陷报告会变得越来越困难．提出了一种自动分发缺陷的方法BUTTER．与其他方法不同的是,BUTTER不仅利用主题模型分析缺陷报告中的文本信息,而且创新性地建立了一个包含提交者、缺陷和开发者三种节点及其相互关系的异构网络,从该异构网络中抽取了更多的结构信息．实验证明,BUTTER进行自动缺陷分发较其他缺陷自动分发方法要好．     

开源软件缺陷报告自动摘要研究综述

在开源软件开发的维护阶段, 开源软件缺陷报告为开发人员解决缺陷提供了大量帮助。然而, 开源软件缺陷报告通常是以用户对话的形式编写, 一个软件缺陷报告可能含有数十条评论和上千个句子, 导致开发人员难以阅读或理解软件缺陷报告。为了缓解这个问题, 人们提出了开源软件缺陷报告自动摘要, 缺陷报告自动摘要可以减少开发人员阅读冗长缺陷报告的时间。本文以综述的方式对开源软件缺陷报告自动摘要的研究做了系统的归纳总结。首先, 根据摘要的表现形式, 将开源软件缺陷报告摘要分类为固定缺陷报告摘要和可视化缺陷报告摘要, 再将固定缺陷报告摘要研究方法分类为基于监督学习方法和基于无监督学习方法, 之后总结了基于监督学习和无监督学习的开源软件缺陷报告摘要生成的工作框架, 并介绍了开源软件缺陷报告摘要领域常用数据集、预处理技术和摘要评估指标。其次, 本文以无监督学习为切入点, 分类阐述和归纳了无监督开源软件缺陷报告摘要方法, 将无监督开源软件缺陷报告摘要方法分类为: 基于特征评分方法、基于深度学习方法、基于图方法和基于启发式方法, 并对每类方法进行讨论与分析。再次, 从缺陷报告摘要的实用性出发, 对现有的缺陷报告可视化摘要研究成果进行总结,并对固定缺陷报告摘要和可视化缺陷报告摘要的实用性做出分析。最后, 对现有研究成果及综述进行讨论和分析, 指出了开源软件缺陷报告摘要领域在缺陷报告数据集、抽取式摘要和黄金标准摘要三个方面面临的挑战和对未来研究的展望。     

High-Impact Bug Report Identification with Imbalanced Learning Strategies

In practice, some bugs have more impact than others and thus deserve more immediate attention. Due to tight schedule and limited human resources, developers may not have enough time to inspect all bugs. Thus, they often concentrate on bugs that are highly impactful. In the literature, high-impact bugs are used to refer to the bugs which appear at unexpected time or locations and bring more unexpected effects (i.e., surprise bugs), or break pre-existing functionalities and destroy the user experience (i.e., breakage bugs). Unfortunately, identifying high-impact bugs from thousands of bug reports in a bug tracking system is not an easy feat. Thus, an automated technique that can identify high-impact bug reports can help developers to be aware of them early, rectify them quickly, and minimize the damages they cause. Considering that only a small proportion of bugs are high-impact bugs, the identification of high-impact bug reports is a difficult task. In this paper, we propose an approach to identify high-impact bug reports by leveraging imbalanced learning strategies. We investigate the effectiveness of various variants, each of which combines one particular imbalanced learning strategy and one particular classification algorithm. In particular, we choose four widely used strategies for dealing with imbalanced data and four state-of-the-art text classification algorithms to conduct experiments on four datasets from four different open source projects. We mainly perform an analytical study on two types of high-impact bugs, i.e., surprise bugs and breakage bugs. The results show that different variants have different performances, and the best performing variants SMOTE (synthetic minority over-sampling technique) + KNN (K-nearest neighbours) for surprise bug identification and RUS (random under-sampling) + NB (naive Bayes) for breakage bug identification outperform the F1-scores of the two state-of-the-art approaches by Thung et al. and Garcia and Shihab.     

基于软件测试的缺陷跟踪技术的研究

本文阐述了如何应用缺陷生命周期控制法解决软件测试中发现的缺陷,并对缺陷进行跟踪管理,为决策者提供支持,提高软件开发的质量。并着重阐述了缺陷跟踪管理中的缺陷状态流转技术,缺陷管理流程,以及基于软件测试的缺陷跟踪技术,通过这样的跟踪技术可以使软件测试更加有效,可以尽早发发现缺陷,减少后期的维护工作。     

基于软件测试的缺陷跟踪技术的研究

本文阐述了如何应用缺陷生命周期控制法解决软件测试中发现的缺陷,并对缺陷进行跟踪管理,为决策者提供支持,提高软件开发的质量.并着重阐述了缺陷跟踪管理中的缺陷状态流转技术,缺陷管理流程,以及基于软件测试的缺陷跟踪技术,通过这样的跟踪技术可以使软件测试更加有效,可以尽早发发现缺陷,减少后期的维护工作.     

软件缺陷报告严重性属性分析

软件缺陷报告的严重性对缺陷的解决具有关键作用。随着软件规模的不断扩大,使用开源的软件缺陷跟踪系统成为海量缺陷信息数据的主要处理方法。分析缺陷报告严重性在数据仓库中的作用,是处理软件缺陷的重要内容。通过对Bugzilla缺陷跟踪系统数据的研究和分析,发现不同项目的属性特征差异较大,同时在修复率、解决时长、开发者、组件等属性上的统计特征具有一致性。对Mozilla项目和Eclipse项目的数据进行系统分析,并根据不同组件和项目中严重性程度分布情况,认为软件缺陷报告严重性程度的提升会导致缺陷修复率的提高,同时严重性程度为normal级别的缺陷解决时长最短,开发者持有缺陷的数量越高其修复率越低。     

缺陷报告质量研究综述

在软件开发和维护过程中,缺陷修复人员通常根据由终端用户或者开发/测试者提交的缺陷报告来定位和修复缺陷.因此,缺陷报告本身的质量对修复人员能否快速准确定位并修复缺陷具有重要的作用.围绕缺陷报告质量的刻画及改进,研究人员开展了大量的研究工作,但尚未进行系统性的归纳.旨在对这些工作进行系统性地梳理,展示该领域的研究现状并为未来的研究方向提供参考意见.首先,总结了已有缺陷报告存在的质量问题,如关键信息缺失、信息错误等;接着,总结了对缺陷报告质量进行自动化建模的技术;然后,描述了一系列对缺陷报告质量进行改进的方法;最后,对未来研究可能面临的挑战和机遇进行了展望.     

Virtual organizational learning in open source software development projects

We studied virtual organizational learning in open source software (OSS) development projects. Specifically, our research focused on learning effects of OSS projects and the factors that affect the learning process. The number and percentage of resolved bugs and bug resolution time of 118 SourceForge.net OSS projects were used to measure the learning effects. Projects were characterized by project type, number and experience of developers, number of bugs, and bug resolution time. Our results provided evidence of virtual organizational learning in OSS development projects and support for several factors as determinants of performance. Team size was a significant predictor, with mid-sized project teams functioning best. Teams of three to seven developers exhibited the highest efficiency over time and teams of eight to 15 produced the lowest mean time for bug resolution. Increasing the percentage of bugs assigned to specific developers or boosting developer participation in other OSS projects also improved performance. Furthermore, project type introduced variability in project team performance.     

浅析软件缺陷管理及工具的选择

软件缺陷是软件开发过程中不可避免的,因此有效的缺陷管理成为过程管理的重要环节,而且开发过程中产生缺陷的数量又非常多,因此选择合适项目本身的缺陷管理工具就显得尤为重要.通过对缺陷产生的原因和工具选择方法,使我们了解什么是软件缺陷,以及软件缺陷管理和使用管理工具的好处.