Firefox缺陷跟踪系统中的用户反馈

缺陷追踪是软件项目管理的一个重要环节，是保证现代大规模开源软件开发顺利进行并持续提高软件质量的必要手段.目前，大部分开源软件都使用开放的缺陷跟踪系统进行软件缺陷的管理.它允许用户向开发者提交系统故障（即defect类型缺陷）以及系统改进建议（即enhancement类型缺陷），但是这些用户的反馈所起的作用尚未得到充分研究.针对这一问题，对Firefox的缺陷跟踪系统进行实证研究，收集了2018年和2019年提交的19 474份Firefox Desktop以及3 057份Firefox for Android缺陷报告.在此基础上，对比分析了普通用户和核心开发者提交的缺陷在数量、严重性、组件分布、修复率、修复速度以及修复者上的差别，并调查了缺陷报告的撰写质量与缺陷处理结果和修复时间的关系.主要发现包括：（1）当前缺陷追踪系统中普通用户人数众多，但参与程度较浅，86%的用户只提交过一个缺陷，其中，高严重等级的缺陷不超过3%；（2）普通用户提交的缺陷主要分布在和用户交互相关的UI组件上（例如地址栏、音频/视频等），然而还有43%的缺陷由于缺乏充分描述信息而难以准确地定位到具体的关联组件；（3）在缺陷处理结果上，由于查重系统以及缺陷填报系统在设计上过于简单，致使普通用户提交的大量缺陷被处理为“无用”缺陷，缺陷修复率低于10%；（4）在缺陷修复流程上，由于普通用户难以准确、充分地描述缺陷，导致系统对其重视程度不足，普通用户提交缺陷的处理流程也比核心开发者提交的复杂，平均需要多花至少8天的时间进行修复.上述研究结果揭示了当前缺陷追踪系统在用户参与激励机制、缺陷自动查重以及缺陷报告填写智能辅助等方面的不足，能够为缺陷跟踪系统开发者和管理者改进系统、提高普通用户对开源软件的贡献提供参考.     

Experience report: investigating bug fixes in machine learning frameworks/libraries

Machine learning (ML) techniques and algorithms have been successfully and widely used in various areas including software engineering tasks. Like other software projects, bugs are also common in ML projects and libraries. In order to more deeply understand the features related to bug fixing in ML projects, we conduct an empirical study with 939 bugs from five ML projects by manually examining the bug categories, fixing patterns, fixing scale, fixing duration, and types of maintenance. The results show that (1) there are commonly seven types of bugs in ML programs; (2) twelve fixing patterns are typically used to fix the bugs in ML programs; (3) 68.80% of the patches belong to micro-scale-fix and small-scale-fix; (4) 66.77% of the bugs in ML programs can be fixed within one month; (5) 45.90% of the bug fixes belong to corrective activity from the perspective of software maintenance. Moreover, we perform a questionnaire survey and send them to developers or users of ML projects to validate the results in our empirical study. The results of our empirical study are basically consistent with the feedback from developers. The findings from the empirical study provide useful guidance and insights for developers and users to effectively detect and fix bugs in MLprojects.     

BUTTER：一种基于主题模型和异构网络的缺陷分发方法

当软件缺陷被提交到缺陷跟踪系统并经过确认之后,它会被分发给开发人员进行缺陷修复．这个过程就叫做缺陷分发．随着被提交到系统的缺陷报告日益增多,手工分发缺陷报告会变得越来越困难．提出了一种自动分发缺陷的方法BUTTER．与其他方法不同的是,BUTTER不仅利用主题模型分析缺陷报告中的文本信息,而且创新性地建立了一个包含提交者、缺陷和开发者三种节点及其相互关系的异构网络,从该异构网络中抽取了更多的结构信息．实验证明,BUTTER进行自动缺陷分发较其他缺陷自动分发方法要好．     

What do developers search for on the web?

Developers commonly make use of a web search engine such as Google to locate online resources to improve their productivity. A better understanding of what developers search for could help us understand their behaviors and the problems that they meet during the software development process. Unfortunately, we have a limited understanding of what developers frequently search for and of the search tasks that they often find challenging. To address this gap, we collected search queries from 60 developers, surveyed 235 software engineers from more than 21 countries across five continents. In particular, we asked our survey participants to rate the frequency and difficulty of 34 search tasks which are grouped along the following seven dimensions: general search, debugging and bug fixing, programming, third party code reuse, tools, database, and testing. We find that searching for explanations for unknown terminologies, explanations for exceptions/error messages (e.g., HTTP 404), reusable code snippets, solutions to common programming bugs, and suitable third-party libraries/services are the most frequent search tasks that developers perform, while searching for solutions to performance bugs, solutions to multi-threading bugs, public datasets to test newly developed algorithms or systems, reusable code snippets, best industrial practices, database optimization solutions, solutions to security bugs, and solutions to software configuration bugs are the most difficult search tasks that developers consider. Our study sheds light as to why practitioners often perform some of these tasks and why they find some of them to be challenging. We also discuss the implications of our findings to future research in several research areas, e.g., code search engines, domain-specific search engines, and automated generation and refinement of search queries.     

重复软件缺陷报告检测方法综述

软件缺陷在软件的开发和维护过程中是不可避免的,软件缺陷报告是软件维护过程中重要的缺陷描述文档,高质量的软件缺陷报告可以有效提高软件缺陷修复的效率.然而,由于存在许多开发人员、测试人员和用户与缺陷跟踪系统交互并提交软件缺陷报告,同一个软件缺陷可能被不同的人员报告,导致了大量重复的软件缺陷报告.重复的软件缺陷报告势必加重人工检测重复缺陷报告的工作量,并造成人力物力的浪费,降低了软件缺陷修复的效率.以系统文献调研的方式,对近年来国内外学者在重复软件缺陷报告检测领域的研究工作进行了系统的分析.主要从研究方法、数据集的选取、性能评价等方面具体分析总结,并提出该领域在后续研究中存在的问题、挑战以及建议.     

Eye movements in software traceability link recovery

Information Retrieval (IR) approaches, such as Latent Semantic Indexing (LSI) and Vector Space Model (VSM), are commonly applied to recover software traceability links. Recently, an approach based on developers’ eye gazes was proposed to retrieve traceability links. This paper presents a comparative study on IR and eye-gaze based approaches. In addition, it reports on the possibility of using eye gaze links as an alternative benchmark in comparison to commits. The study conducted asked developers to perform bug-localization tasks on the open source subject system JabRef. The iTrace environment, which is an eye tracking enabled Eclipse plugin, was used to collect eye gaze data. During the data collection phase, an eye tracker was used to gather the source code entities (SCE’s), developers looked at while solving these tasks. We present an algorithm that uses the collected gaze dataset to produce candidate traceability links related to the tasks. In the evaluation phase, we compared the results of our algorithm with the results of an IR technique, in two different contexts. In the first context, precision and recall metric values are reported for both IR and eye gaze approaches based on commits. In the second context, another set of developers were asked to rate the candidate links from each of the two techniques in terms of how useful they were in fixing the bugs. The eye gaze approach outperforms standard LSI and VSM approaches and reports a 55 % precision and 67 % recall on average for all tasks when compared to how the developers actually fixed the bug. In the second context, the usefulness results show that links generated by our algorithm were considered to be significantly more useful (to fix the bug) than those of the IR technique in a majority of tasks. We discuss the implications of this radically different method of deriving traceability links. Techniques for feature location/bug localization are commonly evaluated on benchmarks formed from commits as is done in the evaluation phase of this study. Although, commits are a reasonable source, they only capture entities that were eventually changed to fix a bug or resolve a feature. We investigate another type of benchmark based on eye tracking data, namely links generated from the bug-localization tasks given to the developers in the data collection phase. The source code entities relevant to subjected bugs recommended from IR methods are evaluated on both commits and links generated from eye gaze. The results of the benchmarking phase show that the use of eye tracking could form an effective (complementary) benchmark and add another interesting perspective in the evaluation of bug-localization techniques.     

Graphs for Mining-Based Defect Localization in Multithreaded Programs

Trends in modern multicore architecture design requires software developers to develop and debug multithreaded programs. Consequently, software developers must face new challenges because of bug patterns occurring at runtime and due to the non-deterministic behavior of multi-threaded program executions. This calls for new defect-localization techniques. There has been much work in the field of defect localization for sequential programs on the one side and on the localization of specific multithreading bugs on the other side, but we are not aware of any general technique for multithreaded programs. This paper proposes such an approach. It generalizes data mining-based defect-localization techniques for sequential programs. The techniques work by analyzing call graphs. More specifically, we propose new graph representations of multithreaded program executions as well as two mining-based localization approaches based on these representations. Our evaluation shows that our technique yields good results and is able to find defects that other approaches cannot localize.     

面向软件安全性缺陷的开发者推荐方法

软件开发与维护过程中常会出现一些安全性缺陷,这些安全性缺陷会给软件和用户带来很大的风险.安全性缺陷在修复过程中,其修复级别和质量要求往往高于一般性的缺陷,因此,推荐出富有安全性经验的开发者及时有效地修复这些安全性缺陷非常重要.现有的开发者推荐技术在推荐开发者时仅仅考虑了开发者的历史开发内容,很少考虑到开发人员的安全性缺陷修复经验和修复质量等因素,所以这些技术不适用于安全性缺陷的开发者推荐.本文针对安全性缺陷的修复提出了一种有效的软件开发者推荐方法SecDR.SecDR在推荐开发者时不仅考虑了开发者的历史开发内容（与安全性相关）,还分析了开发者的修复质量和历史修复缺陷的复杂度等因素.此外,SecDR还实现了开发者的多经验级别推荐：推荐初级开发者修复简单的安全性缺陷,高级开发者修复复杂的安全性缺陷.本文在三个开源项目（Mozilla,Libgdx,ElasticSearch）上分别对SecDR推荐开发者进行有效性验证.通过对比实验证明,SecDR针对安全性缺陷推荐开发者相比于其他方法（如：DR_PSF）的推荐精度平均高出19%~42%.另外,实验对比了SecDR与实际开发人员的分配情况,结果显示SecDR可以更好地规避不合理的软件开发者的推荐.     

Virtual organizational learning in open source software development projects

We studied virtual organizational learning in open source software (OSS) development projects. Specifically, our research focused on learning effects of OSS projects and the factors that affect the learning process. The number and percentage of resolved bugs and bug resolution time of 118 SourceForge.net OSS projects were used to measure the learning effects. Projects were characterized by project type, number and experience of developers, number of bugs, and bug resolution time. Our results provided evidence of virtual organizational learning in OSS development projects and support for several factors as determinants of performance. Team size was a significant predictor, with mid-sized project teams functioning best. Teams of three to seven developers exhibited the highest efficiency over time and teams of eight to 15 produced the lowest mean time for bug resolution. Increasing the percentage of bugs assigned to specific developers or boosting developer participation in other OSS projects also improved performance. Furthermore, project type introduced variability in project team performance.     

基于循环神经网络的缺陷报告分派方法

随着开源软件项目规模的不断增大,人工为缺陷报告分派合适的开发人员（缺陷分派）变得越来越困难.而不合适的缺陷分派往往会严重影响缺陷修复的效率,为此迫切需要一种缺陷分派辅助技术帮助项目管理者更好地完成缺陷分派任务.当前,大部分研究工作都基于缺陷报告文本以及相关元数据信息分析来刻画开发者的特征,忽略了对开发者活跃度的考虑,使得对具有相似特征的开发者进行缺陷报告分派预测时表现较差.本文提出了一个基于循环神经网络的深度学习模型DeepTriage,一方面利用双向循环网络加池化方法提取缺陷报告的文本特征,一方面利用单向循环网络提取特定时刻的开发者活跃度特征,并融合两者,利用已修复的缺陷报告进行监督学习.在Eclipse等四个不同的开源项目数据集上的实验结果表明,DeepTriage较同类工作在缺陷分派预测准确率上有显著提升.     

基于高斯过程的缺陷定位方法

在软件系统中,缺陷定位是缺陷修复的一个关键环节,如果能将缺陷自动定位到很小的范围,将会极大地降低缺陷修复的难度.基于高斯过程提出了一种缺陷定位方法（GPBL）,即针对每个缺陷,向开发人员推荐这个缺陷可能存在于哪些源文件中,从而帮助开发人员快速修复缺陷.为了验证方法的有效性,采集了开源软件Eclipse 和Argouml 中的数据,实验结果表明,高斯过程缺陷定位的查全率和查准率平均分别为87.16%和78.90%.与基于LDA的缺陷定位方法进行比较,表明高斯过程更能准确定位缺陷的位置.     

软件缺陷报告严重性属性分析

软件缺陷报告的严重性对缺陷的解决具有关键作用。随着软件规模的不断扩大,使用开源的软件缺陷跟踪系统成为海量缺陷信息数据的主要处理方法。分析缺陷报告严重性在数据仓库中的作用,是处理软件缺陷的重要内容。通过对Bugzilla缺陷跟踪系统数据的研究和分析,发现不同项目的属性特征差异较大,同时在修复率、解决时长、开发者、组件等属性上的统计特征具有一致性。对Mozilla项目和Eclipse项目的数据进行系统分析,并根据不同组件和项目中严重性程度分布情况,认为软件缺陷报告严重性程度的提升会导致缺陷修复率的提高,同时严重性程度为normal级别的缺陷解决时长最短,开发者持有缺陷的数量越高其修复率越低。     

Understanding the triaging and fixing processes of long lived bugs

ContextBug fixing is an integral part of software development and maintenance. A large number of bugs often indicate poor software quality, since buggy behavior not only causes failures that may be costly but also has a detrimental effect on the user’s overall experience with the software product. The impact of long lived bugs can be even more critical since experiencing the same bug version after version can be particularly frustrating for user. While there are many studies that investigate factors affecting bug fixing time for entire bug repositories, to the best of our knowledge, none of these studies investigates the extent and reasons of long lived bugs.ObjectiveIn this paper, we investigate the triaging and fixing processes of long lived bugs so that we can identify the reasons for delay and improve the overall bug fixing process.MethodologyWe mine the bug repositories of popular open source projects, and analyze long lived bugs from five different perspectives: their proportion, severity, assignment, reasons, as well as the nature of fixes.ResultsOur study on seven open-source projects shows that there are a considerable number of long lived bugs in each system and over 90% of them adversely affect the user’s experience. The reasons for these long lived bugs are diverse including long assignment time, not understanding their importance in advance, etc. However, many bug-fixes were delayed without any specific reasons. Furthermore, 40% of long lived bugs need only small fixes.ConclusionOur overall results suggest that a significant number of long lived bugs may be minimized through careful triaging and prioritization if developers could predict their severity, change effort, and change impact in advance. We believe our results will help both developers and researchers better to understand factors behind delays, improve the overall bug fixing process, and investigate analytical approaches for prioritizing bugs based on bug severity as well as expected bug fixing effort.     

Do bugs foreshadow vulnerabilities? An in-depth study of the chromium project

As developers face an ever-increasing pressure to engineer secure software, researchers are building an understanding of security-sensitive bugs (i.e. vulnerabilities). Research into mining software repositories has greatly increased our understanding of software quality via empirical study of bugs. Conceptually, however, vulnerabilities differ from bugs: they represent an abuse of functionality as opposed to insufficient functionality commonly associated with traditional, non-security bugs. We performed an in-depth analysis of the Chromium project to empirically examine the relationship between bugs and vulnerabilities. We mined 374,686 bugs and 703 post-release vulnerabilities over five Chromium releases that span six years of development. We used logistic regression analysis, ranking analysis, bug type classifications, developer experience, and vulnerability severity metrics to examine the overarching question: are bugs and vulnerabilities in the same files? While we found statistically significant correlations between pre-release bugs and post-release vulnerabilities, we found the association to be weak. Number of features, source lines of code, and pre-release security bugs are, in general, more closely associated with post-release vulnerabilities than any of our non-security bug categories. In further analysis, we examined sub-types of bugs, such as stability-related bugs, and the associations did not improve. Even the files with the most severe vulnerabilities (by measure of CVSS or bounty payouts) did not show strong correlations with number of bugs. These results indicate that bugs and vulnerabilities are empirically dissimilar groups, motivating the need for security engineering research to target vulnerabilities specifically.     

SMTLOC: 基于多源频谱的SMT求解器缺陷定位

SMT求解器作为重要的基础软件, 其存在的缺陷可能会导致依赖于它的软件功能失效, 甚至带来安全事故. 然而, 修复SMT求解器缺陷是一个十分耗时的任务, 因为开发者需要花费大量的时间和精力来理解并找到缺陷的根本原因. 虽然已有许多软件缺陷定位方面的研究, 但尚未有系统的工作研究如何自动定位SMT求解器缺陷. 因此, 提出一种基于多源频谱的SMT求解器缺陷定位方法SMTLOC. 首先, 对于给定的SMT求解器缺陷, SMTLOC提出一种枚举算法, 用以对触发该缺陷的公式进行变异, 从而生成一组不触发缺陷, 但与触发缺陷的公式具有相似执行路径的证人公式. 然后, SMTLOC根据证人公式的执行路径以及SMT求解器的源码信息, 提出一种融合覆盖频谱和历史频谱的文件可疑度计算方法, 从而定位可能存在缺陷的文件. 为了验证SMTLOC的有效性, 收集60个SMT求解器缺陷. 实验结果表明, SMTLOC的缺陷定位效果明显优于传统的频谱缺陷定位方法, SMTLOC可以将46.67%的缺陷定位在TOP-5的文件内, 定位效果提升了133.33%.     

基于信息检索的软件缺陷定位技术研究进展

缺陷定位是软件工程研究最活跃的领域之一.大部分软件缺陷都会被提交到类似于Bugzilla和Jira的缺陷追踪系统中.由于提交的缺陷报告数量过多,开发人员不能及时地处理,因而迫切需要一个自动化工具来帮助开发人员识别缺陷相关源代码文件.研究人员已经提出了大量的缺陷定位技术.基于信息检索的软件缺陷定位技术（Information Retrieval-based Bug Localization,简称IRBL）利用了缺陷报告的文本特性,并且由于计算成本低、对不同的程序语言更具有普适性,成为缺陷定位领域的研究热点,取得了一系列研究成果.然而,IRBL技术也在数据预处理、相似度计算和工程应用等方面存在诸多挑战.鉴于此,本文对现有的IRBL技术进行梳理总结.主要内容包括：（1）梳理了IRBL中数据预处理的过程和信息检索通用方法;（2）对IRBL技术中利用的数据特征进行了详细的分类和总结;（3）总结了技术评估中使用的性能评估指标;（4）归纳出了IRBL技术的关键问题;（5）最后展望了IRBL技术的未来发展.     

High-Impact Bug Report Identification with Imbalanced Learning Strategies

In practice, some bugs have more impact than others and thus deserve more immediate attention. Due to tight schedule and limited human resources, developers may not have enough time to inspect all bugs. Thus, they often concentrate on bugs that are highly impactful. In the literature, high-impact bugs are used to refer to the bugs which appear at unexpected time or locations and bring more unexpected effects (i.e., surprise bugs), or break pre-existing functionalities and destroy the user experience (i.e., breakage bugs). Unfortunately, identifying high-impact bugs from thousands of bug reports in a bug tracking system is not an easy feat. Thus, an automated technique that can identify high-impact bug reports can help developers to be aware of them early, rectify them quickly, and minimize the damages they cause. Considering that only a small proportion of bugs are high-impact bugs, the identification of high-impact bug reports is a difficult task. In this paper, we propose an approach to identify high-impact bug reports by leveraging imbalanced learning strategies. We investigate the effectiveness of various variants, each of which combines one particular imbalanced learning strategy and one particular classification algorithm. In particular, we choose four widely used strategies for dealing with imbalanced data and four state-of-the-art text classification algorithms to conduct experiments on four datasets from four different open source projects. We mainly perform an analytical study on two types of high-impact bugs, i.e., surprise bugs and breakage bugs. The results show that different variants have different performances, and the best performing variants SMOTE (synthetic minority over-sampling technique) + KNN (K-nearest neighbours) for surprise bug identification and RUS (random under-sampling) + NB (naive Bayes) for breakage bug identification outperform the F1-scores of the two state-of-the-art approaches by Thung et al. and Garcia and Shihab.     

Applying CSCW technology to overcome traditional barriers in group interactions

CSCW systems are designed to improve group communication while alleviating negative interactions. However, their developers often focus on the technology, producing highly sophisticated software systems to address problems that may not exist in real-life groups or may not effectively address problems encountered by them. As a result of an extensive review of the literature and a large-scale survey of meeting participants in a University setting, we believe that we have identified the most important problems in group interaction. These are matched with specific features of CSCW technology, as well as with their potential drawbacks. This problem–solution–drawback arrangement is posited as a point of departure for a common knowledge base for future CSCW development.     

DFTracker: detecting double-fetch bugs by multi-taint parallel tracking

A race condition is a common trigger for concurrency bugs. As a special case, a race condition can also occur across the kernel and user space causing a doublefetch bug, which is a field that has received little research attention. In our work, we first analyzed real-world doublefetch bug cases and extracted two specific patterns for doublefetch bugs. Based on these patterns, we proposed an approach of multi-taint parallel tracking to detect double-fetch bugs. We also implemented a prototype called DFTracker (doublefetch bug tracker), and we evaluated it with our test suite. Our experiments demonstrated that it could effectively find all the double-fetch bugs in the test suite including eight realworld cases with no false negatives and minor false positives. In addition, we tested it on Linux kernel and found a new double-fetch bug. The execution overhead is approximately 2x for single-file cases and approximately 9x for the whole kernel test, which is acceptable. To the best of the authors’ knowledge, this work is the first to introduce multi-taint parallel tracking to double-fetch bug detection—an innovative method that is specific to double-fetch bug features—and has better path coverage as well as lower runtime overhead than the widely used dynamic approaches.     

开源软件缺陷报告自动摘要研究综述

在开源软件开发的维护阶段, 开源软件缺陷报告为开发人员解决缺陷提供了大量帮助。然而, 开源软件缺陷报告通常是以用户对话的形式编写, 一个软件缺陷报告可能含有数十条评论和上千个句子, 导致开发人员难以阅读或理解软件缺陷报告。为了缓解这个问题, 人们提出了开源软件缺陷报告自动摘要, 缺陷报告自动摘要可以减少开发人员阅读冗长缺陷报告的时间。本文以综述的方式对开源软件缺陷报告自动摘要的研究做了系统的归纳总结。首先, 根据摘要的表现形式, 将开源软件缺陷报告摘要分类为固定缺陷报告摘要和可视化缺陷报告摘要, 再将固定缺陷报告摘要研究方法分类为基于监督学习方法和基于无监督学习方法, 之后总结了基于监督学习和无监督学习的开源软件缺陷报告摘要生成的工作框架, 并介绍了开源软件缺陷报告摘要领域常用数据集、预处理技术和摘要评估指标。其次, 本文以无监督学习为切入点, 分类阐述和归纳了无监督开源软件缺陷报告摘要方法, 将无监督开源软件缺陷报告摘要方法分类为: 基于特征评分方法、基于深度学习方法、基于图方法和基于启发式方法, 并对每类方法进行讨论与分析。再次, 从缺陷报告摘要的实用性出发, 对现有的缺陷报告可视化摘要研究成果进行总结,并对固定缺陷报告摘要和可视化缺陷报告摘要的实用性做出分析。最后, 对现有研究成果及综述进行讨论和分析, 指出了开源软件缺陷报告摘要领域在缺陷报告数据集、抽取式摘要和黄金标准摘要三个方面面临的挑战和对未来研究的展望。