代码审查自动化研究综述

随着现代软件规模的不断扩大,协作开发成为软件开发的主流趋势,代码审查成为了现代化软件开发的重要工作流程.但由于人工代码评审往往耗费审查者较大精力,且存在审查者不匹配或审查者水平有限等问题,人工代码评审的质量和效率难以保证,且审查后的代码修复也十分费时费力.因此亟需研究人员为代码审查流程进行改进,提供自动化思路.本文对代码审查自动化相关研究进行系统梳理和总结,并重点介绍4种主要方向:审查者推荐、代码变更质量评估、审查意见生成和代码自动修复.并整理了相关方向的148篇研究,对每个方向的研究进行技术分类与分析.随后,本文整理了各方向研究任务的评估方法,并整理出常用的数据集与开源工具.最后,对代码审查自动化领域面临的问题进行梳理,并对未来研究进行展望.     

A systematic literature review to identify and classify software requirement errors

Most software quality research has focused on identifying faults (i.e., information is incorrectly recorded in an artifact). Because software still exhibits incorrect behavior, a different approach is needed. This paper presents a systematic literature review to develop taxonomy of errors (i.e., the sources of faults) that may occur during the requirements phase of software lifecycle. This taxonomy is designed to aid developers during the requirement inspection process and to improve overall software quality. The review identified 149 papers from the software engineering, psychology and human cognition literature that provide information about the sources of requirements faults. A major result of this paper is a categorization of the sources of faults into a formal taxonomy that provides a starting point for future research into error-based approaches to improving software quality.     

What Are They Talking About? Analyzing Code Reviews in Pull-Based Development Model

Code reviews in pull-based model are open to community users on GitHub. Various participants are taking part in the review discussions and the review topics are not only about the improvement of code contributions but also about project evolution and social interaction. A comprehensive understanding of the review topics in pull-based model would be useful to better organize the code review process and optimize review tasks such as reviewer recommendation and pull-request prioritization. In this paper, we first conduct a qualitative study on three popular open-source software projects hosted on GitHub and construct a fine-grained two-level taxonomy covering four level-1 categories (code correctness, pull-request decision-making, project management, and social interaction) and 11 level-2 subcategories (e.g., defect detecting, reviewer assigning, contribution encouraging). Second, we conduct preliminary quantitative analysis on a large set of review comments that were labeled by TSHC (a two-stage hybrid classification algorithm), which is able to automatically classify review comments by combining rule-based and machine-learning techniques. Through the quantitative study, we explore the typical review patterns. We find that the three projects present similar comments distribution on each subcategory. Pull-requests submitted by inexperienced contributors tend to contain potential issues even though they have passed the tests. Furthermore, external contributors are more likely to break project conventions in their early contributions.     

Does certainty tone matter? Effects of review certainty,reviewer characteristics,and organizational niche width on review usefulness

Given the proliferation of online review websites—e.g., Yelp.com—that prominently display a large number of online custome0r reviews, scholars have made efforts to investigate what makes a review “useful.” However, there is little research that offers insight into how review content, reviewer characteristics, and review contexts jointly influence review usefulness. We specially examine the role of review certainty on review usefulness. Drawing on dual-process and social influence theories, we examine the interaction effects of review certainty, reviewer popularity, reviewer expertise, and the niche width of a restaurant on review usefulness. In particular, this study focuses on how review certainty interacts with other contextual factors to influence users’ evaluation of the usefulness of online reviews. Utilizing a zero-inflated negative binomial Poisson regression, we empirically tested our hypotheses based on 10,097 reviews on 2,383 restaurants from Yelp.com. Our results indicated that (1) the impact of review certainty on review usefulness decreases with reviewer popularity but does not vary with reviewer expertise and (2) the niche width of a restaurant—as a contextual feature—interacts with review certainty and reviewer characteristics in influencing review usefulness. Theoretically, these findings contribute to online customer review literature and certainty literature, as well as social media research, provide new guidelines for predicting review usefulness, and add new insights into understanding the role of organizational positioning for customer evaluations. In practice, our findings can help online review platforms better understand how to screen and select useful reviews for visitors.     

Pair programming and the mysterious role of the navigator

Computer programming is generally understood to be highly challenging and since its inception a wide range of approaches, tools and methodologies have been developed to assist in managing its complexity. Relatively recently the potential benefits of collaborative software development have been formalised in the practice of pair programming. Here we attempt to ‘unpick’ the pair programming process through the analysis of verbalisations from a number of commercial studies. We focus particularly on the roles of the two programmers and what their key characteristics and behaviours might be. In particular, we dispute two existing claims: (i) that the programmer who is not currently typing in code (“the navigator”) is constantly reviewing what is typed and highlighting any errors (i.e. acting as a reviewer) and (ii) that the navigator focuses on a different level of abstraction as a way of ensuring coverage at all necessary levels (i.e. acting as a foreman). We provide an alternative model for these roles (“the tag team”) in which the driver and navigator play much more equal roles. We also suggest that a key factor in the success of pair programming may be the associated increase in talk at an intermediate level of abstraction.     

响应时间约束的代码评审人推荐

同行代码评审,即对提交代码进行人工评审,是减少软件缺陷和提高软件质量的有效手段,已被Github等开源社区以及很多软件开发组织广泛采用.在GitHub社区,代码评审是其pull-based软件开发模型的重要组成部分.开源项目往往存在成百上千个候选评审人员,为评审工作推荐合适的评审人员是一项很有价值且挑战性的工作.基于真实开源项目的数据分析发现,评审响应时间过长是普遍存在的问题,这会延长评审周期、降低参与人员积极性,而已有的代码评审人推荐工作均没有考虑响应时间这个因素.因此,提出了响应时间约束的代码评审人推荐问题,即推荐的评审人能否在约定时间内进行评审;进而提出了基于多目标优化的代码评审人推荐方法（MOC2R）,该方法通过最大化代码评审人经验、最大化在约定时间内的响应概率、最大化人员最近时间内的活跃性这3个目标,使用多目标优化算法来推荐代码评审人员.基于6个开源项目的数据进行实验,结果表明,在不同时间窗约束下（2h、4h、8h）,Top-1准确率为41.7%~61.5%,Top-5准确率为66.5%~77.7%,显著优于两条常用且业内领先的基线方法,且3个目标均对人员推荐有贡献,其中,约定时间内的响应概率目标对于人员推荐的贡献最大.该方法能够进一步提升代码评审效率,提高开源社区的活跃性.     

Composing, analyzing and validating software models to assess the performability of competing design candidates

In a perfect world, verification and validation of a software design specification would be possible before any code was generated. Indeed, in a perfect world we would know that the implementation was correct because we could trust the class libraries, the development tools, verification tools and simulations, etc. These features would provide the confidence needed to know that all aspects (complexity, logical and timing correctness) of the design were fully satisfied (i.e., everything was right). Right in the sense that we built it right (it is correct with respect to its specification) and it solves the right problem. Unfortunately, it is not a perfect world, and therefore we must strive to continue to refine, develop and validate useful methods and tools for the creation of safe and correct software. This paper considers the analysis of systems expressed using formal notations. We introduce our framework, the modeling cycle, and motivate the need for tool supported rigorous methods. Our framework is about using systematic formal techniques for the creation and composition of software models that can further enable reasoning about high‐assurance systems. We describe several formal modeling techniques within this context (i.e., reliability and availability models, performance and functional models, performability models, etc.). This discussion includes a more precise discourse on stochastic methods (i.e., DTMC and CTMC) and their formulation. In addition, we briefly review the underlying theories and assumptions that are used to solve these models for the measure of interest (i.e., simulation, numerical and analytical techniques). Finally, we present a simple example that employs generalized stochastic Petri nets and illustrates the usefulness of such analysis methods. This revised version was published online in June 2006 with corrections to the Cover Date.     

The EMISQ method and its tool support-expert-based evaluation of internal software quality

There is empirical evidence that internal software quality, e.g., the quality of source code, has great impact on the overall quality of software. Besides well-known manual inspection and review techniques for source code, more recent approaches utilize tool-based static code analysis for the evaluation of internal software quality. Despite the high potential of code analyzers the application of tools alone cannot replace well-founded expert opinion. Knowledge, experience and fair judgment are indispensable for a valid, reliable quality assessment, which is accepted by software developers and managers. The EMISQ method (Evaluation Method for Internal Software Quality), guides the assessment process for all stakeholders of an evaluation project. The method is supported by the Software Product Quality Reporter (SPQR), a tool which assists evaluators with their analysis and rating tasks and provides support for generating code quality reports. The application of SPQR has already proved its usefulness in various code assessment projects around the world. This paper introduces the EMISQ method and describes the tool support needed for an efficient and effective evaluation of internal software quality.     

Reengineering component-based software systems with Archimetrix

Many software development, planning, or analysis tasks require an up-to-date software architecture documentation. However, this documentation is often outdated, unavailable, or at least not available as a formal model which analysis tools could use. Reverse engineering methods try to fill this gap. However, as they process the system’s source code, they are easily misled by design deficiencies (e.g., violations of component encapsulation) which leaked into the code during the system’s evolution. Despite the high impact of design deficiencies on the quality of the resulting software architecture models, none of the surveyed related works is able to cope with them during the reverse engineering process. Therefore, we have developed the Archimetrix approach which semiautomatically recovers the system’s concrete architecture in a formal model while simultaneously detecting and removing design deficiencies. We have validated Archimetrix on a case study system and two implementation variants of the CoCoME benchmark system. Results show that the removal of relevant design deficiencies leads to an architecture model which more closely matches the system’s conceptual architecture.     

Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining

Many software production processes advocate rigorous development testing alongside functional code writing, which implies that both test code and production code should co-evolve. To gain insight in the nature of this co-evolution, this paper proposes three views (realized by a tool called TeMo) that combine information from a software project’s versioning system, the size of the various artifacts and the test coverage reports. We validate these views against two open source and one industrial software project and evaluate our results both with the help of log messages, code inspections and the original developers of the software system. With these views we could recognize different co-evolution scenarios (i.e., synchronous and phased) and make relevant observations for both developers as well as test engineers.     

无线电调谐单元软件语句覆盖目标的实现

本文以无线电调谐单元设备中的无线电调谐软件为例,通过引入LDRATESTBED自动化测试工具提高测试效率,实现满足DO-178C中的语句覆盖数据的目标.语句覆盖是民用飞机软件研制中最基本的覆盖率测试目标.语句覆盖可以有效检测出源代码中的多余代码,提高代码质量.通过实验表明LDRATESTBED可以快速完成源代码插桩以及代码覆盖率数据分析任务,同时,该工具自动标识源代码中未执行的代码,为开发者提供了分析的基础.工具生成的语句覆盖率数据满足DO-178C标准目标要求,可以作为证据提交给局方进行审查,提高软件的置信度.     

静态软件缺陷预测方法研究

静态软件缺陷预测是软件工程数据挖掘领域中的一个研究热点.通过分析软件代码或开发过程,设计出与软件缺陷相关的度量元;随后,通过挖掘软件历史仓库来创建缺陷预测数据集,旨在构建出缺陷预测模型,以预测出被测项目内的潜在缺陷程序模块,最终达到优化测试资源分配和提高软件产品质量的目的.对近些年来国内外学者在该研究领域取得的成果进行了系统的总结.首先,给出了研究框架并识别出了影响缺陷预测性能的3个重要影响因素:度量元的设定、缺陷预测模型的构建方法和缺陷预测数据集的相关问题;接着,依次总结了这3个影响因素的已有研究成果;随后,总结了一类特殊的软件缺陷预测问题(即,基于代码修改的缺陷预测)的已有研究工作;最后,对未来研究可能面临的挑战进行了展望.     

Detecting logic errors in discrete-event simulation: reverse engineering through event graphs

Several inherent constraints remain in the model development process, even though modern enhancements to simulation environments have provided tools for code generation, debugging, and tracing. To develop a simulation model, the simulation analyst still needs to have expertise in a number of different fields, e.g., probability, statistics, design of experiments, modeling, systems engineering, software engineering, and computer programming. Although several simulation packages implement syntactic-checks and semantic-consistency-checks, typically, the simulation analyst needs to possess output-analysis-knowledge specifically aimed at verifying and checking the simulation code.Reverse engineering a graphical model, e.g., an event graph, from general purpose simulation code demonstrates an enhancement to the model development process. A reverse engineering step allows an analyst to check, both, the static and dynamic properties of the coded simulation model. Even though the reverse engineering produces an event-oriented view, the enhanced model development process provides a systematic approach for conversion from other world views. Overall, this enhanced process provides a framework which yields better analysis techniques.Better diagnostic assistance is achieved when viewing a combination of static and dynamic properties of the simulation code. Now, the analyst is able to find logical/execution errors, e.g., errors related to resource deadlocks, before running simulation experiments. Since the graphical model is generated from the simulation code, and the process combines views, the analyst also has a better framework for verifying the coded simulation model. Also, the reverse engineering step provides a structural model useful in converting between different simulation languages or systems. Improvements to the techniques for conversion between languages will facilitate reuse of existing programmed models.     

Automating ETL processes using the domain-specific modeling approach

The development of Extract–Transform–Load (ETL) processes is the most complex, time-consuming and expensive phase of data warehouse development. Yet, the dynamics of modern business systems demand a more agile and flexible approach to their development. As a result, current research in this area is focused on ETL process conceptualization and the automation of ETL process development. This paper proposes a novel solution for automating ETL processes using the domain-specific modeling (DSM) approach. The proposed solution is based on the formal specification of ETL processes and the implementation of such formal specifications. Thus, in accordance with the DSM approach, several new domain-specific languages (DSLs) are introduced, each defining concepts relevant for a specific aspect of an ETL process. The focus of this paper is the actual implementation of the formal specification of an ETL process. To this end, a specific ETL platform (ETL-PL) is introduced to technologically support both the modeling of ETL processes (i.e., the creation of models in accordance with the introduced DSLs) and the automated transformation of the created models into the executable code of a specific application framework (representing ETL-PL’s execution environment). It should be emphasized that ETL-PL actually presumes the dynamic execution of ETL models or, more precisely, the executable code is generated at runtime. Thus the execution environment consists of code generator components and the components implementing the application framework. ETL-PL has been implemented as an extension of the .NET platform.     

Legal ontology of sales law application to ecommerce

Legal codes, such as the Uniform Commercial Code (UCC) examined in this article, are good points of entry for AI and ontology work because of their more straightforward adaptability to relationship linking and rules-based encoding. However, approaches relying on encoding solely on formal code structure are incomplete, missing the rich experience of practitioner expertise that identifies key relationships and decision criteria often supplied by experienced practitioners and process experts from various disciplines (e.g., sociology, political economics, logistics, operations research). This research focuses on the UCC because it transcends the limitations of a formal code, functioning essentially as a composite. AI work can benefit from real-world codes like the UCC, which are essentially formal codes enlightened from a more realistic experience-base from centuries of development in international commercial transactions settings. This paper then describes our initial work in converting an expert system on the U.S. law governing the sale of goods from Article II of the Uniform Commercial Code (UCC), into a knowledge-based system using the Web Ontology Language OWL.     

航空机载嵌入式控制软件需求建模的形式化工程方法

嵌入式控制软件是现代航空飞行器的核心部件之一。构建软件需求的形式化规约精确地刻画人们对软件期望的功能和运行场景,是确保此类安全攸关软件质量的根本途径。在工业界,形式化需求建模的大规模应用尽管有成功的案例,但仍面临众多的困难。其根本性难点在于缺少一种系统化的工程方法来引导工业界软件实践者,从原始需求开始最终完成形式化需求规约,并能确认该规约真实、充分地反映了人们对软件期望的功能。针对上述挑战,提出了一种面向机载控制软件需求建模的形式化工程方法ACSDL-MV,以形式化方法为理论基础,结合软件需求工程的基本原理,引导工程人员从原始需求出发以演化式的过程逐步完成需求规约的构建;定制了航空控制软件的形式化描述语言ACSDL,用以构建形式化规约;为了确认软件需求规约准确、充分地描述了人们对软件期望的功能,该方法给出了基于图形的静态审查和基于模型的动态模拟技术。在航空发动机公司中的实验结果表明,该方法相比传统方法探测到了更多的潜在错误。     

A group process for defining local software quality: Field applications and validation experiments

‘Global’ measures of software quality are generally not used by practitioners because they have not been calibrated for local operating environments. A participative process for defining local software quality has been defined, applied in the field, and evaluated in formal validation experiments. In four field applications of the process, a ‘jury’ of seasoned practitioners ranked a sample of modules from the organization's inventory of code and produced a ‘software quality rule set’. The results of each field application were tested in a formal validation experiment to determine whether the obtained rule set was a valid representation of the organization's sense of quality, i.e. whether the locally-derived rules could enable non-organizational programmers to make judgments about quality that were similar to those of the jury. In each experiment, non-organizational programmers ranked the same set of modules that were ranked by the organization's jury, using either the locally-derived rules, generic (textbook) rules, placebo (non-helpful) rules, or no rules. The correlation between the judgments of non-organizational programmers using local rules and the jury was statistically significant at p < 0–005; all other correlations were significant at p < 0–005. The results support a conclusion that the group process for defining local software quality is reliable in producing valid rule sets.     

On the automatic modularization of software systems using the Bunch tool

Since modern software systems are large and complex, appropriate abstractions of their structure are needed to make them more understandable and, thus, easier to maintain. Software clustering techniques are useful to support the creation of these abstractions by producing architectural-level views of a system's structure directly from its source code. This paper examines the Bunch clustering system which, unlike other software clustering tools, uses search techniques to perform clustering. Bunch produces a subsystem decomposition by partitioning a graph of the entities (e.g., classes) and relations (e.g., function calls) in the source code. Bunch uses a fitness function to evaluate the quality of graph partitions and uses search algorithms to find a satisfactory solution. This paper presents a case study to demonstrate how Bunch can be used to create views of the structure of significant software systems. This paper also outlines research to evaluate the software clustering results produced by Bunch.