Design and evaluation of a multi-recommendation system for local code search

Searching for relevant code in the local code base is a common activity during software maintenance. However, previous research indicates that 88% of manually composed search queries retrieve no relevant results. One reason that many searches fail is existing search tools’ dependence on string matching algorithms, which cannot find semantically related code. To solve this problem by helping developers compose better queries, researchers have proposed numerous query recommendation techniques, relying on a variety of dictionaries and algorithms. However, few of these techniques are empirically evaluated by usage data from real-world developers. To fill this gap, we designed a multi-recommendation system that relies on the cooperation between several query recommendation techniques. We implemented and deployed this recommendation system within the Sando code search tool and conducted a longitudinal field study. Our study shows that over 34% of all queries were adopted from recommendation; and recommended queries retrieved results 11% more often than manual queries.     

一种高效的源代码搜索算法

多年来的软件开发积累了大量的源代码,同时不少代码搜索工具也被开发出来,但是,现有的工具都不够精确,因而很少有人使用。本文提出一种高效的源代码搜索算法,通过识别查询语句与API的关系,以提高代码搜索的准确性。基于该算法,本文实现一个针对C#源代码的搜索工具,并通过客观实验与用户调研对算法进行评估。实验结果表明本文提出的搜索算法是十分有效的。      

What do developers search for on the web?

Developers commonly make use of a web search engine such as Google to locate online resources to improve their productivity. A better understanding of what developers search for could help us understand their behaviors and the problems that they meet during the software development process. Unfortunately, we have a limited understanding of what developers frequently search for and of the search tasks that they often find challenging. To address this gap, we collected search queries from 60 developers, surveyed 235 software engineers from more than 21 countries across five continents. In particular, we asked our survey participants to rate the frequency and difficulty of 34 search tasks which are grouped along the following seven dimensions: general search, debugging and bug fixing, programming, third party code reuse, tools, database, and testing. We find that searching for explanations for unknown terminologies, explanations for exceptions/error messages (e.g., HTTP 404), reusable code snippets, solutions to common programming bugs, and suitable third-party libraries/services are the most frequent search tasks that developers perform, while searching for solutions to performance bugs, solutions to multi-threading bugs, public datasets to test newly developed algorithms or systems, reusable code snippets, best industrial practices, database optimization solutions, solutions to security bugs, and solutions to software configuration bugs are the most difficult search tasks that developers consider. Our study sheds light as to why practitioners often perform some of these tasks and why they find some of them to be challenging. We also discuss the implications of our findings to future research in several research areas, e.g., code search engines, domain-specific search engines, and automated generation and refinement of search queries.     

Concept location using program dependencies and information retrieval (DepIR)

ContextThe functionality of a software system is most often expressed in terms of concepts from its problem or solution domains. The process of finding where these concepts are implemented in the source code is known as concept location and it is a prerequisite of software change.ObjectiveWe investigate a static approach to concept location named DepIR that combines program dependency search (DepS) with information retrieval-based search (IR). In this approach, programmers explore the static program dependencies of the source code components retrieved by the IR search engine.MethodThe paper presents an empirical study that compares DepIR with its constituent techniques. The evaluation is based on an empirical method of reenactment that emulates the steps of concept location for 50 past changes mined from software repositories of five software systems.ResultsThe results of the study indicate that DepIR significantly outperforms both DepS and IR.ConclusionDepIR allows developers to perform concept location efficiently. It allows finding concepts even with queries that do not rank the relevant software components highly. Since formulating a good query is not always easy, this tolerance of lower-quality queries significantly broadens the usability of DepIR compared to the traditional IR.     

How are Java software developers using the Elipse IDE?

The Eclipse integrated development environment continues to gain popularity among Java developers. Our usage monitoring approach allows tool builders to sample how developers are using their tools in the wild. The data gathered about tool use can be used to prevent feature bloat and to evolve the environments according to user needs. Information about how developers work in a development environment can also provide a baseline for assessing new software development tools. We hope this report provides a start in defining which in formation to collect and distribute on an on going basis to help improve Eclipse and other similar platforms and tools.     

Inventive Tool Use to Comprehend Big Code

Software developers often need to understand a large body of unfamiliar code with little or no documentation, no experts to consult, and little time to do it. A post appeared in January 2008 on Slashdot, a technology news Web site, asking for tools and techniques that could help. This article analyzes 301 often passionate and sometimes articulate responses to this query, including the themes and the associated tool recommendations. The most common suggestions were to use a code navigation tool, use a design recovery tool, use a debugger to step through the code, create a runtime trace, use problem-based learning, ask people for help, study the code from top down, and print out all the code. This analysis presents an intriguing snapshot of how software developers in industry go about comprehending big code.     

An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks

Much of software developers' time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an unfamiliar program and asked to work on two debugging tasks and three enhancement tasks for 70 minutes. The study found that developers interleaved three activities. They began by searching for relevant code both manually and using search tools; however, they based their searches on limited and misrepresentative cues in the code, environment, and executing program, often leading to failed searches. When developers found relevant code, they followed its incoming and outgoing dependencies, often returning to it and navigating its other dependencies; while doing so, however, Eclipse's navigational tools caused significant overhead. Developers collected code and other information that they believed would be necessary to edit, duplicate, or otherwise refer to later by encoding it in the interactive state of Eclipse's package explorer, file tabs, and scroll bars. However, developers lost track of relevant code as these interfaces were used for other tasks, and developers were forced to find it again. These issues caused developers to spend, on average, 35 percent of their time performing the mechanics of navigation within and between source files. These observations suggest a new model of program understanding grounded in theories of information foraging and suggest ideas for tools that help developers seek, relate, and collect information in a more effective and explicit manner     

Improving software security with static automated code analysis in an industry setting

Software security can be improved by identifying and correcting vulnerabilities. In order to reduce the cost of rework, vulnerabilities should be detected as early and efficiently as possible. Static automated code analysis is an approach for early detection. So far, only few empirical studies have been conducted in an industrial context to evaluate static automated code analysis. A case study was conducted to evaluate static code analysis in industry focusing on defect detection capability, deployment, and usage of static automated code analysis with a focus on software security. We identified that the tool was capable of detecting memory related vulnerabilities, but few vulnerabilities of other types. The deployment of the tool played an important role in its success as an early vulnerability detector, but also the developers perception of the tools merit. Classifying the warnings from the tool was harder for the developers than to correct them. The correction of false positives in some cases created new vulnerabilities in previously safe code. With regard to defect detection ability, we conclude that static code analysis is able to identify vulnerabilities in different categories. In terms of deployment, we conclude that the tool should be integrated with bug reporting systems, and developers need to share the responsibility for classifying and reporting warnings. With regard to tool usage by developers, we propose to use multiple persons (at least two) in classifying a warning. The same goes for making the decision of how to act based on the warning. Copyright © 2012 John Wiley & Sons, Ltd.     

一种基于子图搜索的特征定位方法

确定源代码中与给定需求特征相关的程序元素的过程称为特征定位。但现在大部分的特征定位方法仍是以特征相关的描述和代码结构信息为输入,以特征相关的代码元素为输出。这样的结果缺乏代码元素间的关系信息,难以让开发者快速理解相关特征的程序结构。针对这个问题,提出一种基于子图搜索的特征定位方法。该方法能找出与特征相关的代码元素,并以依赖调用图的形式将结果展示出来,让开发者快速了解代码结构。根据该方法实现了相应工具并经过实验验证了方法的有效性。该方法的平均准确率为40.41%,平均召回率为50.28%。     

Sourcerer: An infrastructure for large-scale collection and analysis of open-source code

A large amount of open source code is now available online, presenting a great potential resource for software developers. This has motivated software engineering researchers to develop tools and techniques to allow developers to reap the benefits of these billions of lines of source code. However, collecting and analyzing such a large quantity of source code presents a number of challenges. Although the current generation of open source code search engines provides access to the source code in an aggregated repository, they generally fail to take advantage of the rich structural information contained in the code they index. This makes them significantly less useful than Sourcerer for building state-of-the-art software engineering tools, as these tools often require access to both the structural and textual information available in source code.We have developed Sourcerer, an infrastructure for large-scale collection and analysis of open source code. By taking full advantage of the structural information extracted from source code in its repository, Sourcerer provides a foundation upon which state-of-the-art search engines and related tools can easily be built. We describe the Sourcerer infrastructure, present the applications that we have built on top of it, and discuss how existing tools could benefit from using Sourcerer.     

The SEXTANT Software Exploration Tool

In this paper, we discuss a set of functional requirements for software exploration tools and provide initial evidence that various combinations of these features are needed to effectively assist developers in understanding software. We observe that current tools for software exploration only partly support these features. This has motivated the development of SEXTANT, a software exploration tool tightly integrated into the Eclipse IDE that has been developed to fill this gap. By means of case studies, we demonstrate how the requirements fulfilled by SEXTANT are conducive to an understanding needed to perform a maintenance task     

CODAS:一个易扩展的静态代码缺陷分析服务

利用静态代码缺陷分析技术对软件进行早期缺陷检测,是提高软件质量的重要途径。静态代码缺陷分析工具(如FINDBUGS,JLINT,ESC/JAVA,PMD,COVERITY等)已经被证实可以成功地识别出大量的软件潜在缺陷[1-3]。然而,这类工具在可用性和有效性方面的不足严重限制了它们的进一步广泛使用。可用性不足包括a)每个独立缺陷检测工具只擅于检测特定类型的缺陷,需要配合使用才能全面检测缺陷;b)每个缺陷检测工具的安装、配置和运行占用了用户大量的时间、精力。有效性不足包括静态缺陷分析结果往往存在大量误报,并且会包括许多不重要的(不会引起程序员修复行为的)缺陷报告。为了解决上述问题,提出并构建了一个易扩展的"静态代码缺陷分析"服务(Code Defect Analysis Service,CODAS)。CODAS基于一个高度可扩展的架构设计,对多个独立的缺陷检测工具进行了封装和集成,并对缺陷检测报告进行了有效汇总和排序,从而充分发挥了各个独立工具的优势,大大提升了静态缺陷分析工具的可用性和有效性。     

Explaining software developer acceptance of methodologies: a comparison of five theoretical models

Many organizations attempt to deploy methodologies intended to improve software development processes. However, resistance by individual software developers against using such methodologies often obstructs their successful deployment. To better explain why individual developers accept or resist methodologies, five theoretical models of individual intentions to accept information technology tools were examined. In a field study of 128 developers in a large organization that implemented a methodology, each model explained significant variance in developers' intentions to use the methodology. Similar to findings from the tool adoption context, we found that, if a methodology is not regarded as useful by developers, its prospects for successful deployment may be severely undermined. In contrast to the typical pattern of findings in a tool context, however, we found that methodology adoption intentions are driven by: 1) the presence of an organizational mandate to use the methodology, 2) the compatibility of the methodology with how developers perform their work, and 3) the opinions of developers' coworkers and supervisors toward using the methodology. Collectively, these results provide surprising new insights into why software developers accept or resist methodologies and suggest what software engineering managers might do to overcome developer resistance.     

A methodology to infer and refactor an object‐oriented model from C applications

When analyzing legacy code, generating a high‐level model of an application during the reverse engineering process helps the developers understand how the application is structured and how the dependencies relate the different software entities. Within the context of procedural programming languages (such as C), the existing approaches to get a model of the code require documentation and/or implicit knowledge that stakeholders acquire during the software building. These approaches use the code itself to build a syntactic model where we see the different software artifacts, such as variables, functions, and modules. However, there is no supporting methodology to detect and analyze if there are relationships/dependencies between those artifacts, such as which variable in a module is declared using an abstract data type described in another one, or which are the functions that are using parameters typed with an abstract data type; or any design decision taken by original developers, such as how the developer has implemented functions in different modules. On the other hand, current developers use object‐oriented (OO) paradigm to implement not only business applications but also useful methodologies/tools that allow semiautomatic analysis of any application. We must remark the legacy procedural code still has worth and is working in several industries, and as any evolving code, the developers have to be able to perform maintenance tasks minimizing the limitations offered by the language. Based on useful properties that the OO paradigm (and their supporting analysis tools) provide, such as UML models, we propose M2K as a methodology to generate a high‐level model from legacy procedural code, mainly written in Ansi C. To understand how C‐based applications were implemented is not a new problem in software reengineering. However, our contribution is based on building an OO model and suggesting different refactorings that help the developer to improve it and to eventually guide a new implementation of the target application. Specifically, the methodology builds cohesive software entities mapped from procedural code and makes the coupling between C entities explicit in the high‐level model. The result of our methodology is a set of refactored class candidates: a structure that groups a set of variables and a set of functions obtained from the C applications. Based on the class candidate model, we propose refactorings based on OO design principles to improve the design of the application. The most relevant design improvements were obtained with algorithm abstraction by applying the strategy pattern, attributes/methods relocalization, variables types generalization, and removing/renaming methods/attributes. Besides a methodology and the supporting tool, we provide 14 case studies based on real projects implemented in C, and we showed how the results validate our proposal.     

Assisting web search using query suggestion based on word similarity measure and query modification patterns

One of the useful tools offered by existing web search engines is query suggestion (QS), which assists users in formulating keyword queries by suggesting keywords that are unfamiliar to users, offering alternative queries that deviate from the original ones, and even correcting spelling errors. The design goal of QS is to enrich the web search experience of users and avoid the frustrating process of choosing controlled keywords to specify their special information needs, which releases their burden on creating web queries. Unfortunately, the algorithms or design methodologies of the QS module developed by Google, the most popular web search engine these days, is not made publicly available, which means that they cannot be duplicated by software developers to build the tool for specifically-design software systems for enterprise search, desktop search, or vertical search, to name a few. Keyword suggested by Yahoo! and Bing, another two well-known web search engines, however, are mostly popular currently-searched words, which might not meet the specific information needs of the users. These problems can be solved by WebQS, our proposed web QS approach, which provides the same mechanism offered by Google, Yahoo!, and Bing to support users in formulating keyword queries that improve the precision and recall of search results. WebQS relies on frequency of occurrence, keyword similarity measures, and modification patterns of queries in user query logs, which capture information on millions of searches conducted by millions of users, to suggest useful queries/query keywords during the user query construction process and achieve the design goal of QS. Experimental results show that WebQS performs as well as Yahoo! and Bing in terms of effectiveness and efficiency and is comparable to Google in terms of query suggestion time.     

Toward Automated Support for Transparent Interoperable Queries

Many Object-Oriented Databases (OODBs) use programming languages that predate Java, such as C++, CLOS and Smalltalk. Thus, there is a growing need for interoperating these OODBs with new applications that require Java. Unfortunately, there are few mechanisms that allow software developers to easily integrate Java applications with non-Java OODBs. Although various interoperability mechanisms have been developed over the years, these approaches have some significant drawbacks in practice. They are often difficult to use, provide little, if any, automated support, and produce software that is difficult to engineer and maintain. In this paper, we describe an interoperability approach that allows application developers to seamlessly and transparently access non-Java OODBs from Java applications. We first present JOQL, a Java-based object query language that we are developing. JOQL queries are embedded in Java applications and are used to perform queries over C++-based OODBs. We also describe an accompanying toolset that processes Java programs containing JOQL queries. The toolset produces all the necessary code allowing Java applications to access and manipulate a C++-based OODB. As a result, application developers are free to work in Java without having to concern themselves with the details of interoperating with C++. Finally, we provide some preliminary experimental data that demonstrates our approach incurs a reasonable performance overhead.     

Heuristic expansion of feature mappings in evolving program families

Establishing explicit mappings between features and their implementation elements in code is one of the critical factors to maintain and evolve software systems successfully. This is especially important when developers have to evolve program families, which have evolved from one single core system to similar but different systems to accommodate various requirements from customers. Many techniques and tools have emerged to assist developers in the feature mapping activity. However, existing techniques and tools for feature mapping are limited as they operate on a single program version individually. Additionally, existing approaches are limited to recover features on demand, that is, developers have to run the tools for each family member version individually. In this paper, we propose a cohesive suite of five mapping heuristics addressing those two limitations. These heuristics explore the evolution history of the family members in order to expand feature mappings in evolving program families. The expansion refers to the action of automatically generating the feature mappings for each family member version by systematically considering its previous change history. The mapping expansion starts from seed mappings and continually tracks the features of the program family, thus eliminating the need of on demand algorithms. Additionally, we present the MapHist tool that provides support to the application of the proposed heuristics. We evaluate the accuracy of our heuristics through two evolving program families from our industrial partners. Copyright © 2013 John Wiley & Sons, Ltd.     

Supporting simultaneous versions for software evolution assessment

When reengineering software systems, maintainers should be able to assess and compare multiple change scenarios for a given goal, so as to choose the most pertinent one. Because they implicitly consider one single working copy, revision control systems do not scale up well to perform simultaneous analyses of multiple versions of systems. We designed Orion, an interactive prototyping tool for reengineering, to simulate changes and compare their impact on multiple versions of software source code models. Our approach offers an interactive simulation of changes, reuses existing assessment tools, and has the ability to hold multiple and branching versions simultaneously in memory. Specifically, we devise an infrastructure which optimizes memory usage of multiple versions for large models. This infrastructure uses an extension of the FAMIX source code meta-model but it is not limited to source code analysis tools since it can be applied to models in general. In this paper, we validate our approach by running benchmarks on memory usage and computation time of model queries on large models. Our benchmarks show that the Orion approach scales up well in terms of memory usage, while the current implementation could be optimized to lower its computation time. We also report on two large case studies on which we applied Orion.     

Javigator:Java代码导读及分析管理工具的设计

为克服阅读理解源代码的困难,根据现有的软件开发辅助工具的特点,对源代码导读和管理工具进行了深入的分析和研究.针对业界大量使用的Java语言,设计并实现了一个旨在以Java源代码导读为主、并具代码抽取等分析管理功能的交互式软件开发辅助工具Javigator.作为Eclipse的插件,能帮助开发人员快速地阅读理解Java源程序,并支撑对关注代码的快速测试及代码重用.Javigator使用灵活方便,可明显降低开发中理解及管理源代码的难度,提高开发的效率.     

A `crystal ball' for software liability

Software developers are living in a liability grace period, but it won't last. To adequately insure themselves against potential liability, developers need tools to identify worst-case scenarios and help them quantify the risks associated with a piece of software. For assessing such risks associated with software, the authors recommend fault injection, which provides worst-case predictions about how badly a piece of code might behave and how frequently it might behave that way. By contrast, software testing states how good software is. But even correct code can have “bad days”, when external influences keep it from working as desired. Fault injection is arguably the next best thing to having a crystal ball, and it certainly beats facing the future with no predictions at all. It should be a regular part of risk assessment. The greatest benefit from fault injection occurs when a piece of software does not tolerate injected anomalies. False optimism gives way to the only honest claim-that the software presents risks