一种基于图数据库的代码结构解析与搜索方法

软件复用是在软件开发中避免重复劳动的解决方案.在复用一个已有的软件项目时,软件开发人员通常需要理解某些代码元素以及其间的关联关系,称之为代码结构.软件开发人员一般通过浏览软件源代码的方式理解代码结构.由于源代码往往规模较大且结构复杂,理解代码结构通常会耗费大量的时间与精力.因此,将软件开发人员想要理解的代码结构自动、清晰地展示出来是很有帮助的.提出一种基于图数据库的代码结构解析与搜索方法以实现这一目的.这一方法可对软件的代码结构进行解析,并在图数据库中对其进行有效的组织和管理.搜索时,软件开发人员输入自然语言查询语句,该方法中的搜索机制会分析查询语句,并从图数据库中截取出与其相对应的代码结构进行展示.该方法具有高度的可扩展性：不同粒度的结点与多样化的关联关系可以容易地存储进图数据库中,且面向不同搜索目的的代码结构搜索算法亦可以容易地集成进搜索机制中.这一方法已在相应的工具中得到了实现,其有效性在一个商业案例研究中得到了验证.     

A field study of how developers locate features in source code

Our current understanding of how programmers perform feature location during software maintenance is based on controlled studies or interviews, which are inherently limited in size, scope and realism. Replicating controlled studies in the field can both explore the findings of these studies in wider contexts and study new factors that have not been previously encountered in the laboratory setting. In this paper, we report on a field study about how software developers perform feature location within source code during their daily development activities. Our study is based on two complementary field data sets: one that reflects complete IDE activity of 67 professional developers over approximately one month, and the other that reflects usage of an IR-based code search tool by nearly 600 developers. Analyzing this data, we report results on how often developers use which type of code search tools, on the types of queries and retreival strategies used by developers, and on patterns of developer feature location behavior following code search. The results of the study suggest that there is (1) a need for helping developers to devise better code search queries; (2) a lack of adoption of niche code search tools; (3) a need for code search tool to handle both lookup and exploratory queries; and (4) a need for better integration between code search, structured navigation, and debugging tools in feature location tasks.     

一种高效的源代码搜索算法

多年来的软件开发积累了大量的源代码,同时不少代码搜索工具也被开发出来,但是,现有的工具都不够精确,因而很少有人使用。本文提出一种高效的源代码搜索算法,通过识别查询语句与API的关系,以提高代码搜索的准确性。基于该算法,本文实现一个针对C#源代码的搜索工具,并通过客观实验与用户调研对算法进行评估。实验结果表明本文提出的搜索算法是十分有效的。      

Approximate Structural Context Matching: An Approach to Recommend Relevant Examples

When coding to an application programming interface (API), developers often encounter difficulties, unsure of which class to subclass, which objects to instantiate, and which methods to call. Example source code that demonstrates the use of the API can help developers make progress on their task. This paper describes an approach to provide such examples in which the structure of the source code that the developer is writing is matched heuristically to a repository of source code that uses the API. The structural context needed to query the repository is extracted automatically from the code, freeing the developer from learning a query language or from writing their code in a particular style. The repository is generated automatically from existing applications, avoiding the need for handcrafted examples. We demonstrate that the approach is effective, efficient, and more reliable than traditional alternatives through four empirical studies     

Exploring architecture blueprints for prioritizing critical code anomalies: Experiences and tool support

The manifestation of code anomalies in software systems often indicates symptoms of architecture degradation. Several approaches have been proposed to detect such anomalies in the source code. However, most of them fail to assist developers in prioritizing anomalies harmful to the software architecture of a system. This article presents an investigation on how developers, when supported by architecture blueprints, are able to prioritize architecturally relevant code anomalies. First, we performed a controlled experiment where participants explored both blueprints and source code to reveal architecturally relevant code anomalies. Although the use of blueprints has the potential to improve code anomaly prioritization, the participants often made several mistakes. We found these mistakes might occur because developers miss relationships between implementation and blueprint elements when they prioritize anomalies in an ad hoc manner. Furthermore, the time spent on the prioritization process was considerably high. Aiming to improve the accuracy and effectiveness of the process, we provided means to automate the prioritization process. In particular, we explored 3 prioritization criteria, which establish different ways of relating the blueprint elements with code anomalies. These criteria were implemented in the JSpIRIT tool. The approach was evaluated in the context of 2 applications with satisfactory precision results.     

Privacy and security constraints for code contributions

In collaborative software development, developers submit their contributions to repositories that are used to integrate code from various collaborators. To avoid privacy and security issues, code contributions are often reviewed before integration. Although careful manual code review can detect such issues, it might be time-consuming, expensive, and error-prone. Automatic analysis tools can also detect privacy and security issues, but they often demand significant developer effort, or are domain specific, considering fixed framework specific vulnerability sources and sinks. To reduce these problems, in this paper we propose the Salvum policy language to support the specification of constraints that help to protect sensitive information from being inadvertently accessed by specific code contributions. We implement a tool that automatically checks Salvum policies for systems of different technical domains. We also investigate whether Salvum can find policy violations for a number of open-source projects. We find evidence that Salvum helps to detect violations even for well-supported and highly active projects. Moreover, our tool helps to find 80 violations in benchmark projects.     

To var or not to var: how do C# developers use and misuse implicit and explicit typing?

When implicit typing with the “var” keyword was introduced into C#, it prompted contradictory opinions among developers. This paper starts by explaining the difference between implicit and explicit typing and then provides an overview of developers’ opinions and guidelines that are available online. This paper then reports on the results of a study that investigated how C# developers use and misuse implicit and explicit typing. This study involved analyzing the source code of 10 different open-source software projects including more than 16,500,000 lines of code and more than 930,000 variables. This study investigated to what extent developers use a form of typing that affects the readability of a variable’s type and the length of its declaration. It also investigated whether or not there is an adoption of a consistent set of guidelines in general and across each software project. A tool called “Code Analysis and Refactoring Engine for C#” (Care#) was developed and used to conduct the code analysis for this study.

     

An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks

Much of software developers' time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an unfamiliar program and asked to work on two debugging tasks and three enhancement tasks for 70 minutes. The study found that developers interleaved three activities. They began by searching for relevant code both manually and using search tools; however, they based their searches on limited and misrepresentative cues in the code, environment, and executing program, often leading to failed searches. When developers found relevant code, they followed its incoming and outgoing dependencies, often returning to it and navigating its other dependencies; while doing so, however, Eclipse's navigational tools caused significant overhead. Developers collected code and other information that they believed would be necessary to edit, duplicate, or otherwise refer to later by encoding it in the interactive state of Eclipse's package explorer, file tabs, and scroll bars. However, developers lost track of relevant code as these interfaces were used for other tasks, and developers were forced to find it again. These issues caused developers to spend, on average, 35 percent of their time performing the mechanics of navigation within and between source files. These observations suggest a new model of program understanding grounded in theories of information foraging and suggest ideas for tools that help developers seek, relate, and collect information in a more effective and explicit manner     

TipTracer:基于安全提示的安卓应用通用漏洞检测框架

为了使开发者能安全准确地使用第三库接口,库设计者提供了各种类型的安全提示(安全规约),进而保护应用程序免受因库函数的误用而造成的安全攻击.然而,研究表明:开发者经常性不遵守这些安全规约,导致应用程序中引入了各种各样的安全漏洞.为了评估该问题的影响与规模,进行了系统性的、大规模的对安全规约在安卓应用程序中违反情况的研究.结果表明:已有的安全规约由于不精确的描述、误导性的代码示例、错误的默认设置、碎片化以及缺少强制性检查等原因而大大影响了其在实际运用中的有效性.为了使开发者能更好地遵守安全规约,提出了TipTracer,一个自动化的通用漏洞分析框架.TipTracer主要包含2个部分:1)TipTracer定义了一个能形式化描述安全规约的安全性语言,并利用该语言对已知的安全规约进行形式化表述;2)TipTracer实现了一个静态代码分析器,用于检查应用程序是否满足安全规约.最后,通过大规模的实验分析,证明了TipTracer能有效且准确地对大规模的真实应用程序进行安全性分析.     

Improving software security with static automated code analysis in an industry setting

Software security can be improved by identifying and correcting vulnerabilities. In order to reduce the cost of rework, vulnerabilities should be detected as early and efficiently as possible. Static automated code analysis is an approach for early detection. So far, only few empirical studies have been conducted in an industrial context to evaluate static automated code analysis. A case study was conducted to evaluate static code analysis in industry focusing on defect detection capability, deployment, and usage of static automated code analysis with a focus on software security. We identified that the tool was capable of detecting memory related vulnerabilities, but few vulnerabilities of other types. The deployment of the tool played an important role in its success as an early vulnerability detector, but also the developers perception of the tools merit. Classifying the warnings from the tool was harder for the developers than to correct them. The correction of false positives in some cases created new vulnerabilities in previously safe code. With regard to defect detection ability, we conclude that static code analysis is able to identify vulnerabilities in different categories. In terms of deployment, we conclude that the tool should be integrated with bug reporting systems, and developers need to share the responsibility for classifying and reporting warnings. With regard to tool usage by developers, we propose to use multiple persons (at least two) in classifying a warning. The same goes for making the decision of how to act based on the warning. Copyright © 2012 John Wiley & Sons, Ltd.     

Design and evaluation of a multi-recommendation system for local code search

Searching for relevant code in the local code base is a common activity during software maintenance. However, previous research indicates that 88% of manually composed search queries retrieve no relevant results. One reason that many searches fail is existing search tools’ dependence on string matching algorithms, which cannot find semantically related code. To solve this problem by helping developers compose better queries, researchers have proposed numerous query recommendation techniques, relying on a variety of dictionaries and algorithms. However, few of these techniques are empirically evaluated by usage data from real-world developers. To fill this gap, we designed a multi-recommendation system that relies on the cooperation between several query recommendation techniques. We implemented and deployed this recommendation system within the Sando code search tool and conducted a longitudinal field study. Our study shows that over 34% of all queries were adopted from recommendation; and recommended queries retrieved results 11% more often than manual queries.     

J2EE平台上的HTML电子表格工具的设计

随着Internet的发展,Web应用也得到了快速发展和广泛普及.同时这也给Web应用开发提出了新的要求.与传统应用程序开发相比,Web应用开发具有如下特点：开发周期短,开发成本高,实现技术复杂.为了适应Web应用开发的新特点,开发人员需要一种可以快速开发Web应用的工具.本文介绍了一种J2EE平台上动态HT-ML表格的设计和实现.开发者可以使用该工具定义一个和数据库相联系的动态HTML表格.这个工具可以根据用户的定义自动生成相应的Servlet代码.     

What do developers search for on the web?

Developers commonly make use of a web search engine such as Google to locate online resources to improve their productivity. A better understanding of what developers search for could help us understand their behaviors and the problems that they meet during the software development process. Unfortunately, we have a limited understanding of what developers frequently search for and of the search tasks that they often find challenging. To address this gap, we collected search queries from 60 developers, surveyed 235 software engineers from more than 21 countries across five continents. In particular, we asked our survey participants to rate the frequency and difficulty of 34 search tasks which are grouped along the following seven dimensions: general search, debugging and bug fixing, programming, third party code reuse, tools, database, and testing. We find that searching for explanations for unknown terminologies, explanations for exceptions/error messages (e.g., HTTP 404), reusable code snippets, solutions to common programming bugs, and suitable third-party libraries/services are the most frequent search tasks that developers perform, while searching for solutions to performance bugs, solutions to multi-threading bugs, public datasets to test newly developed algorithms or systems, reusable code snippets, best industrial practices, database optimization solutions, solutions to security bugs, and solutions to software configuration bugs are the most difficult search tasks that developers consider. Our study sheds light as to why practitioners often perform some of these tasks and why they find some of them to be challenging. We also discuss the implications of our findings to future research in several research areas, e.g., code search engines, domain-specific search engines, and automated generation and refinement of search queries.     

Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining

Many software production processes advocate rigorous development testing alongside functional code writing, which implies that both test code and production code should co-evolve. To gain insight in the nature of this co-evolution, this paper proposes three views (realized by a tool called TeMo) that combine information from a software project’s versioning system, the size of the various artifacts and the test coverage reports. We validate these views against two open source and one industrial software project and evaluate our results both with the help of log messages, code inspections and the original developers of the software system. With these views we could recognize different co-evolution scenarios (i.e., synchronous and phased) and make relevant observations for both developers as well as test engineers.     

Predicting source code changes by mining change history

Software developers are often faced with modification tasks that involve source which is spread across a code base. Some dependencies between source code, such as those between source code written in different languages, are difficult to determine using existing static and dynamic analyses. To augment existing analyses and to help developers identify relevant source code during a modification task, we have developed an approach that applies data mining techniques to determine change patterns - sets of files that were changed together frequently in the past - from the change history of the code base. Our hypothesis is that the change patterns can be used to recommend potentially relevant source code to a developer performing a modification task. We show that this approach can reveal valuable dependencies by applying the approach to the Eclipse and Mozilla open source projects and by evaluating the predictability and interestingness of the recommendations produced for actual modification tasks on these systems.     

A data-oriented profiler to assist in data partitioning and distribution for heterogeneous memory in HPC

Profiling is of great assistance in understanding and optimizing an application’s behavior. Today’s profiling techniques help developers focus on the pieces of code leading to the highest penalties according to a given performance metric. In this paper we describe a profiling tool we have developed by extending the Valgrind framework and one of its tools: Callgrind. Our extended profiling tool provides new object-differentiated profiling capabilities that help software developers and hardware designers (1) understand access patterns, (2) identify unexpected access patterns, and (3) determine whether a particular memory object is consistently featuring a troublesome access pattern. We use this tool to assist in the partition of big data objects so that smaller portions of them can be placed in small, fast memory subsystems of heterogeneous memory systems such as scratchpad memories. We showcase the potential benefits of this technique by means of the XSBench miniapplication from the CESAR codesign project. The benefits include being able to identify the optimal portion of data to be placed in a small scratchpad memory, leading to more than 19% performance improvement, compared with nonassisted partitioning approaches, in our proposed scratchpad-equipped compute node.     

Symbolic modular deadlock analysis

Methods in object-oriented concurrent libraries often encapsulate internal synchronization details. As a result of information hiding, clients calling the library methods may cause thread safety violations by invoking methods in an unsafe manner. This is frequently a cause of deadlocks. Given a concurrent library, we present a technique for inferring interface contracts that specify permissible concurrent method calls and patterns of aliasing among method arguments. In this work, we focus on deriving contracts that guarantee deadlock-free execution for the methods in the library. The contracts also help client developers by documenting required assumptions about the library methods. Alternatively, the contracts can be statically enforced in the client code to detect potential deadlocks in the client. Our technique combines static analysis with a symbolic encoding scheme for tracking lock dependencies, allowing us to synthesize contracts using a SMT solver. Additionally, we investigate extensions of our technique to reason about deadlocks in libraries that employ signaling primitives such as wait-notify for cooperative synchronization. Our prototype tool analyzes over a million lines of code for some widely-used Java libraries within an hour, thus demonstrating its scalability and efficiency. Furthermore, the contracts inferred by our approach have been able to pinpoint real deadlocks in clients, i.e. deadlocks that have been a part of bug-reports filed by users and developers of client code.     

The real costs of aspect-oriented programming

As a software developer, one of my principal objectives is to deliver the highest quality software possible. Unfortunately, as a human being, my abilities are also my limitations, and I always make mistakes of one kind or another. To remedy this, I use tools to help ameliorate as many of my limitations as possible. One of rny concerns is to understand how using a particular tool affects my ability to meet objectives. How does it help with the various software development tasks? What does it Cost me to use a particular tool? Do the benefits outweigh the costs? If so, what degree is the benefit? If not, then the tool is of little use. There must be a significant benefit gain to justify using a tool. The essence of aspect-oriented programming is that crosscutting concerns are factored into distinct abstractions (that is, aspects) that are solely responsible for a particular crosscutting concern. This raises the system's modularity and increases cohesion of the abstractions that are primary concerns. Researchers and software developers must understand and address the various significant weaknesses before AOP's benefits can be realized.     

Performance Metrics for All

When protocol developers or network engineers seek to understand their situation, performance measurements are one tool in the tool box that can help. But what if the technology is new and no relevant performance metrics exist? This article describes a process for developing performance metrics with sufficient detail for them to become industry standards.