Locating features in source code

Understanding the implementation of a certain feature of a system requires identification of the computational units of the system that contribute to this feature. In many cases, the mapping of features to the source code is poorly documented. In this paper, we present a semiautomatic technique that reconstructs the mapping for features that are triggered by the user and exhibit an observable behavior. The mapping is in general not injective; that is, a computational unit may contribute to several features. Our technique allows for the distinction between general and specific computational units with respect to a given set of features. For a set of features, it also identifies jointly and distinctly required computational units. The presented technique combines dynamic and static analyses to rapidly focus on the system's parts that relate to a specific set of features. Dynamic information is gathered based on a set of scenarios invoking the features. Rather than assuming a one-to-one correspondence between features and scenarios as in earlier work, we can now handle scenarios that invoke many features. Furthermore, we show how our method allows incremental exploration of features while preserving the "mental map" the analyst has gained through the analysis.     

Discovery of architectural layers and measurement of layering violations in source code

The layers architectural pattern has been widely adopted by the developer community in order to build large software systems. In reality, as the system evolves over time, rarely does the system remain conformed to the intended layers pattern, causing a significant degradation of the system maintainability. As a part of re-factoring such a system, practitioners often undertake a mostly manual exercise to discover the intended layers and organize the modules into these layers. In this paper, we present a method for semi-automatically detecting layers in the system and propose a quantitative measurement to compute the amount of non-conformance of the system from the set of layered design principles. We have applied the layer detection method and the non-conformance measurement on a set of open source and proprietary enterprise applications.     

Addendum to "locating features in source code"

For original paper by T. Eisenbarth et al. see ibid., vol.29, no.3, p.210-24 (2003).We compare three approaches that apply formal concept analysis on execution profiles. This survey extends the discourse of related research by Bojic and Velasevic (2000).     

A field study of how developers locate features in source code

Our current understanding of how programmers perform feature location during software maintenance is based on controlled studies or interviews, which are inherently limited in size, scope and realism. Replicating controlled studies in the field can both explore the findings of these studies in wider contexts and study new factors that have not been previously encountered in the laboratory setting. In this paper, we report on a field study about how software developers perform feature location within source code during their daily development activities. Our study is based on two complementary field data sets: one that reflects complete IDE activity of 67 professional developers over approximately one month, and the other that reflects usage of an IR-based code search tool by nearly 600 developers. Analyzing this data, we report results on how often developers use which type of code search tools, on the types of queries and retreival strategies used by developers, and on patterns of developer feature location behavior following code search. The results of the study suggest that there is (1) a need for helping developers to devise better code search queries; (2) a lack of adoption of niche code search tools; (3) a need for code search tool to handle both lookup and exploratory queries; and (4) a need for better integration between code search, structured navigation, and debugging tools in feature location tasks.     

Examining the significance of high-level programming features in source code author classification

The use of Source Code Author Profiles (SCAP) represents a new, highly accurate approach to source code authorship identification that is, unlike previous methods, language independent. While accuracy is clearly a crucial requirement of any author identification method, in cases of litigation regarding authorship, plagiarism, and so on, there is also a need to know why it is claimed that a piece of code is written by a particular author. What is it about that piece of code that suggests a particular author? What features in the code make one author more likely than another? In this study, we describe a means of identifying the high-level features that contribute to source code authorship identification using as a tool the SCAP method. A variety of features are considered for Java and Common Lisp and the importance of each feature in determining authorship is measured through a sequence of experiments in which we remove one feature at a time. The results show that, for these programs, comments, layout features and package-related naming influence classification accuracy whereas user-defined naming, an obvious programmer related feature, does not appear to influence accuracy. A comparison is also made between the relative feature contributions in programs written in the two languages.     

Re-creation of source code from reverse polish form

A method is described where, by adding redundant operators when compiling into reverse Polish notation, it is possible to re-create source code in exactly its original form. An algorithm for performing this re-creation is outlined. The method is useful for incremental compilers since it permits partial (and in some cases total) compilation without the need to maintain a copy of the original program.     

程序源代码中的切片摘要提取及在搜索中的应用

在编写程序代码的过程中,软件工程师常常花费大量的时间来搜索和重用代码,为了快速和准确搜索源代码,提高源代码重用效率,开展了程序源代码切片摘要获取方法研究。通过分析源代码的结构特征,给出了变量切片的定义。选取与函数代码段核心功能密切相关的变量,联合其变量切片形成切片摘要,以反映源代码的结构和功能信息。然后分别设计了变量切片算法和切片摘要提取算法,并且将切片摘要应用于开源代码搜索引擎中。应用实验表明:本文提出的方法具有可行性,且较传统搜索方法在搜索精度上有较大的提高。     

Coherent clusters in source code

This paper presents the results of a large scale empirical study of coherent dependence clusters. All statements in a coherent dependence cluster depend upon the same set of statements and affect the same set of statements; a coherent cluster's statements have ‘coherent’ shared backward and forward dependence. We introduce an approximation to efficiently locate coherent clusters and show that it has a minimum precision of 97.76%. Our empirical study also finds that, despite their tight coherence constraints, coherent dependence clusters are in abundance: 23 of the 30 programs studied have coherent clusters that contain at least 10% of the whole program. Studying patterns of clustering in these programs reveals that most programs contain multiple substantial coherent clusters. A series of subsequent case studies uncover that all clusters of significant size map to a logical functionality and correspond to a program structure. For example, we show that for the program acct, the top five coherent clusters all map to specific, yet otherwise non-obvious, functionality. Cluster visualization also brings out subtle deficiencies in program structure and identifies potential refactoring candidates. A study of inter-cluster dependence is used to highlight how coherent clusters are connected to each other, revealing higher-level structures, which can be used in reverse engineering. Finally, studies are presented to illustrate how clusters are not correlated with program faults as they remain stable during most system evolution.     

An empirical study of the textual similarity between source code and source code summaries

Source code documentation often contains summaries of source code written by authors. Recently, automatic source code summarization tools have emerged that generate summaries without requiring author intervention. These summaries are designed for readers to be able to understand the high-level concepts of the source code. Unfortunately, there is no agreed upon understanding of what makes up a “good summary.” This paper presents an empirical study examining summaries of source code written by authors, readers, and automatic source code summarization tools. This empirical study examines the textual similarity between source code and summaries of source code using Short Text Semantic Similarity metrics. We found that readers use source code in their summaries more than authors do. Additionally, this study finds that accuracy of a human written summary can be estimated by the textual similarity of that summary to the source code.     

More on the re-creation of source code from reverse polish

A convenient way of processing BASIC and other interactive, incremental languages is to maintain the program in reverse Polish notation or the like, and to re-create the source from this if needed. The properties of various algorithms for performing this re-creation are examined, and a new algorithm, offering good generality and conciseness, is presented.     

Using convert to transform source code

There are a number of reasons why one might wish to transform the source code of an operational program:

• 1 To make the program conform to a standard layout.
• 2 To make the program conform to syntax and semantics standards.
• 3 To improve the performance of the program.

The primary advantage of transforming source code into a standard form is that the programs become easier to maintain. The cost-benefit of standardization is thus realized at a later stage in the lifetime of the programs. Alternatively the cost-benefit of improving performance is immediate. The desirability of transforming source code is affected by several parameters:

• 1 The benefit to be realized from transformation.
• 2 The cost of transformation.
• 3 The time involved in transformation.
• 4 The risk associated with transformation.

If the benefit of transformation is significant, the cost, time and risk associated with the process can be considerably reduced by automating the process. In this paper, application of the CONVERT language to the transformation problem is discussed. CONVERT was developed as a vehicle for writing automatic language and dialect converters. Clearly, the features useful for converters are also applicable when the application involves transformation of source code.     

A note on recreating source code from the reverse polish form

Algorithms are presented which recreate infix form from reverse Polish form of algebraic expressions. The use of these algorithms in incremental compilers is discussed.     

Software model checking without source code

We present a framework, called air, for verifying safety properties of assembly language programs via software model checking. air extends the applicability of predicate abstraction and counterexample guided abstraction refinement to the automated verification of low-level software. By working at the assembly level, air allows verification of programs for which source code is unavailable—such as legacy and COTS software—and programs that use features—such as pointers, structures, and object-orientation—that are problematic for source-level software verification tools. In addition, air makes no assumptions about the underlying compiler technology. We have implemented a prototype of air and present encouraging results on several non-trivial examples.     

Recording concerns in source code using annotations

A concern can be characterized as a developer׳s intent behind a piece of code, often not explicitly captured in it. We discuss a technique of recording concerns using source code annotations (concern annotations). Using two studies and two controlled experiments, we seek to answer the following 3 research questions: (1) Do programmers׳ mental models overlap? (2) How do developers use shared concern annotations when they are available? (3) Does using annotations created by others improve program comprehension and maintenance correctness, time and confidence? The first study shows that developers׳ mental models, recorded using concern annotations, overlap and thus can be shared. The second study shows that shared concern annotations can be used during program comprehension for the following purposes: hypotheses confirmation, feature location, obtaining new knowledge, finding relationships and maintenance notes. The first controlled experiment with students showed that the presence of annotations significantly reduced program comprehension and maintenance time by 34%. The second controlled experiment was a differentiated replication of the first one, focused on industrial developers. It showed a 33% significant improvement in correctness. We conclude that concern annotations are a viable way to share developers׳ thoughts.     

An extensible system for source code analysis

Constructing code analyzers may be costly and error prone if inadequate technologies and tools are used. If they are written in a conventional programming language, for instance, several thousand lines of code may be required even for relatively simple analyses. One way of facilitating the development of code analyzers is to define a very high-level domain-oriented language and implement an application generator that creates the analyzers from the specification of the analyses they are intended to perform. This paper presents a system for developing code analyzers that uses a database to store both a no-loss fine-grained intermediate representation and the results of the analyses. The system uses an algebraic representation, called F(p), as the user-visible intermediate representation. Analyzers are specified in a declarative language, called F(p)-l, which enables an analysis to be specified in the form of a traversal of an algebraic expression, with access to, and storage of, the database information the algebraic expression indices. A foreign language interface allows the analyzers to be embedded in C programs. This is useful for implementing the user interface of an analyzer, for example, or to facilitate interoperation of the generated analyzers with pre-existing tools. The paper evaluates the strengths and limitations of the proposed system, and compares it to other related approaches     

Extracting discriminative color features for face recognition

This paper presents a discriminative color features (DCF) method, which applies a simple yet effective color model, a novel similarity measure, and effective color feature extraction methods, for improving face recognition performance. First, the new color model is constructed according to the principle of Ockham’s razor from a number of available models that take advantage of the subtraction of the primary colors for boosting pattern recognition performance. Second, the novel similarity measure integrates both the angular and the distance information for improving upon the broadly applied similarity measures. Finally, the discriminative color features are extracted from a compact color image representation by means of discriminant analysis with enhanced generalization capabilities. Experiments on the Face Recognition Grand Challenge (FRGC) version 2 Experiment 4, which contains 12,776 training images, 16,028 controlled target images, and 8,014 uncontrolled query images, show the feasibility of the proposed method.     

Analysis of source code metrics from ns-2 and ns-3 network simulators

Ns-2 and its successor ns-3 are discrete-event simulators which are closely related to each other as they share common background, concepts and similar aims. Ns-3 is still under development, but it offers some interesting characteristics for developers while ns-2 still has a large user base. While other studies have compared different network simulators, focusing on performance measurements, in this paper we adopted a different approach by focusing on technical characteristics and using software metrics to obtain useful conclusions. We chose ns-2 and ns-3 for our case study because of the popularity of the former in research and the increasing use of the latter. This reflects the current situation where ns-3 has emerged as a viable alternative to ns-2 due to its features and design. The paper assesses the current state of both projects and their respective evolution supported by the measurements obtained from a broad set of software metrics. By considering other qualitative characteristics we obtained a summary of technical features of both simulators including, architectural design, software dependencies or documentation policies.     

Automatic code features extraction using bio-inspired algorithms

The number of malicious applications that appear everyday has reached beyond any manual analysis. In the attempt to spread beyond personal computers, malware authors use new platforms like Android, iOS and .NET. The later has the advantage of being present on both desktop computers running Windows Vista or later and also on Windows Phone devices. Previous studies in the malware classification field have used the concept of OpCode \(n\) -grams. These are sequences of consecutive operation codes, that can be extracted from any type of application. In this paper we will show an improvement to this method, by eliminating some of the OpCodes, in order to get better classification results. The OpCodes selection is performed by two bio-inspired algorithms. First, a fitness function was designed, that measured how well we can detect some clusters of methods. Then, we encoded a possible solution as a chromosome in a Genetic Algorithm and as a particle, in Particle Swarm Optimization. Both methods found good OpCodes subsets that were successful in detecting clusters from the cross-validation tests. The results presented in this paper show that biology can be a source of inspiration not only for computer viruses but also for new methods to combat them.     

Analyzing the co-evolution of comments and source code

Source code comments are a valuable instrument to preserve design decisions and to communicate the intent of the code to programmers and maintainers. Nevertheless, commenting source code and keeping comments up-to-date is often neglected for reasons of time or programmers obliviousness. In this paper, we investigate the question whether developers comment their code and to what extent they add comments or adapt them when they evolve the code. We present an approach to associate comments with source code entities to track their co-evolution over multiple versions. A set of heuristics are used to decide whether a comment is associated with its preceding or its succeeding source code entity. We analyzed the co-evolution of code and comments in eight different open source and closed source software systems. We found with statistical significance that (1) the relative amount of comments and source code grows at about the same rate; (2) the type of a source code entity, such as a method declaration or an if-statement, has a significant influence on whether or not it gets commented; (3) in six out of the eight systems, code and comments co-evolve in 90% of the cases; and (4) surprisingly, API changes and comments do not co-evolve but they are re-documented in a later revision. As a result, our approach enables a quantitative assessment of the commenting process in a software system. We can, therefore, leverage the results to provide feedback during development to increase the awareness of when to add comments or when to adapt comments because of source code changes.