Does syntax highlighting help programming novices?

Program comprehension is an important skill for programmers – extending and debugging existing source code is part of the daily routine. Syntax highlighting is one of the most common tools used to support developers in understanding algorithms. However, most research in this area originates from a time when programmers used a completely different tool chain. We examined the influence of syntax highlighting on novices’ ability to comprehend source code. Additional analyses cover the influence of task type and programming experience on the code comprehension ability itself and its relation to syntax highlighting. We conducted a controlled experiment with 390 undergraduate students in an introductory Java programming course. We measured the correctness with which they solved small coding tasks. Each test subject received some tasks with syntax highlighting and some without. The data provided no evidence that syntax highlighting improves novices’ ability to comprehend source code. There are very few similar experiments and it is unclear as of yet which factors impact the effectiveness of syntax highlighting. One major limitation may be the types of tasks chosen for this experiment. The results suggest that syntax highlighting squanders a feedback channel from the IDE to the programmer that can be used more effectively.     

An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks

Much of software developers' time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an unfamiliar program and asked to work on two debugging tasks and three enhancement tasks for 70 minutes. The study found that developers interleaved three activities. They began by searching for relevant code both manually and using search tools; however, they based their searches on limited and misrepresentative cues in the code, environment, and executing program, often leading to failed searches. When developers found relevant code, they followed its incoming and outgoing dependencies, often returning to it and navigating its other dependencies; while doing so, however, Eclipse's navigational tools caused significant overhead. Developers collected code and other information that they believed would be necessary to edit, duplicate, or otherwise refer to later by encoding it in the interactive state of Eclipse's package explorer, file tabs, and scroll bars. However, developers lost track of relevant code as these interfaces were used for other tasks, and developers were forced to find it again. These issues caused developers to spend, on average, 35 percent of their time performing the mechanics of navigation within and between source files. These observations suggest a new model of program understanding grounded in theories of information foraging and suggest ideas for tools that help developers seek, relate, and collect information in a more effective and explicit manner     

基于神经网络的自动源代码摘要技术综述

源代码的摘要可以帮助软件开发人员快速地理解代码,帮助维护人员更快地完成维护任务.但是,手工编写摘要代价高、效率低,因此人们试图利用计算机自动地为源代码生成摘要.近年来,基于神经网络的代码摘要技术成为自动源代码摘要研究的主流技术和软件工程领域的研究热点.首先阐述了代码摘要的概念和自动代码摘要的定义,回顾了自动代码摘要技术...     

Awareness of Organizational Expertise

Comprehension of computer programs involves identifying important program parts and inferring relationships between them. The ability to comprehend a computer program is a skill that begins its development in the novice programmer and reaches maturity in the expert programmer. This research examined the beginning of this process, that of comprehension of computer programs by novice programmers. The mental representations of the program text that novices form, which indicate the comprehension strategies being used, were examined. In the first study, 80 novice programmers were tested on their comprehension of short program segments. The results suggested that novices form detailed, concrete mental representations of the program text, supporting work that has previously been done with novice comprehension. Their mental representations were primarily procedural in nature, with little or no modeling using real‐world referents. In a second study, the upper and lower quartile comprehenders from Study 1 were tested on their comprehension of a longer program. Results supported the conclusions from Study 1 in that the novices tended towards detailed representations of the program text with little real‐world reference. However, the comprehension strategies used by high comprehenders differed substantially from those used by low comprehenders. Results indicated that the more advanced novices were using more abstract concepts in their representations, although their abstractions were detailed in nature.     

Using Developer Activity Data to Enhance Awareness during Collaborative Software Development

Software development is a global activity unconstrained by the bounds of time and space. A major effect of this increasing scale and distribution is that the shared understanding that developers previously acquired by formal and informal face-to-face meetings is difficult to obtain. This paper proposes a shared awareness model that uses information gathered automatically from developer IDE interactions to make explicit orderings of tasks, artefacts and developers that are relevant to particular work contexts in collaborative, and potentially distributed, software development projects. The research findings suggest that such a model can be used to: identify entities (developers, tasks, artefacts) most associated with a particular work context in a software development project; identify relevance relationships amongst tasks, developers and artefacts e.g. which developers and artefacts are currently most relevant to a task or which developers have contributed to a task over time; and, can be used to identify potential bottlenecks in a project through a ‘social graph’ view. Furthermore, this awareness information is captured and provided as developers work in different locations and at different times.     

The impacts of function extraction technology on program comprehension: A controlled experiment

Program comprehension is a critical, time-consuming, and highly error-prone task for software developers. Function extraction (FX) is a theory and technology that automates and supports program comprehension by calculating the semantic behaviors of programs at many levels of abstraction and displaying those behaviors in a standard, readable format in terms of the “as-built” specification of the program. In this experimental study, developers using an FX prototype tool to assist them in determining the behavior of software modules have significantly more effective program comprehension, in both increased accuracy of understanding and reduced time on task. Moreover, developers have a positive reaction toward the use of the FX technology, and use of FX does not reduce their overall comprehension of the program.     

Program comprehension of domain-specific and general-purpose languages: replication of a family of experiments using integrated development environments

Domain-specific languages (DSLs) allow developers to write code at a higher level of abstraction compared with general-purpose languages (GPLs). Developers often use DSLs to reduce the complexity of GPLs. Our previous study found that developers performed program comprehension tasks more accurately and efficiently with DSLs than with corresponding APIs in GPLs. This study replicates our previous study to validate and extend the results when developers use IDEs to perform program comprehension tasks. We performed a dependent replication of a family of experiments. We made two specific changes to the original study: (1) participants used IDEs to perform the program comprehension tasks, to address a threat to validity in the original experiment and (2) each participant performed program comprehension tasks on either DSLs or GPLs, not both as in the original experiment. The results of the replication are consistent with and expanded the results of the original study. Developers are significantly more effective and efficient in tool-based program comprehension when using a DSL than when using a corresponding API in a GPL. The results indicate that, where a DSL is available, developers will perform program comprehension better using the DSL than when using the corresponding API in a GPL.     

Procedures for Obtaining and Testing User-Selected Terminologies

Four experiments were conducted to assess procedures for obtaining and testing user-selected terms for task-specific concepts in complex, unfamiliar word-processing instructions. Experiment 1 tested user-selected terms against both user-nominated and the original technical terms. The final three experiments employed only the user-selected and technical terms. The effect of terminology on subjects' abilities to follow the instructions was evaluated by measuring errors and task completion times during the practice period. Comprehension of the instructions was assessed by performance on a transfer task. Extensive practice produced acceptable and comparable performance for all term types. However, instruction comprehension, as measured by the transfer task, was clearly influenced by terminology. User-selected word-processing terms were more understandable than both user-nominated and the original technical terms. In addition, the present study demonstrated that transfer tasks can be more sensitive (and often more appropriate) evaluations of the goodness of a term than are learning measures.     

Effects of Auditory Cues for Linear and Differential Magnification Methods on Visual Inspection Performance

The unfavorable consequence of biased attention to peripheral objects was reported with the use of a newly proposed differential magnification method by the authors. Given the favorable influence of auditory cueing on attention orientation, this study investigated the effectiveness of auditory cues in directing participants’ attention from the peripheral to foveal areas of the differential magnifying lens for improving inspection performance. Thirty‐nine Chinese undergraduates performed a simulated inspection task using a particular magnification method (between‐participant factor). Three within‐participant factors of auditory cue interval, target difficulty, and magnification power were tested. Objective performance measures of speed and accuracy and subjective evaluation using NASA task load index were collected and analyzed. The results showed that positive effects of auditory cues were found for the differential magnification methods but not for the traditional linear magnification method. Significant effects of target difficulty and magnification power were also found. It was concluded that the aid of auditory cues, which did not share the same attentional resource with the visual inspection task, proved to be an appropriate reminder for the differential magnification methods to improve participants’ visual inspection performance. Attention should also be paid to selecting an appropriate auditory cue interval for search tasks of different target difficulties. The findings of this study have implications for the design of video magnifiers in the context of multimodal human–machine interfaces.     

An empirical investigation into the design of auditory cues to enhance computer program comprehension

Decades of research have led to notable improvements in the representations used to aid human comprehension of computer programs. Much of this research has focused on visual representations, which leaves open the question of how best to design auditory representations of computer programs. While this question has particular relevance for visually impaired programmers, sighted programmers might also benefit from enhanced auditory representations of their programs. In order to investigate this question empirically, first, we introduce artifact encoding, a novel approach to rigorously measuring the comprehensibility of auditory representations of computer programs. Using this approach as a foundation, we present an experimental study that compared the comprehensibility of two alternative auditory program representations: one with lexical scoping cues that convey the nesting level of program statements, and another without such scoping cues. The results of our first experiment validate both artifact encoding and the scoping cues we used. To see whether auditory cues validated through our paradigm can aid program comprehension in a realistic task scenario, we experimentally compared programmers' ability to debug programs using three alternative environments: (1) an auditory execution environment with our empirically derived auditory cues; (2) an auditory execution environment with the current state-of-the-art auditory cues generated by a screen reader running on top of Microsoft Visual Studio; and (3) a visual version of the execution environment. The results of our second experiment showed that our comprehensible auditory cues are significantly better than the state-of-the-art, affording human performance approaching the effectiveness of visual representations within the statistical margin of error. This research contributes a novel methodology and foundational empirical data that can guide the design of effective auditory representations of computer programs.     

Taupe: Visualizing and analyzing eye-tracking data

Program comprehension is an essential part of any maintenance activity. It allows developers to build mental models of the program before undertaking any change. It has been studied by the research community for many years with the aim to devise models and tools to understand and ease this activity. Recently, researchers have introduced the use of eye-tracking devices to gather and analyze data about the developers’ cognitive processes during program comprehension. However, eye-tracking devices are not completely reliable and, thus, recorded data sometimes must be processed, filtered, or corrected. Moreover, the analysis software tools packaged with eye-tracking devices are not open-source and do not always provide extension points to seamlessly integrate new sophisticated analyses. Consequently, we develop the Taupe software system to help researchers visualize, analyze, and edit the data recorded by eye-tracking devices. The two main objectives of Taupe are compatibility and extensibility so that researchers can easily: (1) apply the system on any eye-tracking data and (2) extend the system with their own analyses. To meet our objectives, we base the development of Taupe: (1) on well-known good practices, such as design patterns and a plug-in architecture using reflection, (2) on a thorough documentation, validation, and verification process, and (3) on lessons learned from existing analysis software systems. This paper describes the context of development of Taupe, the architectural and design choices made during its development, and its documentation, validation and verification process. It also illustrates the application of Taupe in three experiments on the use of design patterns by developers during program comprehension.     

Prospective memory in the ICU: the effect of visual cues on task execution in a representative simulation

Despite the potential dangers of clinical tasks being forgotten, few researchers have investigated prospective memory (PM) – the ability to remember to execute future tasks – in health-care contexts. Visual cues help people remember to execute intentions at the appropriate moment. Using an intensive care unit simulator, we investigated whether nurses' memory for future tasks improves when visual cues are present, and how nurses manage PM demands. Twenty-four nurses participated in a 40-minute scenario simulating the start of a morning shift. The scenario included eight PM tasks. The presence or absence of a visually conspicuous cue for each task was manipulated. The presence of a visual cue improved recall compared to no cue (64% vs. 50%, p = 0.03 one-tailed, η _p ² = 0.15). Nurses used deliberate reminders to manage their PM demands. PM in critical care might be supported by increasing the visibility of cues related to tasks.

Practitioner summary: Nurses must remember to execute multiple future tasks to ensure patient safety. We investigated the effect of visual cues on nurses' ability to remember future tasks. Experimental manipulation of cues in a representative intensive care unit simulation indicated that visual cues increase the likelihood that future tasks are executed.     

Effect of personality type on structured tool comprehension performance

Accurate understanding of software requirements by end users and software developers is important to ensure a high quality software product. While comprehension performance on systems analysis tools has been studied in the past, there is little research that examined the influence of personality type of an individual on his/her performance. This paper has two objectives. First, the research uncovers the relationships between personality types (introvert/extrovert, sensing/intuitive, feeling/thinking, and perceptive/judging) and comprehension performance (accuracy and speed) of users/developers using the structured tools: Decision Tables (DT), Nassi–Schneiderman Charts (NS) and Structured English (SE). Second, it examines the trade-offs between comprehension accuracy and speed for each personality type. Using laboratory experiments, we measured individual performance with the three structured tools. We found that introverts and feeling personalities comprehended more accurately with DT; thinking and intuitive personalities comprehended more accurately with NS and SE. The comprehension accuracy increased with time more for SE than for DT and NS. The results show the most suitable combinations of structured tools and personality types for high comprehension. The results also provide guidelines to managers with tight project schedules, such as structured tools that are easier/faster to understand and the matching personalities who can comprehend faster.     

Object-Oriented Program Comprehension: Effect of Expertise, Task and Phase

The goal of our study is to evaluate the effect on program comprehension of three factors that have not previously been studied in a single experiment. These factors are programmer expertise (expert versus novice), programming task (documentation versus reuse), and the development of understanding over time (phase 1 versus phase 2). This study is carried out in the context of the mental model approach to comprehension based on van Dijk and Kintsch's model [(1983) Strategies of Discourse Comprehension. New York: Academic]. One key aspect of this model is the distinction between two kinds of representation the reader might construct from a text: (1) the textbase, which refers to what is said in the text and how it is said, and (2) the situation model, which represents the situation referred to by the text. We have evaluated the effect of the three factors mentioned above on the development of both the textbase (or program model) and the situation model in object-oriented program comprehension. We found a four-way interaction of expertise, phase, task and type of model. For the documentation group we found that experts and novices differ in the elaboration of their situation model but not their program model. There was no interaction of expertise with phase and type of model in the documentation group. For the reuse group, there was a three-way interaction between phase, expertise and type of model. For the novice reuse group, the effect of the phase was to increase the construction of the situation model but not the program model. With respect to the task, our results show that novices do not spontaneously construct a strong situation model but are able to do so if the task demands it.     

Effects of increasing visual load on aurally and visually guided target acquisition in a virtual environment

The aim of the present study is to investigate interactions between vision and audition during a target acquisition task performed in a virtual environment. We measured the time taken to locate a visual target (acquisition time) signalled by auditory and/or visual cues in conditions of variable visual load. Visual load was increased by introducing a secondary visual task. The auditory cue was constructed using virtual three-dimensional (3D) sound techniques. The visual cue was constructed in the form of a 3D updating arrow. The results suggested that both auditory and visual cues reduced acquisition time as compared to an uncued condition. Whereas the visual cue elicited faster acquisition time than the auditory cue, the combination of the two cues produced the fastest acquisition time. The introduction of secondary visual task differentially affected acquisition time depending on cue modality. In conditions of high visual load, acquiring a target signalled by the auditory cue led to slower and more error-prone performance than acquiring a target signalled by either the visual cue alone or by both the visual and auditory cues.     

Hipikat: a project memory for software development

Sociological and technical difficulties, such as a lack of informal encounters, can make it difficult for new members of noncollocated software development teams to learn from their more experienced colleagues. To address this situation, we have developed a tool, named Hipikat that provides developers with efficient and effective access to the group memory for a software development project that is implicitly formed by all of the artifacts produced during the development. This project memory is built automatically with little or no change to existing work practices. After describing the Hipikat tool, we present two studies investigating Hipikat's usefulness in software modification tasks. One study evaluated the usefulness of Hipikat's recommendations on a sample of 20 modification tasks performed on the Eclipse Java IDE during the development of release 2.1 of the Eclipse software. We describe the study, present quantitative measures of Hipikat's performance, and describe in detail three cases that illustrate a range of issues that we have identified in the results. In the other study, we evaluated whether software developers who are new to a project can benefit from the artifacts that Hipikat recommends from the project memory. We describe the study, present qualitative observations, and suggest implications of using project memory as a learning aid for project newcomers.     

Interruption management: the use of attention-directing tactile cues

Previous research has suggested that providing informative cues about interrupting stimuli aids management of multiple tasks. However, auditory and visual cues can be ineffective in certain situations. The objective of the present study was to explore whether attention-directing tactile cues aid or interfere with performance. A two-group posttest-only randomized experiment was conducted. Sixty-one participants completed a 30-min performance session consisting of aircraft-monitoring and gauge-reading computer tasks. Tactile signals were administered to a treatment group to indicate the arrival and location of interrupting tasks. Control participants had to remember to visually check for the interrupting tasks. Participants in the treatment group responded to more interrupting tasks and responded faster than did control participants. Groups did not differ on error rates for the interrupting tasks, performance of the primary task, or subjective workload perceptions. In the context of the tasks used in the present research, tactile cues allowed participants to effectively direct attention where needed without disrupting ongoing information processing. Tactile cues should be explored in a variety of other visual, interrupt-laden environments. Potential applications exist for aviation, user-interface design, vigilance tasks, and team environments.     

Using mobile devices as scientific measurement instruments: Reliable android task scheduling

In various usage scenarios, smartphones are used as measuring instruments to systematically and unobtrusively collect data measurements (e.g., sensor data, user activity, phone usage data). Unfortunately, in the race towards extending battery life and improving privacy, mobile phone manufacturers are gradually restricting developers in (frequently) scheduling background (sensing) tasks and impede the exact scheduling of their execution time (i.e., Android’s “best effort” approach). This evolution hampers successful deployment of smartphones in sensing applications in scientific contexts, with unreliable and incomplete sampling rates frequently reported in literature. In this article, we discuss the ins and outs of Android’s background tasks scheduling mechanism, and formulate guidelines for developers to successfully implement reliable task scheduling. Implementing these guidelines, we present a software library, agnostic from the underlying Android scheduling mechanisms and restrictions, that allows Android developers to reliably schedule tasks with a maximum sampling rate of one minute. Our evaluation demonstrates the use and versatility of our task scheduler, and experimentally confirms its reliability and acceptable energy usage.     

LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity

Cross-task generalization is a significant outcome that defines mastery in natural language understanding. Humans show a remarkable aptitude for this, and can solve many different types of tasks, given definitions in the form of textual instructions and a small set of examples. Recent work with pre-trained language models mimics this learning style: users can define and exemplify a task for the model to attempt as a series of natural language prompts or instructions. While prompting approaches have led to higher cross-task generalization compared to traditional supervised learning, analyzing ‘bias’ in the task instructions given to the model is a difficult problem, and has thus been relatively unexplored. For instance, are we truly modeling a task, or are we modeling a user's instructions? To help investigate this, we develop LINGO, a novel visual analytics interface that supports an effective, task-driven workflow to (1) help identify bias in natural language task instructions, (2) alter (or create) task instructions to reduce bias, and (3) evaluate pre-trained model performance on debiased task instructions. To robustly evaluate LINGO, we conduct a user study with both novice and expert instruction creators, over a dataset of 1,616 linguistic tasks and their natural language instructions, spanning 55 different languages. For both user groups, LINGO promotes the creation of more difficult tasks for pre-trained models, that contain higher linguistic diversity and lower instruction bias. We additionally discuss how the insights learned in developing and evaluating LINGO can aid in the design of future dashboards that aim to minimize the effort involved in prompt creation across multiple domains.