The effectiveness of pair programming: A meta-analysis

Several experiments on the effects of pair versus solo programming have been reported in the literature. We present a meta-analysis of these studies. The analysis shows a small significant positive overall effect of pair programming on quality, a medium significant positive overall effect on duration, and a medium significant negative overall effect on effort. However, between-study variance is significant, and there are signs of publication bias among published studies on pair programming. A more detailed examination of the evidence suggests that pair programming is faster than solo programming when programming task complexity is low and yields code solutions of higher quality when task complexity is high. The higher quality for complex tasks comes at a price of considerably greater effort, while the reduced completion time for the simpler tasks comes at a price of noticeably lower quality. We conclude that greater attention should be given to moderating factors on the effects of pair programming.     

Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise

A total of 295 junior, intermediate, and senior professional Java consultants (99 individuals and 98 pairs) from 29 international consultancy companies in Norway, Sweden, and the UK were hired for one day to participate in a controlled experiment on pair programming. The subjects used professional Java tools to perform several change tasks on two alternative Java systems with different degrees of complexity. The results of this experiment do not support the hypotheses that pair programming in general reduces the time required to solve the tasks correctly or increases the proportion of correct solutions. On the other hand, there is a significant 84 percent increase in effort to perform the tasks correctly. However, on the more complex system, the pair programmers had a 48 percent increase in the proportion of correct solutions but no significant differences in the time taken to solve the tasks correctly. For the simpler system, there was a 20 percent decrease in time taken but no significant differences in correctness. However, the moderating effect of system complexity depends on the programmer expertise of the subjects. The observed benefits of pair programming in terms of correctness on the complex system apply mainly to juniors, whereas the reductions in duration to perform the tasks correctly on the simple system apply mainly to intermediates and seniors. It is possible that the benefits of pair programming will exceed the results obtained in this experiment for larger, more complex tasks and if the pair programmers have a chance to work together over a longer period of time     

The factors that affect the performance of open source software development – the perspective of social capital and expertise integration

Drawing on social capital theory, we develop a theoretical model aiming to explore how open source software (OSS) project effectiveness (in terms of team size, team effort and team's level of completion) is affected by expertise integration. This in turn is influenced by three types of social capital – relational capital, cognitive capital and structural capital. In addition, this study also examines two moderating effects – the impact of technical complexity on the relationship between cognitive capital and expertise integration, and of task interdependence on the relationship between expertise integration and task completion. Through a field survey of 160 OSS members from five Taiwanese communities, there is support for some of the proposed hypotheses. Both reciprocity and centrality affect expertise integration as expected, but the influence of commitment and cognitive capital (including expertise and tenure) on expertise integration is not significant. Finally, expertise integration affects both team size and team effort, which in turn jointly influence task completion. This research contributes to advancing theoretical understanding of the effectiveness of free OSS development as well as providing OSS practitioners with insight into how to leverage social capital for improving the performance of OSS development.     

Pair programming teams and high-quality knowledge sharing: A comparative study of coopetitive reward structures

There has been a growing research interest in understanding knowledge sharing in agile development. Yet, empirical research that sheds light on its underlying practices, such as pair programming, is evolving. This study uses insights from coopetition and software literature to focus inquiry on the relation between coopetitive rewards and high-quality knowledge sharing in pair programming teams. Theoretical hypotheses are developed and validated, suggesting that: ‘coopetitive rewards influence high-quality knowledge sharing both directly and over time through their impact on the level of knowledge sharing satisfaction’, and, ‘the impact of coopetitive rewards on high-quality knowledge sharing is dependent upon task complexity and the history of working under similar reward structure’. This study generates new understanding related to the use of rewards in pair programming teams, and offers a rigorous and replicable seven-step experimental process for simulating coopetitive structures and investigating their role in pair programming and in similar collaborative contexts.     

Visual creation of inhabited 3D environments

The creation of virtual reality applications and 3D environments is a complex task that requires good programming skills and expertise in computer graphics and many other disciplines. The complexity increases when we want to include complex entities such as virtual characters and animate them. In this paper we present a system that assists in the tasks of setting up a 3D scene and configuring several parameters affecting the behavior of virtual entities like objects and autonomous virtual humans. Our application is based on a visual programming paradigm, supported by a semantic representation, an ontology for virtual environments. The ontology allows us to store and organize the components of a 3D scene, together with the knowledge associated with them. It is also used to expose functionalities in the given 3D engine. Based on a formal representation of its components, the proposed architecture provides a scalable VR system. Using this system, non-experts can set up interactive scenarios with minimum effort; no programming skills or advanced knowledge is required.     

Modeling the cognitive content of displays

An approach for measuring the cognitive complexity of visual displays is discussed and applied to a dynamic display of avionic information. A semantic network formalism is used to model two interrelated knowledge systems, world knowledge and display knowledge. The information the operator receives during training about the general display format characteristics and the task requirements, along with other previously stored information, constitutes world knowledge. The semantic content of a particular configuration of information encountered during task performance constitutes display knowledge. Four orthogonal predictor measures of cognitive complexity were derived from the networks. In an experiment three of the orthogonal predictors were significantly correlated with task performance. After averaging across operators, the three significant predictors accounted for 99% of the variation of display effectiveness. Results indicate that a model of cognitive complexity based on a semantic network formalism may provide a useful technique for quantitatively evaluating the quality of competing display format concepts.     

Predicting vigilance: A fresh look at an old problem

Individual differences in vigilance are ubiquitous and relevant to a variety of work environments in industrial, transportation, medical and security settings. Despite much previous work, mostly on personality traits, it remains difficult to identify vigilant operators. This paper reviews recent research that may point towards practically useful predictor variables for vigilance. Theoretical approaches to identifying predictors that accommodate the heterogeneous nature of vigilance tasks are compared. The article surveys recent empirical studies using personality measures, ability tests and scales for stress and coping as predictors of vigilance. Promising new constructs include trait scales linked to fatigue, abnormal personality and the stress state of task engagement. Implications of the data reviewed for occupational selection are discussed. Selection should be based on a multivariate assessment strategy, cognitive task analysis of the operational vigilance task and use of work sample measures to capture typical stress responses to the task. This review paper surveys recent research that may point towards practically useful predictor variables for vigilance. The article surveys recent empirical studies using personality measures, ability tests and scales for stress and coping as predictors of vigilance. Selection should be based on a multivariate assessment strategy.     

The links between human error diversity and software diversity: Implications for fault diversity seeking

Software diversity is known to improve fault tolerance in N-version software systems by independent development. As the leading cause of software faults, human error is considered an important factor in diversity seeking. However, there is little scientific research focusing on how to seek software fault diversity based on human error mechanisms. A literature review was conducted to extract factors that may differentiate people with respect to human error-proneness. In addition, we constructed a conceptual model of the links between human error diversity and software diversity. An experiment was designed to validate the hypotheses, in the form of a programming contest, accompanied by a survey of cognitive styles and personality traits. One hundred ninety-two programs were submitted for the identical problem, and 70 surveys were collected. Code inspection revealed 23 faults, of which 10 were coincident faults. The results show that personality traits seems not effective predictors for fault diversity as a whole model, whereas cognitive styles and program measurements moderately account for the variation of fault density. The results also show causal relations between performance levels and coincident faults: coincident faults are unlikely to occur at skill-based performance level; the coincident faults introduced in rule-based performances show a high probability of occurrence, and the coincident faults introduced in knowledge-based performances are shaped by the content and formats of the task itself. Based on these results, we have proposed a model to seek software diversity and prevent coincident faults.     

Controlled experimentation on adaptations of pair programming

The use of agile methods is growing in industrial practice due to the documented benefits of increased software quality, shared programmer expertise, and user satisfaction. These methods include pair programming (two programmers working side-by-side producing the code) and test-driven approaches (test cases written first to prepare for coding). In practice, software development organizations adapt agile methods to their environment. The purpose of this research is to understand better the impacts of adapting these methods. We perform a set of controlled experiments to investigate how adaptations, or variations, to the pair programming method impact programming performance and user satisfaction. We find that method variations do influence programming results. In particular, better performance and satisfaction outcomes are achieved when the pair programming is performed in face-to-face versus virtual settings, in combination with the test-driven approach, and with more experienced programmers. We also find that limiting the extent of collaboration can be effective, especially when programmers are more experienced. These experimental results provide a rigorous foundation for deciding how to adapt pair programming methods into specific project contexts.     

Identifying Traits and Values of Top-Performing Information Security Personnel

Enterprise information security is a talent-centric proposition. Information assurance is a product of the combined expertise, attention-to-detail, and creativity of an information security team. A competitive edge can be obtained by hiring the top information security professionals. Therefore, identifying the right people is a mission-critical task. To assist in the candidate selection process, this research analyzes the enduring traits and values of top security performers. It identifies the personality traits and values which distinguish high-performing information security workers. In a laboratory study, a series of simulations were administered to 61 subjects to assess their ability to solve various information security problems. The characteristics of top information security performers were contrasted against the rest of the cohort. In terms of personality, the top performers have higher levels of conscientiousness and openness. With respect to values, the top performers have stronger theoretical and economic values.     

Excellent software professionals: experience, work activities, and perception by peers

This paper reports findings of a field study examining expertise in 29 software development projects. Using a peer nomination method, 33 out of 200 subjects were characterized as excellent software professionals. Excellent software professionals are described as having high technical and computational knowledge, a high level of social skills, and as using a method-oriented working style. They have a broader, not longer professional experience than do their colleagues. Excellent and average software professionals do not differ with respect to time spent on typical software development activities such as design, coding, or testing, but excellent software professionals are more often engaged in review meetings and consultations than are other team members.     

Task dynamics in self-organising task groups: expertise, motivational, and performance differences of specialists and generalists

Multi-agent simulation is applied to explore how different types of task variety cause workgroups to change their task allocation accordingly. We studied two groups, generalists and specialists. We hypothesised that the performance of the specialists would decrease when task variety increases. The generalists, on the other hand, would perform better in a high task variety condition. The results show that these hypotheses were only partly supported because both learning and motivational effects changed the task allocation process in a much more complex way. We conclude that although no task variety leads to specialisation and high task variety leads to generalisation, in general, performance is better when task variety is low. Further, in case of no task variety, specialists outperform generalists. In case of moderate variety the opposite is true. With high task variety, since there is no space for any expertise and motivational development, the behaviour of specialists and generalists becomes more similar, and, consequently also their performance.     

Excellent software professionals: experience,work activities,and perception by peers

Abstract

This paper reports findings of a field study examining expertise in 29 software development projects. Using a peer nomination method, 33 out of 200 subjects were characterized as excellent software professionals. Excellent software professionals are described as having high technical and computational knowledge, a high level of social skills, and as using a method-oriented working style. They have a broader, not longer professional experience than do their colleagues. Excellent and average software professionals do not differ with respect to time spent on typical software development activities such as design, coding, or testing, but excellent software professionals are more often engaged in review meetings and consultations than are other team members.     

An industry experiment on the effects of test-driven development on external quality and productivity

Existing empirical studies on test-driven development (TDD) report different conclusions about its effects on quality and productivity. Very few of those studies are experiments conducted with software professionals in industry. We aim to analyse the effects of TDD on the external quality of the work done and the productivity of developers in an industrial setting. We conducted an experiment with 24 professionals from three different sites of a software organization. We chose a repeated-measures design, and asked subjects to implement TDD and incremental test last development (ITLD) in two simple tasks and a realistic application close to real-life complexity. To analyse our findings, we applied a repeated-measures general linear model procedure and a linear mixed effects procedure. We did not observe a statistical difference between the quality of the work done by subjects in both treatments. We observed that the subjects are more productive when they implement TDD on a simple task compared to ITLD, but the productivity drops significantly when applying TDD to a complex brownfield task. So, the task complexity significantly obscured the effect of TDD. Further evidence is necessary to conclude whether TDD is better or worse than ITLD in terms of external quality and productivity in an industrial setting. We found that experimental factors such as selection of tasks could dominate the findings in TDD studies.     

Some surprising differences between novice and expert errors in computerized office work

This paper investigates the impact of different levels of expertise on errors in human-computer interaction. In a field study 174 clerical workers from 12 different companies were observed during their normal office work and were questioned on their expertise with computers. The level of expertise was determined by (a) the length of time an employee had worked with a computer (computer expertise); (b) the number of programs she knew (program expertise); and (c) the daily time s/he spent working with the computer (daily work-time expertise). These different operationalizations of novices and experts led to different results. In contrast to widespread assumptions, experts did not make fewer errors than novices (except in knowledge errors). On the other hand, experts spent less time handling the errors than novices. A cluster analysis produced four groups in the workforce: occasional users, frequent users, beginners, and general users.     

Complexity measures for rule-based programs

Software complexity measures are quantitative estimates of the amount of effort required by a programmer to comprehend a piece of code. Many measures have been designed for standard procedural languages, but little work has been done to apply software complexity concepts to nontraditional programming paradigms. This paper presents a collection of software complexity measures that were specifically designed to quantify the conceptual complexity of rule-based programs. These measures are divided into two classes: bulk measures, which estimate complexity by examining aspects of program size, and rule measures, which gauge complexity based on the ways in which program rules interact with data and other rules. A pilot study was conducted to assess the effectiveness of these measures. Several measures were found to correlate well with the study participants' ratings of program difficulty and the time required by them to answer questions that required comprehension of program elements. The physical order of program rules was also shown to affect comprehension. The authors conclude that the development of software complexity measures for particular programming paradigms may lead to better tools for managing program development and predicting maintenance effort in nontraditional programming environments     

Some surprising differences between novice and expert errors in computerized office work

Abstract

This paper investigates the impact of different levels of expertise on errors in human-computer interaction. In a field study 174 clerical workers from 12 different companies were observed during their normal office work and were questioned on their expertise with computers. The level of expertise was determined by (a) the length of time an employee had worked with a computer (computer expertise); (b) the number of programs she knew (program expertise); and (c) the daily time s/he spent working with the computer (daily work-time expertise). These different operationalizations of novices and experts led to different results. In contrast to widespread assumptions, experts did not make fewer errors than novices (except in knowledge errors). On the other hand, experts spent less time handling the errors than novices. A cluster analysis produced four groups in the workforce: occasional users, frequent users, beginners, and general users.     

Forty years of research on personality in software engineering: A mapping study

In this article, we present a systematic mapping study of research on personality in software engineering. The goal is to plot the landscape of current published empirical and theoretical studies that deal with the role of personality in software engineering. We applied the systematic review method to search and select published articles, and to extract and synthesize data from the selected articles that reported studies about personality. Our search retrieved more than 19,000 articles, from which we selected 90 articles published between 1970 and 2010. Nearly 72% of the studies were published after 2002 and 83% of the studies reported empirical research findings. Data extracted from the 90 studies showed that education and pair programming were the most recurring research topics, and that MBTI was the most used test. Research related to pair programming, education, team effectiveness, software process allocation, software engineer personality characteristics, and individual performance concentrated over 88% of the studies, while team process, behavior and preferences, and leadership performance were the topics with the smallest number of studies. We conclude that the number of articles has grown in the last few years, but contradictory evidence was found that might have been caused by differences in context, research method, and versions of the tests used in the studies. While this raises a warning for practitioners that wish to use personality tests in practice, it shows several opportunities for the research community to improve and extend findings in this field.     

A comparative analysis of parallel programming language complexity and performance

Several parallel programming languages, libraries and environments have been developed to ease the task of writing programs for multiprocessors. Proponents of each approach often point out various language features that are designed to provide the programmer with a simple programming interface. However, virtually no data exist that quantitatively evaluate the relative ease of use of different parallel programming languages. The paper borrows techniques from the software engineering field to quantify the complexity of three predominant programming models: shared-memory, message-passing and high-performance Fortran. It is concluded that traditional software complexity metrics are effective indicators of the relative complexity of parallel programming languages. The impact of complexity on run-time performance is also discussed in the context of message-passing vs. HPF on an IBM SP2. © 1998 John Wiley & Sons, Ltd.     

A polynomial relational class of binary CSP

Finding a solution to a constraint satisfaction problem (CSP) is known to be an NP-hard task. Considerable effort has been spent on identifying tractable classes of CSP, in other words, classes of constraint satisfaction problems for which there are polynomial time recognition and resolution algorithms. In this article, we present a relational tractable class of binary CSP. Our key contribution is a new ternary operation that we name mjx. We first characterize mjx-closed relations which leads to an optimal algorithm to recognize such relations. To reduce space and time complexity, we define a new storage technique for these relations which reduces the complexity of establishing a form of strong directional path consistency, the consistency level that solves all instances of the proposed class (and, indeed, of all relational classes closed under a majority polymorphism).