The role of replications in Empirical Software Engineering |
| |
Authors: | Forrest J Shull Jeffrey C Carver Sira Vegas Natalia Juristo |
| |
Affiliation: | (1) Fraunhofer Center for Experimental Software Engineering, College Park, MD, USA;(2) Mississippi State University, Mississippi State, MS, USA;(3) Universidad Politécnica de Madrid, Madrid, Spain |
| |
Abstract: | Replications play a key role in Empirical Software Engineering by allowing the community to build knowledge about which results
or observations hold under which conditions. Therefore, not only can a replication that produces similar results as the original
experiment be viewed as successful, but a replication that produce results different from those of the original experiment
can also be viewed as successful. In this paper we identify two types of replications: exact replications, in which the procedures of an experiment are followed as closely as possible; and conceptual replications, in which the same research question is evaluated by using a different experimental procedure. The focus of this paper is
on exact replications. We further explore them to identify two sub-categories: dependent replications, where researchers attempt to keep all the conditions of the experiment the same or very similar and independent replications, where researchers deliberately vary one or more major aspects of the conditions of the experiment. We then discuss the role
played by each type of replication in terms of its goals, benefits, and limitations. Finally, we highlight the importance
of producing adequate documentation for an experiment (original or replication) to allow for replication. A properly documented
replication provides the details necessary to gain a sufficient understanding of the study being replicated without requiring
the replicator to slavishly follow the given procedures.
|
| |
Keywords: | Experimental replications Experimental infrastructure Lab package |
本文献已被 SpringerLink 等数据库收录! |
|