Understanding replication of experiments in software engineering: A classification
Understanding replication of experiments in software engineering: A classification
复制标题
DOI：
10.1016/j.infsof.2014.04.004
复制
发表时间：
2014-08-01
影响因子：
3.9
通讯作者：
Vegas, Sira
中科院分区：
文献类型：
--
作者：
Gomez, Omar S.;Juristo, Natalia;Vegas, Sira
背景：复制在实验学科中扮演着重要的角色。关于如何进行SE实验的重复，仍有许多不确定因素。复制者是否应该重复使用基线实验材料？如果有的话，原始实验和复制实验之间应该有多少联系？可以改变实验配置的哪些元素，使实验被视为重复而不是新实验？目的：为了提高我们对SE实验复制的理解，本文提出了一个分类，旨在为实验者提供他们可以进行哪些类型的复制的指导。方法：接下来的研究方法是根据以下活动进行的：(1)对SE和其他学科的实验复制进行文献综述；(2)确定组成实验配置的典型元素；(3)确定不同的重复目的；(4)制定SE的实验重复分类。结果：我们提出了复制的分类，为SE的实验人员提供了关于他们可以在复制中进行哪些更改的指导，并基于这些指导，这样的复制可以服务于哪些验证目的。拟议的分类有助于在更广泛的框架内容纳反对意见，它能够解释关于基线实验的更相似的重复较少。结论：复制的目的是验证结果，但不同类型的复制具有特殊的验证目的，并且具有不同程度的变化。每种复制类型都有助于发现可能影响结果的特定实验条件。建议的分类可用于识别复制中的更改，并在此基础上了解验证级别。(C) 2014 Elsevier B.V.版权所有
Context: Replication plays an important role in experimental disciplines. There are still many uncertainties about how to proceed with replications of SE experiments. Should replicators reuse the baseline experiment materials? How much liaison should there be among the original and replicating experimenters, if any? What elements of the experimental configuration can be changed for the experiment to be considered a replication rather than a new experiment?Objective: To improve our understanding of SE experiment replication, in this work we propose a classification which is intend to provide experimenters with guidance about what types of replication they can perform.Method: The research approach followed is structured according to the following activities: (1) a literature review of experiment replication in SE and in other disciplines, (2) identification of typical elements that compose an experimental configuration, (3) identification of different replications purposes and (4) development of a classification of experiment replications for SE.Results: We propose a classification of replications which provides experimenters in SE with guidance about what changes can they make in a replication and, based on these, what verification purposes such a replication can serve. The proposed classification helped to accommodate opposing views within a broader framework, it is capable of accounting for less similar replications to more similar ones regarding the baseline experiment.Conclusion: The aim of replication is to verify results, but different types of replication serve special verification purposes and afford different degrees of change. Each replication type helps to discover particular experimental conditions that might influence the results. The proposed classification can be used to identify changes in a replication and, based on these, understand the level of verification. (C) 2014 Elsevier B.V. All rights reserved.