Thoughts about Artifact Badging
Thoughts about Artifact Badging
复制标题
关于神器徽章的想法
DOI：
10.1145/3402413.3402422
复制
发表时间：
2020
影响因子：
2.8
通讯作者：
A. Moore
中科院分区：
文献类型：
--
作者：
Noa Zilberman;A. Moore
可重复性：重复实验时获得一致的结果的程度非常重要，作为验证实验结果，促进研究完整性并加速后续工作的一种手段。对工件审查和徽章的承诺旨在促进可重复性并对提交的工件的质量进行排名。但是，如本期所示，当前的徽章方案的重点是可重复使用的工件，可能无法确定建筑，实施或评估的局限性。我们建议，要改善对伪影可重复性的洞察力，人工制品评估的深度和性质必须不仅仅是考虑是否可以重复使用工件。人工制品评估应考虑该评估的方法，以及对该评估的投入的变化。为了实现这一目标，我们建议扩展对工件徽章范围的范围，并描述其他社区出现的方法和最佳实践。我们寻求促进对话并呼吁采取行动，以加强我们领域内的科学方法。
Reproducibility: the extent to which consistent results are obtained when an experiment is repeated, is important as a means to validate experimental results, promote integrity of research, and accelerate follow up work. Commitment to artifact reviewing and badging seeks to promote reproducibility and rank the quality of submitted artifacts. However, as illustrated in this issue, the current badging scheme, with its focus upon an artifact being reusable, may not identify limitations of architecture, implementation, or evaluation. We propose that to improve the insight into artifact reproducibility, the depth and nature of artifact evaluation must move beyond simply considering if an artifact is reusable. Artifact evaluation should consider the methods of that evaluation alongside the varying of inputs to that evaluation. To achieve this, we suggest an extension to the scope of artifact badging, and describe both approaches and best practice arising in other communities. We seek to promote conversation and make a call to action intended to strengthen the scientific method within our domain.
Dagstuhl 实验网络研究再现性初学者指南
影响因子：
2.8
作者：
Bajpai, Vaibhav;Brunstrom, Anna;Feldmann, Anja;Kellerer, Wolfgang;Pras, Aiko;Schulzrinne, Henning;Smaragdakis, Georgios;Wählisch, Matthias;Wehrle, Klaus
通讯作者：
Wehrle, Klaus