Sustainable data analysis with Snakemake.
Sustainable data analysis with Snakemake.
复制标题
DOI：
10.12688/f1000research.29032.1
复制
发表时间：
2021-01-01
期刊：
影响因子：
--
通讯作者：
Koster, Johannes
中科院分区：
文献类型：
--
作者：
Molder, Felix;Jablonski, Kim Philipp;Koster, Johannes
数据分析通常需要大量的异构步骤，从应用各种命令行工具到使用脚本语言（如R或Python）来生成图和表。人们普遍认为，数据分析最好以可再现的方式进行。可再现性使原始数据甚至新数据的结果得到技术验证和再生。然而，仅再现性绝不足以提供具有持久影响的分析（即，可持续发展）的领域，甚至只是一个研究小组。我们认为，确保适应性和透明度同样重要。前者描述了修改分析以回答扩展或略有不同的研究问题的能力。后者描述了理解分析的能力，以判断它是否不仅在技术上有效，而且在方法上有效。在这里，我们分析了数据分析所需的属性，以实现可重现性，适应性和透明性。我们展示了如何流行的工作流管理系统Snakemake可以用来保证这一点，以及它如何使一个符合人体工程学的，结合的，统一表示的所有步骤涉及的数据分析，从原始数据处理，质量控制和细粒度的，交互式的探索和绘制的最终结果。
Data analysis often entails a multitude of heterogeneous steps, from the application of various command line tools to the usage of scripting languages like R or Python for the generation of plots and tables. It is widely recognized that data analyses should ideally be conducted in a reproducible way.Reproducibility enables technical validation and regeneration of results on the original or even new data. However, reproducibility alone is by no means sufficient to deliver an analysis that is of lasting impact (i.e., sustainable) for the field, or even just one research group. We postulate that it is equally important to ensure adaptability and transparency. The former describes the ability to modify the analysis to answer extended or slightly different research questions. The latter describes the ability to understand the analysis in order to judge whether it is not only technically, but methodologically valid. Here, we analyze the properties needed for a data analysis to become reproducible, adaptable, and transparent. We show how the popular workflow management system Snakemake can be used to guarantee this, and how it enables an ergonomic, combined, unified representation of all steps involved in data analysis, ranging from raw data processing, to quality control and fine-grained, interactive exploration and plotting of final results.