signac: A Python framework for data and workflow management
signac: A Python framework for data and workflow management
复制标题
DOI：
10.25080/majora-4af1f417-016
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Vyas Ramasubramani;C. Adorf;P. Dodd;Bradley D Dice;S. Glotzer
中科院分区：
文献类型：
--
作者：
Vyas Ramasubramani;C. Adorf;P. Dodd;Bradley D Dice;S. Glotzer
计算研究需要多功能的数据和工作流程管理工具，可以轻松适应科学研究的高度动态要求。许多现有工具需要严格遵守特定的使用模式，因此研究人员经常使用他们发现更容易采用的不太健壮的临时解决方案。由此产生的数据碎片和方法不兼容严重阻碍了研究。我们的演讲展示了signac，这是一个开源Python框架，它为这个问题提供了高度模块化和可扩展的解决方案。该框架以点画派画家 Paul Signac 的名字命名，其强大的工作流程管理工具使用户能够构建和自动化从笔记本电脑无缝过渡到 HPC 集群的工作流程。至关重要的是，底层数据模型完全独立于工作流程。灵活、无服务器且无模式的 Signac 数据库可以引入到其他工作流程中，基本上没有任何开销，也无需求助于 Signac 工作流程模型。此外，数据模型的简单性使得无需使用signac就可以轻松解析底层数据。这种模块化和简单性消除了跨项目一致数据管理的重大障碍，有助于以最小的成本改进来源管理和数据共享。
Computational research requires versatile data and workflow management tools that can easily adapt to the highly dynamic requirements of scientific investigations. Many existing tools require strict adherence to a particular usage pattern, so researchers often use less robust ad hoc solutions that they find easier to adopt. The resulting data fragmentation and methodological incompatibilities significantly impede research. Our talk showcases signac, an open-source Python framework that offers highly modular and scalable solutions for this problem. Named for the Pointillist painter Paul Signac, the framework’s powerful workflow management tools enable users to construct and automate workflows that transition seamlessly from laptops to HPC clusters. Crucially, the underlying data model is completely independent of the workflow. The flexible, serverless, and schema-free signac database can be introduced into other workflows with essentially no overhead and no recourse to the signac workflow model. Additionally, the data model’s simplicity makes it easy to parse the underlying data without using signac at all. This modularity and simplicity eliminates significant barriers for consistent data management across projects, facilitating improved provenance management and data sharing with minimal