Reproducible Workflow on a Public Cloud for Computational Fluid Dynamics
Reproducible Workflow on a Public Cloud for Computational Fluid Dynamics
复制标题
DOI：
10.1109/mcse.2019.2941702
复制
发表时间：
2019-04
影响因子：
2.1
通讯作者：
O. Mesnard;L. Barba
中科院分区：
文献类型：
--
作者：
O. Mesnard;L. Barba
为了使我们的研究透明并可由其他人复制，我们开发了一个工作流程，在公共云Microsoft Azure上运行和共享计算研究。它使用Docker容器来创建应用程序软件堆栈的映像。我们还采用了几种工具，便于在计算节点上创建和管理虚拟机，并将作业提交给这些节点。这些工具的配置文件是扩展的“可再现性包”的一部分，除了输入文件和说明外，还包括云计算的工作流定义。这有助于重新创建云环境，以在相同条件下重新计算。尽管云计算提供商已经改进了他们的产品，但许多使用高性能计算（HPC）的研究人员仍然对云计算持怀疑态度。因此，我们运行了紧密耦合应用程序的基准测试，以确认Microsoft Azure的最新HPC节点确实是传统现场HPC集群的可行替代方案。我们还表明，云产品现在足以完成计算流体动力学研究与内部研究软件，使用并行计算与GPU。最后，我们与社区分享了近两年来使用Azure云来提高计算模拟的透明度和可重复性的经验。
In a new effort to make our research transparent and reproducible by others, we developed a workflow to run and share computational studies on the public cloud Microsoft Azure. It uses Docker containers to create an image of the application software stack. We also adopt several tools that facilitate creating and managing virtual machines on compute nodes and submitting jobs to these nodes. The configuration files for these tools are part of an expanded “reproducibility package” that includes workflow definitions for cloud computing, in addition to input files and instructions. This facilitates recreating the cloud environment to rerun the computations under the same conditions. Although cloud providers have improved their offerings, many researchers using high-performance computing (HPC) are still skeptical about cloud computing. Thus, we ran benchmarks for tightly coupled applications to confirm that the latest HPC nodes of Microsoft Azure are indeed a viable alternative to traditional on-site HPC clusters. We also show that cloud offerings are now adequate to complete computational fluid dynamics studies with in-house research software that uses parallel computing with GPUs. Finally, we share with the community what we have learned from nearly two years of using Azure cloud to enhance the transparency and reproducibility in our computational simulations.