Exploring MPI Collective I/O and File-per-process I/O for Checkpointing a Logical Inference Task
Exploring MPI Collective I/O and File-per-process I/O for Checkpointing a Logical Inference Task
复制标题
探索 MPI 集体 I/O 和每进程文件 I/O 以对逻辑推理任务设置检查点
DOI：
10.1109/ipdpsw52791.2021.00153
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Sidharth Kumar
中科院分区：
文献类型：
--
作者：
Ke Fan;Kristopher K. Micinski;Thomas Gilray;Sidharth Kumar
我们提供了一个可扩展的并行I/O系统，用于在演绎数据库上构建的逻辑提示应用程序。 Datalog是一种通常用于指定使用DataLog的应用程序的语言或家族。流程分析）在我们以前的论文中，我们介绍了使用MPI构建的数据并行数据。 TAR DATALOG实现的状态，例如soufflé，仅支持串行I/O，主要是因为实现本身不支持多个节点并行execution.computing图形的及时闭合图是使用最简单的逻辑提示应用程序之一Datalog;我们将其用作微型基准，以表明我们的并行I/O系统的效率。因此，系统涉及两个步骤，将应用程序数据显示（嵌套B-Tree）转换为一个字节流，然后是实际的并行I/O。我们探索了两个流行的I/O技术。 /o用于从MPI集体I/O中提取性能，我们使用自适应条纹，对于POSIX I/O，我们使用每个程序I/O。在Argonne国家实验室。
We present a scalable parallel I/O system for a logical-inferencing application built atop a deductive database. Deductive databases can make logical deductions (i.e. conclude additional facts), based on a set of program rules, derived from facts already in the database. Datalog is a language or family of languages commonly used to specify rules and queries for a deductive database. Applications built using Datalog can range from graph mining (such as computing transitive closure or k-cliques) to program analysis (control and data-flow analysis). In our previous papers, we presented the first implementation of a data-parallel Datalog built using MPI. In this paper, we present a parallel I/O system used to checkpoint and restart applications built on top of our Datalog system. State of the art Datalog implementations, such as Soufflé, only support serial I/O, mainly because the implementation itself does not support many-node parallel execution.Computing the transitive closure of a graph is one of the simplest logical-inferencing applications built using Datalog; we use it as a micro-benchmark to demonstrate the efficacy of our parallel I/O system. Internally, we use a nested B-tree data-structure to facilitate fast and efficient in-memory access to relational data. Our I/O system therefore involves two steps, converting the application data-layout (a nested B-tree) to a stream of bytes followed by the actual parallel I/O. We explore two popular I/O techniques POSIX I/O and MPI collective I/O. For extracting performance out of MPI Collective I/O we use adaptive striping, and for POSIX I/O we use file-per-process I/O. We demonstrate the scalability of our system at up to 4,096 processes on the Theta supercomputer at the Argonne National Laboratory.