Composing effects into tasks and workflows
Composing effects into tasks and workflows
复制标题
DOI：
10.1145/3406088.3409023
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
Yves Parès;Jean-Philippe Bernardy;R. Eisenberg
中科院分区：
文献类型：
--
作者：
Yves Parès;Jean-Philippe Bernardy;R. Eisenberg
数据科学应用程序往往是通过组成任务来构建的：数据的离散操作。这些任务在有向的无环图中安排，并且在支持这种结构的数据科学界中存在许多框架，这称为工作流程。在现实的应用程序中，我们希望能够在没有数据的情况下分析工作流程，并使用数据执行工作流程。本文将效果处理程序与类似箭头结构结合在一起，以抽象数据科学任务。这种技术组合可以实现工作流的模块化设计。此外，可以在运行之前分析这些工作流程（例如，提供早期失败）并方便地运行。我们的工作是由现实情况直接激励的，我们认为我们的方法适用于新的数据科学和机器学习应用程序和框架。
Data science applications tend to be built by composing tasks: discrete manipulations of data. These tasks are arranged in directed acyclic graphs, and many frameworks exist within the data science community supporting such a structure, which is called a workflow. In realistic applications, we want to be able to both analyze a workflow in the absence of data, and to execute the workflow with data. This paper combines effect handlers with arrow-like structures to abstract out data science tasks. This combination of techniques enables a modular design of workflows. Additionally, these workflows can both be analyzed prior to running (e.g., to provide early failure) and run conveniently. Our work is directly motivated by real-world scenarios, and we believe that our approach is applicable to new data science and machine learning applications and frameworks.