Function passing: a model for typed, distributed functional programming
Function passing: a model for typed, distributed functional programming
复制标题
DOI：
10.1145/2986012.2986014
复制
发表时间：
2016-10
期刊：
影响因子：
--
通讯作者：
Heather Miller;Philipp Haller;N. Müller;Jocelyn Boullier
中科院分区：
文献类型：
--
作者：
Heather Miller;Philipp Haller;N. Müller;Jocelyn Boullier
最成功的“大数据”处理系统都采用了功能性API。我们提出了一个新的编程模型，我们称之为函数传递，旨在提供一个更有原则的基础上，建立以数据为中心的分布式系统。一个关键的想法是建立一个持久的函数数据结构，表示分布式不可变数据上的转换，方法是通过网络传递类型良好的可序列化函数，并将它们应用于分布式数据。因此，函数传递模型可以被认为是分布式的持久函数数据结构，其中对数据的转换存储在其节点中，而不是分布式数据本身。该模型通过设计简化了故障恢复-在发生故障时，通过在从稳定存储加载的不可变数据上重放功能应用程序来恢复数据。延迟求值也是我们模型的核心;通过仅在启动网络通信时将延迟求值纳入我们的设计，函数传递模型仍然易于推理，同时在时间和内存方面保持高效。我们正式使用小步操作语义我们的编程模型，我们提供了一个开源的实现我们的模型和Scala编程语言，沿着的几个例子框架和最终用户程序的案例研究，写在这个模型之上。
The most successful systems for "big data" processing have all adopted functional APIs. We present a new programming model we call function passing designed to provide a more principled substrate on which to build data-centric distributed systems. A key idea is to build up a persistent functional data structure representing transformations on distributed immutable data by passing well-typed serializable functions over the wire and applying them to this distributed data. Thus, the function passing model can be thought of as a persistent functional data structure that is distributed, where transformations to data are stored in its nodes rather than the distributed data itself. The model simplifies failure recovery by design-in the event of a failure, data is recovered by replaying function applications atop immutable data loaded from stable storage. Deferred evaluation is also central to our model; by incorporating deferred evaluation into our design only at the point of initiating network communication, the function passing model remains easy to reason about while remaining efficient in time and memory. We formalize our programming model using small-step operational semantics, and we provide an open-source implementation of our model in and for the Scala programming language, along with a case study of several example frameworks and end-user programs written atop of this model.