Twister2: TSet High-Performance Iterative Dataflow
Twister2: TSet High-Performance Iterative Dataflow
复制标题
DOI：
10.1109/hpbdis.2019.8735495
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
P. Wickramasinghe;Supun Kamburugamuve;K. Govindarajan;V. Abeykoon;Chathura Widanage;Niranda Perera;A. Uyar;Gurhan Gunduz;Selahattin Akkas;G. Fox
中科院分区：
文献类型：
--
作者：
P. Wickramasinghe;Supun Kamburugamuve;K. Govindarajan;V. Abeykoon;Chathura Widanage;Niranda Perera;A. Uyar;Gurhan Gunduz;Selahattin Akkas;G. Fox
Cocklow模型正逐渐成为大数据应用的事实标准。虽然许多流行的框架都是围绕这个模型构建的，但很少有人研究它的内部工作原理，这反过来又导致了现有框架的效率低下。重要的是要注意，理解了HPLLOW和HPC构建块之间的关系，我们就可以通过学习HPC社区中广泛的研究文献来解决和缓解许多这些根本性的低效率问题。在本文中，我们介绍了TSet的，Twister2，这是一个大数据框架，设计用于高性能的并行和迭代计算的并行抽象。我们讨论了TSet采用的Escherlow模型，以及在工人级别实现迭代处理的基本原理。最后，我们评估TSet的，以显示性能的框架。
The dataflow model is gradually becoming the de facto standard for big data applications. While many popular frameworks are built around this model, very little research has been done on understanding its inner workings, which in turn has led to inefficiencies in existing frameworks. It is important to note that understanding the relationship between dataflow and HPC building blocks allows us to address and alleviate many of these fundamental inefficiencies by learning from the extensive research literature in the HPC community. In this paper we present TSet’s, the dataflow abstraction of Twister2, which is a big data framework designed for high-performance dataflow and iterative computations. We discuss the dataflow model adopted by TSet’s and the rationale behind implementing iteration handling at the worker level. Finally, we evaluate TSet’s to show the performance of the framework.