Proxy Responses by FPGA-Based Switch for MapReduce Stragglers
Proxy Responses by FPGA-Based Switch for MapReduce Stragglers
复制标题
DOI：
10.1587/transinf.2017edp7287
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
Koya Mitsuzuka;M. Koibuchi;H. Amano;Hiroki Matsutani
中科院分区：
文献类型：
--
作者：
Koya Mitsuzuka;M. Koibuchi;H. Amano;Hiroki Matsutani
在并行处理应用程序中，一些被称为“掉队者”的工作者节点，其执行任务的速度明显慢于其他任务，增加了作业的执行时间。在本文中，我们提出了一个基于网络交换机的掉队处理系统，以减轻计算节点的负担。我们还提出了如何卸载检测掉队者和计算他们的结果在网络交换机，没有额外的工作节点之间的通信。我们介绍了一些近似的技术，在交换机的代理计算和响应，因此我们的开关被称为“近似SW”。作为一个模拟实验的结果，所提出的近似基于任务的相似性达到最好的准确性方面的质量生成的地图输出。我们还分析了如何抑制不必要的代理计算的ApproxSW。我们在NetFPGA-SUME板上实现了ApproxSW，该板具有四个10 GbE以太网（10 GbE）接口和一个Virtex-7 FPGA。实验结果表明，ApproxSW功能不会降低原有10 GbE交换机的性能。关键词：FPGA，MapReduce，离散器
In parallel processing applications, a few worker nodes called “stragglers”, which execute their tasks significantly slower than other tasks, increase the execution time of the job. In this paper, we propose a network switch based straggler handling system to mitigate the burden of the compute nodes. We also propose how to offload detecting stragglers and computing their results in the network switch with no additional communications between worker nodes. We introduce some approximate techniques for the proxy computation and response at the switch; thus our switch is called “ApproxSW.” As a result of a simulation experiment, the proposed approximation based on task similarity achieves the best accuracy in terms of quality of generated Map outputs. We also analyze how to suppress unnecessary proxy computation by the ApproxSW. We implement ApproxSW on NetFPGA-SUME board that has four 10Gbit Ethernet (10GbE) interfaces and a Virtex-7 FPGA. Experimental results shows that the ApproxSW functions do not degrade the original 10GbE switch performance. key words: FPGA, MapReduce, straggler