Exploring the effect of noise on the performance benefit of nonblocking allreduce
Exploring the effect of noise on the performance benefit of nonblocking allreduce
复制标题
探索噪声对非阻塞 allreduce 性能优势的影响
DOI：
10.1145/2642769.2642786
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
T. Hoefler
中科院分区：
文献类型：
--
作者：
Patrick M. Widener;Kurt B. Ferreira;Scott Levy;T. Hoefler
宽松的同步允许许多进程在某些进程延迟时独立进行，从而提供了维护应用程序可伸缩性的潜力。然而，这种方法在重要的并行工作负载中的好处还没有被详细研究。在本文中，我们使用一个验证的模拟方法来探讨非阻塞allreduce的噪声缓解效果的工作负载，allreduce是一个主要的贡献者总执行时间。虽然非阻塞allreduce不太可能为下一代HPC系统中预期的低OS噪声环境中的应用程序提供显着的好处，但我们表明，它可以潜在地提高应用程序的运行时间相对于其他噪声类型。
Relaxed synchronization offers the potential of maintaining application scalability by allowing many processes to make independent progress when some processes suffer delays. Yet, the benefits of this approach in important parallel workloads have not been investigated in detail. In this paper, we use a validated simulation approach to explore the noise mitigation effects of nonblocking allreduce in workloads where allreduce is a major contributor to total execution time. Although a nonblocking allreduce is unlikely to provide significant benefit to applications in the low-OS-noise environments expected in next-generation HPC systems, we show that it can potentially improve application runtime with respect to other noise types.