CSR: Small: IRON: Reducing Workload Interference on Massively Parallel Platforms

CSR：小：IRON：减少大规模并行平台上的工作负载干扰

• 批准号: 1717763
• 负责人: Zhiling Lan
• 金额: $ 49.72万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717763&HistoricalAwards=false
• 关键词: CSR; Small; IRON; Reducing; Workload

Interconnection networks based on dragonfly and fat tree designs are becoming increasingly dominant in high-performance computing facilities and data centers. A key challenge of managing such networks is workload interference. In a multi-user computing environment, interference among applications for shared network resources can cause a vicious cycle of events (workload interference, low productivity, selfish user behavior, and poor scheduling) aggravating each other. This project seeks to tackle the vicious circle of workload interference.This project aims to develop a software framework to systematically analyze and mitigate workload interference on dragonfly and fat tree systems. Completion of the project will create novel interference-aware scheduling policies and scalable software tools for interference analysis and reduction on dragonfly and fat tree networks. The resulting data and tools collected from simulations and experiments will be made available to the broad community under an open source license. An integrated education and outreach plan will enhance the Computer Science curriculum, broaden the participation by underrepresented groups, and outreach to the surrounding communities that are predominantly African-American and Latino.The proposed research focuses on three inter-related research thrusts: (1) high-fidelity network simulation to gain insights into network interference among applications, (2) interference aware scheduling to avoid or mitigate network contention among applications, and (3) real-world experiments to quantitatively measure workload interference of representative applications on production systems and assess interference aware scheduling. The resulting IRON will provide an integrated interference analysis and reduction framework for advancing the research of workload interference on massively parallel platforms. The project website (http://www.cs.iit.edu/~lan/iron.html) provides all the information about the project (publications, software tools, data, etc.). This project repository will be maintained for at least five years after the project completion.

在高性能计算设施和数据中心中，基于树和胖树设计的互连网络正变得越来越占主导地位。管理此类网络的一个关键挑战是工作负载干扰。在多用户计算环境中，共享网络资源的应用程序之间的干扰可能导致事件（工作负载干扰，低生产力，自私的用户行为和不良调度）的恶性循环，相互加剧。本项目旨在解决工作负载干扰的恶性循环。本项目旨在开发一个软件框架，以系统地分析和减轻工作负载干扰对树和胖树系统的影响。该项目的完成将创建新的干扰感知调度策略和可扩展的软件工具，用于干扰分析和减少冗余和胖树网络。从模拟和实验中收集的数据和工具将在开源许可证下提供给广大社区。一个综合的教育和推广计划将加强计算机科学课程，扩大代表性不足的群体的参与，并推广到主要是非洲裔美国人和拉丁美洲人的周边社区。拟议的研究集中在三个相互关联的研究重点：（1）高保真网络仿真，以深入了解应用程序之间的网络干扰，（2）干扰感知调度，以避免或减轻应用之间的网络争用，以及（3）真实世界实验，以定量地测量生产系统上的代表性应用的工作负载干扰并评估干扰感知调度。由此产生的IRON将提供一个集成的干扰分析和减少框架，推进大规模并行平台上的工作负载干扰的研究。 项目网站（http：//www.cs.iit.edu/cnlan/iron.html）提供了关于该项目的所有信息（出版物、软件工具、数据等）。该项目资料库将在项目完成后至少维持五年。

项目成果

DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling

• DOI: 10.1016/j.simpa.2021.100077
• 发表时间: 2021-05
• 期刊: Softw. Impacts
• 作者: Yuping Fan;Z. Lan

Scheduling Beyond CPUs for HPC

• DOI: 10.1145/3307681.3325401
• 发表时间: 2019-06
• 期刊: Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Yuping Fan;Z. Lan;Paul M. Rich;W. Allcock;M. Papka;Brian Austin;D. Paul

The Effect of System Utilization on Application Performance Variability

系统利用率对应用程序性能变化的影响

• 发表时间: 2019
• 期刊: ACM Digital Library
• 作者: Li, Boyang;Chunduri, Sudheer;Harms, Kevin;Fan, Yuping;Lan, Zhiling
• 通讯作者: Lan, Zhiling

Trade-Off Study of Localizing Communication and Balancing Network Traffic on a Dragonfly System

Dragonfly 系统上本地化通信和平衡网络流量的权衡研究

• 发表时间: 2018
• 期刊: IPDPS .... [proceedings]
• 作者: Wang, Xin;Mubarak, Misbah;Yang, Xu;Ross, Rob;Lan, Zhiling
• 通讯作者: Lan, Zhiling

Union: An Automatic Workload Manager for Accelerating Network Simulation

Union：用于加速网络仿真的自动工作负载管理器

• DOI: 10.1109/ipdps47924.2020.00089
• 发表时间: 2020
• 期刊: 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS
• 作者: Wang, Xin;Mubarak, Misbah;Kang, Yao;Ross, Robert B.;Lan, Zhiling
• 通讯作者: Lan, Zhiling