CSR: Small: IRON: Reducing Workload Interference on Massively Parallel Platforms

CSR：小：IRON：减少大规模并行平台上的工作负载干扰

• 批准号: 1717763
• 负责人: Zhiling Lan
• 金额: $ 49.72万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717763&HistoricalAwards=false
• 关键词: CSR; Small; IRON; Reducing; Workload

Interconnection networks based on dragonfly and fat tree designs are becoming increasingly dominant in high-performance computing facilities and data centers. A key challenge of managing such networks is workload interference. In a multi-user computing environment, interference among applications for shared network resources can cause a vicious cycle of events (workload interference, low productivity, selfish user behavior, and poor scheduling) aggravating each other. This project seeks to tackle the vicious circle of workload interference.This project aims to develop a software framework to systematically analyze and mitigate workload interference on dragonfly and fat tree systems. Completion of the project will create novel interference-aware scheduling policies and scalable software tools for interference analysis and reduction on dragonfly and fat tree networks. The resulting data and tools collected from simulations and experiments will be made available to the broad community under an open source license. An integrated education and outreach plan will enhance the Computer Science curriculum, broaden the participation by underrepresented groups, and outreach to the surrounding communities that are predominantly African-American and Latino.The proposed research focuses on three inter-related research thrusts: (1) high-fidelity network simulation to gain insights into network interference among applications, (2) interference aware scheduling to avoid or mitigate network contention among applications, and (3) real-world experiments to quantitatively measure workload interference of representative applications on production systems and assess interference aware scheduling. The resulting IRON will provide an integrated interference analysis and reduction framework for advancing the research of workload interference on massively parallel platforms. The project website (http://www.cs.iit.edu/~lan/iron.html) provides all the information about the project (publications, software tools, data, etc.). This project repository will be maintained for at least five years after the project completion.

在高性能计算设施和数据中心中，基于蜻蜓和脂肪树设计的互连网络越来越占主导地位。管理此类网络的关键挑战是工作负载干扰。在多用户计算环境中，共享网络资源应用程序之间的干扰可能会导致事件的恶性循环（工作负载干扰，生产力低，自私的用户行为和计划差）会互相加剧。该项目旨在应对工作负载干扰的恶性循环。该项目旨在开发一个软件框架，以系统地分析和减轻对蜻蜓和胖树系统的工作负载干扰。该项目的完成将创建新颖的干扰计划调度策略和可扩展的软件工具，以干扰分析和减少蜻蜓和胖树网络。从模拟和实验收集的最终数据和工具将根据开源许可提供给广泛的社区。一项综合的教育和宣传计划将增强计算机科学课程，扩大代表性不足的群体的参与，并向周围的社区推广，这些社区主要是非洲裔美国人和拉丁裔的周围社区。拟议的研究集中于三个相关的研究推力。应用程序，以及（3）实验，用于定量测量代表性应用在生产系统上的工作负载干扰并评估干涉意识计划。最终的铁将提供一个集成的干扰分析和减少框架，以推进工作负载干扰对大规模并行平台的研究。 项目网站（http://www.cs.iit.edu/~lan/iron.html）提供有关项目（出版物，软件工具，数据等）的所有信息。该项目存储库将在项目完成后至少维护五年。

项目成果

DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling

• DOI: 10.1016/j.simpa.2021.100077
• 发表时间: 2021-05
• 期刊: Softw. Impacts
• 作者: Yuping Fan;Z. Lan

Scheduling Beyond CPUs for HPC

• DOI: 10.1145/3307681.3325401
• 发表时间: 2019-06
• 期刊: Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Yuping Fan;Z. Lan;Paul M. Rich;W. Allcock;M. Papka;Brian Austin;D. Paul

The Effect of System Utilization on Application Performance Variability

系统利用率对应用程序性能变化的影响

• 发表时间: 2019
• 期刊: ACM Digital Library
• 作者: Li, Boyang;Chunduri, Sudheer;Harms, Kevin;Fan, Yuping;Lan, Zhiling
• 通讯作者: Lan, Zhiling

Union: An Automatic Workload Manager for Accelerating Network Simulation

Union：用于加速网络仿真的自动工作负载管理器

• DOI: 10.1109/ipdps47924.2020.00089
• 发表时间: 2020
• 期刊: 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS
• 作者: Wang, Xin;Mubarak, Misbah;Kang, Yao;Ross, Robert B.;Lan, Zhiling
• 通讯作者: Lan, Zhiling

Modeling and Analysis of Application Interference on Dragonfly+

Dragonfly应用干扰建模与分析

• 发表时间: 2019
• 期刊: ACM SIGSIM PADS
• 作者: Kang, Yao;Wang, Xin;McGlohon, Neil;Mubarak, Misbah;Chunduri, Sudheer;Lan, Zhiling
• 通讯作者: Lan, Zhiling