SHF: CSR: Small: Toward Smart HPC through Active Learning and Intelligent Scheduling

SHF：CSR：小型：通过主动学习和智能调度迈向智能 HPC

• 批准号: 1422009
• 负责人: Zhiling Lan
• 金额: $ 49.88万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-15 至 2019-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1422009&HistoricalAwards=false
• 关键词: SHF; CSR; Small; Toward; Smart

As high performance computing (HPC) continues to grow in scale, energy and resilience become first-class concerns, in addition to the pursuit of performance. These concerns demand significant changes in many aspects of the system stack including resource management and job scheduling. In order to harness the great potential of extreme scale systems, this project aims to incorporate intelligence into resource management and job scheduling. More specifically, it will develop a framework named SPEaR (Scheduling for Performance, Energy, and Resilience efficiency) for dynamically optimizing the three-dimensional performance, energy, and resilience scheduling. The research focuses on two thrusts: one is active learning to automatically extract valuable performance, energy, and resilience patterns and tradeoffs out of application and system data, and the other is intelligent scheduling to improve and control performance, resilience, and energy efficiency in resource management and scheduling. An event-driven scheduling simulator is being developed for comprehensively evaluating scheduling policies and their aggregate effects. The simulator, along with system logs, will be made available to the broad community under an open source license. This project creates critical technologies to promote system productivity and makes important advances essential toward smart HPC. Additionally, the learning techniques developed in this project are useful to other big data problems of national interests. The education plan enhances the undergraduate and graduate curricula and broadens the participation from underrepresented groups.

随着高性能计算（HPC）的规模不断扩大，除了追求性能外，能源和弹性也成为首要问题。这些问题需要在系统堆栈的许多方面进行重大更改，包括资源管理和作业调度。为了利用极端规模系统的巨大潜力，该项目旨在将智能纳入资源管理和作业调度。更具体地说，它将开发一个名为SPEaR（性能，能量和弹性效率调度）的框架，用于动态优化三维性能，能量和弹性调度。该研究集中在两个方面：一是主动学习，以自动提取有价值的性能，能量和弹性模式以及应用程序和系统数据的权衡，另一个是智能调度，以提高和控制资源管理和调度中的性能，弹性和能源效率。一个事件驱动的调度模拟器正在开发的综合评估调度策略和它们的聚合效果。该模拟器，沿着系统日志，将在开源许可证下提供给广大社区。该项目创造了关键技术来提高系统生产力，并为智能HPC带来了重要的进步。此外，该项目中开发的学习技术对国家利益的其他大数据问题也很有用。教育计划加强了本科生和研究生课程，扩大了代表性不足群体的参与。

项目成果

Modeling and Analysis of Application Interference on Dragonfly+

Dragonfly应用干扰建模与分析

• 发表时间: 2019
• 期刊: ACM SIGSIM PADS
• 作者: Kang, Yao;Wang, Xin;McGlohon, Neil;Mubarak, Misbah;Chunduri, Sudheer;Lan, Zhiling
• 通讯作者: Lan, Zhiling

The Effect of System Utilization on Application Performance Variability

系统利用率对应用程序性能变化的影响

• 发表时间: 2019
• 期刊: ACM Digital Library
• 作者: Li, Boyang;Chunduri, Sudheer;Harms, Kevin;Fan, Yuping;Lan, Zhiling
• 通讯作者: Lan, Zhiling