EAGER: Exploring Automatic Optimization of Multi-tiered HPC Storage Systems via Practical Reinforcement Learning

如今，科学发现越来越多地涉及生成和分析大量数据。这些数据密集型的科学应用对异构且极其复杂的高性能计算（HPC）集群的存储系统提出了重大挑战。需要高速数据访问的科学家在有效使用这些异构存储选项时常常会遇到挫败感。需要构建长期缺失的自动化 HPC I/O（输入/输出）中间件，以透明地帮助科学家实现最佳数据访问性能，而无需手动操作。为大规模、异构和共享的 HPC 存储系统设计自动化 HPC I/O 中间件是一项极具挑战性的任务。受此资助计划支持的研究人员利用机器学习技术来了解请求和当前系统状态，智能地、自适应地调度和协调 I/O 请求。这项研究的成果预计将与现有的存储组件配合使用，并最大限度地减少对科学应用和 HPC 系统的影响。该项目计划通过探索实用的基于强化学习 (RL) 的方法并在 HPC 环境中构建相关的软件基础设施来应对这一巨大挑战。该项目有两个主要关注点：1）基于强化学习的数据放置，以实现高存储利用率；2）基于强化学习的 I/O 协调，以实现共享存储。这两项任务都依赖于确定有效的强化学习方法并将这些方法有效地集成到 HPC 系统中。为了实现这一目标，将开发一种新颖的、以系统为中心的强化学习框架。此外，在每个研究重点中，各种强化学习算法、深度神经网络设计和奖励塑造都将被提出、实施、严格的基准测试，并与最先进的解决方案进行比较。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Nowadays, scientific discovery increasingly involves generating and analyzing large amounts of data. These data-intensive scientific applications pose significant challenges to the storage systems of high-performance computing (HPC) clusters, that are heterogeneous and extremely complex. Scientists who need high-speed data access often experience frustration in effectively using these heterogeneous storage options. There is need to build the long-missing automated HPC I/O (Input/Output) middleware to transparently help scientists achieve optimal data access performance without their manual efforts. Designing automated HPC I/O middleware for large-scale, heterogeneous, and shared HPC storage systems is an extremely challenging task. The researchers supported by this grant plan to leverage machine learning techniques to understand the requests and the current system status, intelligently and adaptively scheduling and coordinating I/O requests. The outcomes of this research are expected to work with existing storage components and minimize the impacts on both scientific applications and the HPC systems.This project plans to tackle this grand challenge by exploring practical reinforcement learning-based (RL) methods and building relevant software infrastructure in an HPC environment. There are two main focuses in the project: 1) RL-based data placement for high storage utilization, and 2) RL-based I/O coordination for shared storage. Both tasks depend on identifying effective reinforcement learning methods and integrating these methods effectively into HPC systems. To achieve this goal, a novel, system-centric reinforcement learning framework will be developed. Moreover, in each research focus, various RL algorithms, deep neural network designs, and reward shaping will be proposed, implemented, rigorously benchmarked, and compared with state-of-the-art solutions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.