CyberTraining: Pilot: Cross-Layer Training of High-Performance Deep Learning Technologies and Applications for Research Workforce Development in Central Valley

高性能计算（HPC）已经彻底改变了许多科学领域，包括气候研究、野生动物健康、农业科学以及科学模拟和建模。随着hpc加速深度学习（HPC-DL）系统和应用的出现，迫切需要全面的跨层培训材料来教育研究人员了解这些先进技术。该试点项目的主要目标是通过向广泛的网络基础设施（CI）用户提供全面的跨层HPC-DL培训来满足这一需求。目标受众包括本科生和研究生、博士后、教师和研究人员，他们可以从利用HPC-DL CI技术和资源的增强知识和技能中受益。通过为他们提供必要的培训，该项目旨在提高他们的研究效率，最大限度地发挥HPC-DL在各自领域的潜力。此外，该项目还特别注重促进加州中央山谷地区代表性不足的社区的包容性和扩大机会。这将通过赋予个人在HPC-DL领域取得优异成绩所需的知识和技能来促进国家利益。该项目通过开发综合培训材料、促进同行咨询项目、举办研讨会和建立包容性学习文化，解决了高性能计算机与深度学习融合领域的关键培训需求。它以跨层的方式集成了科学应用、高性能计算技术和深度学习。培训计划涵盖了几个重要的CI主题，包括远程直接内存访问（RDMA），基于gpu的分布式计算，Slurm， MPI和NCCL，这对于实现高性能的HPC-DL工作负载至关重要。培训还将深入研究分布式深度学习培训框架，如PyTorch， TensorFlow和Horovod，使参与者能够有效地利用这些工具进行研究。此外，培训还结合了实际的深度学习应用案例研究，提供了现实世界的例子和见解。短期目标是使个人具备HPC-DL知识和跨层优化技能，最大限度地利用HPC-DL CI资源，提高研究效率。本项目还将考察以实践为中心的模式和以HPC-DL为中心的讲习班在促进代表性不足的社区采用HPC-DL方面的有效性。该项目的长期目标是培养一支对HPC-DL ci有深刻理解的强大研究队伍。通过建立学习文化和瞄准大量CI用户，该项目解决了劳动力短缺问题，并将其影响扩展到中央山谷以外。通过合作和传播开源培训材料，它将有助于推进计算和数据密集型科学模拟和知识发现。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

High-Performance Computing (HPC) has revolutionized various scientific fields, including climate research, wildlife health, agricultural sciences, and scientific simulations and modeling. With the emergence of HPC-accelerated deep learning (HPC-DL) systems and applications, there is a pressing need for comprehensive cross-layer training materials to educate the research workforce on these advanced technologies. The primary objective of this pilot project is to address this need by providing comprehensive cross-layer HPC-DL training to a wide range of cyberinfrastructure (CI) users. The target audience includes undergraduate and graduate students, postdocs, faculty, and research staff who can benefit from enhanced knowledge and skills in utilizing HPC-DL CI technologies and resources. By equipping them with the necessary training, the project aims to improve their research efficiency and maximize the potential of HPC-DL in their respective fields. In addition, the project has a specific focus on fostering inclusivity and expanding opportunities for underrepresented communities in the Central Valley area of California. This will contribute to the national interest by empowering individuals with the knowledge and skills necessary to excel in the HPC-DL field. This project addresses the critical training needs of the converged HPC-DL field by developing comprehensive training materials, fostering peer consultant programs, conducting workshops, and building an inclusive learning culture. It includes an integration of scientific applications, HPC technologies, and DL in a cross-layer approach. The training program covers several important CI topics, including Remote Direct Memory Access (RDMA), GPU-based distributed computing, Slurm, MPI, and NCCL, which are critical to achieving high performance for HPC-DL workloads. The training will also dive into distributed DL training frameworks such as PyTorch, TensorFlow, and Horovod, enabling participants to effectively leverage these tools for their research. Moreover, the training incorporates practical DL application case studies, offering real-world examples and insights. The short-term goal is to empower individuals with HPC-DL knowledge and cross-layer optimization skills to maximize the utilization of HPC-DL CI resources and improve research efficiency. This project will also examine the effectiveness of practice-central models and HPC-DL-centered workshops in promoting HPC-DL adoption in underrepresented communities. The project's long-term aim is to cultivate a robust research workforce with a deep understanding of HPC-DL CIs. By establishing a learning culture and targeting a significant number of CI users, this project addresses workforce shortages and extends its impact beyond the Central Valley. Through collaborations and the dissemination of open-source training materials, it will contribute to advancing compute- and data-intensive scientific simulations and knowledge discovery.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.