Collaborative Research: Frameworks: Diamond: Democratizing Large Neural Network Model Training for Science

Diamond是一项服务，旨在通过抽象HPC资源的使用来民主化对尖端DL方法的访问。Diamond将新颖的计算机科学研究与翻译计算机科学相结合，以减少阻碍在科学中采用DL方法的重大障碍。借助Diamond，领域科学家可以专注于神经网络架构设计，以解决其特定领域的挑战，而无需担心网络基础设施管理。钻石还有助于关键的教育成果。博士生直接致力于项目目标，项目中开发的工具将用于本科和研究生课程。这些工具还将用于TACC，UChicago和NCSA的暑期学校和项目。Diamond在软件生态系统管理、并行计算、深度学习和数据管理方面的工作基础上，将不同的功能结合到一个有凝聚力的、用户友好的框架中。它提供了一个支持Web服务的编程接口，支持从开发到部署和分发的DL生命周期。它提供容器配置、分布式训练的自动缩放、超参数调整和模型共享。它还应用了关键的性能优化，包括规划长时间训练作业、性能感知模型放置、跨集群训练和数据管理。钻石奖的结果提供给领域科学家、计算机科学家和支持HPC中心DL应用的工程师。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Diamond is a service designed to democratize access to cutting-edge DL methods by abstracting the use of HPC resources. Diamond combines novel computer science research with translational computer science to reduce the significant barriers that impede adoption of DL methods in science. With Diamond, domain scientists can focus on the neural network architecture design to solve their domain-specific challenges without worrying about Cyberinfrastructure management. Diamond also contributes to key educational outcomes. PhD students work directly on project goals, and tools developed in the project will be used in undergraduate and graduate-level courses. The tools will also be used in summer schools and programs at TACC, UChicago, and NCSA. Targeted recruitment of students from underserved communities at the graduate, undergraduate, and high-school levels will address diversity and outreach goals.Diamond builds upon prior work in software ecosystem management, parallel computing, deep learning, and data management, combining disparate capabilities into a cohesive and user-friendly framework. It provides a web service-enabled programming interface supporting the DL lifecycle from development to deployment and dissemination. It offers container configuration, automatic scaling for distributed training, hyper-parameter tuning, and model sharing. It also applies crucial performance optimizations, including planning for long training jobs, performance-aware model placement, cross-cluster training, and data management. Diamond results are made available to domain scientists, computer scientists, and engineers supporting DL applications in HPC centers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.