Category I: Anvil - A National Composable Advanced Computational Resource for the Future of Science and Engineering

第一类：Anvil - 面向科学与工程未来的国家级可组合高级计算资源

• 批准号: 2005632
• 负责人: Xiaohui Carol Song
• 金额: $ 995.22万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Cooperative Agreement
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2005632&HistoricalAwards=false
• 关键词: Category; Anvil; National; Composable; Advanced

As computing permeates nearly all fields of science and engineering, there is an exponential growth of computing needs from both the traditional computing-intensive domains and the emerging new and more diverse fields of research. The rise of machine learning and artificial intelligence applications has accelerated and broadened the use of computational resources from research in creating new and more environmentally friendly materials to improving medicine in our fight against deadly diseases. There are three main challenges to meeting this rapidly evolving landscape of national computational needs: a shortage of capacity, increasingly diverse applications, and computational literacy and training. This project aims to meet these challenges and transform the way computing is delivered by developing and deploying a composable advanced computing resource, Anvil, to the national research community to significantly increase both the computing capacity and accessibility. Anvil integrates a large-capacity high-performance computing (HPC) cluster with a comprehensive ecosystem of software, access interfaces, programming environments, and composable services to form a seamless environment able to support a broad range of current and future science and engineering applications. Through a carefully designed student training program and partnerships with regional and other universities, XSEDE, and Women in HPC programs, this project will develop computing competency in the next-generation workforce, and engage and train a broader audience including underrepresented students at minority-serving and EPSCoR (Established Program to Stimulate Competitive Research) institutions.Built with a forward-looking architecture with a high core count, and improved memory bandwidth and I/O, Anvil can effectively support traditional HPC with fast turnaround for high throughput, mid-scale computation jobs. Anvil consists of 1000 128-core computing nodes based on the next-generation AMD Epyc “Milan" architecture that can deliver a total peak performance of 5.3 Petaflops. Each node has 256 GB of memory, and a 100 gigabits/second bandwidth from the Mellanox HDR InfiniBand interconnect, allowing multiple jobs of up to 1024 cores to be run at full speed over the interconnect fabric. These nodes are complemented by 32 large-memory nodes with 1 TB of RAM each, and 16 Nvidia GPU nodes with 4 “Volta Next” GPUs per node. The GPU nodes are capable of 1.57 petaflops of single-precision performance to support machine learning and a wide range of current and future science and engineering applications. Anvil’s multiple tiers of storage systems include a long-term archive, persistent file and campaign storage, a 10 PB scratch file system, a 3 PB flash burst buffer, and object storage to support a variety of workflows and storage needs. Anvil will lower the barrier to entry to advanced computing CI by providing interactive computing and desktop environments that ease the transition for users from diverse domains new to HPC. By providing feature-rich interactive environments such as Open OnDemand and ThinLinc, users can rapidly become productive on Anvil through Linux and Windows desktops, or familiar tools through their browser (e.g., Jupyter, RStudio). Complex scientific software environments and application stacks will be supported via containers orchestrated within a powerful composable subsystem. Anvil supports cloud-bursting of computational workloads as well as use of public cloud machine learning platforms including GPU and FPGA accelerators and software tools to automate hyperparameter tuning and algorithm selection for exploratory ML research. An existing production-quality science gateway at Purdue will support XSEDE researchers to share their data and tools online and facilitate easy access to Anvil and other XSEDE resources in classroom instruction and training activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着计算几乎渗透到科学和工程的所有领域，传统的计算密集型领域和新兴的、更多样化的研究领域对计算的需求都呈指数级增长。机器学习和人工智能应用的兴起加速并扩大了计算资源的使用，从创造更环保的新材料的研究，到在抗击致命疾病方面改进医学。满足这种快速发展的国家计算需求面临三个主要挑战：能力不足、日益多样化的应用程序以及计算能力和培训。该项目旨在通过开发和部署可组合的高级计算资源Anvil来应对这些挑战并转变提供计算的方式，以显著提高计算能力和可访问性。ANVIL将大容量高性能计算(HPC)集群与软件、访问接口、编程环境和可组合服务的全面生态系统集成在一起，形成能够支持当前和未来各种科学和工程应用的无缝环境。通过精心设计的学生培训计划，以及与地区和其他大学、XSEDE和女性参与HPC计划的合作伙伴关系，该项目将培养下一代劳动力的计算能力，并吸引和培训更广泛的受众，包括少数族裔服务机构和EPSCoR(既定计划以刺激竞争研究)机构中代表性不足的学生。Anvil构建了具有高核心计数的前瞻性架构，并改进了内存带宽和I/O，可以有效地支持传统HPC快速完成高吞吐量、中型计算任务。ANVIL由1000个128核计算节点组成，基于新一代AMD EPYEC“米兰”架构，可提供总计5.3千万亿次峰值性能。每个节点具有256 GB内存和来自Mellanox HDR InfiniBand互连的100 Gb/秒带宽，允许在互连交换矩阵上全速运行多达1024个核心的多个作业。这些节点还包括32个大内存节点和16个NVIDIA GPU节点，每个节点具有1 TB的RAM和4个“Volta Next”图形处理器。GPU节点能够提供1.57千万亿次单精度性能，以支持机器学习以及当前和未来的各种科学和工程应用。Anvil的多层存储系统包括长期归档、持久文件和活动存储、10PB临时文件系统、3PB闪存猝发缓冲区和对象存储，以支持各种工作流程和存储需求。ANVIL将通过提供交互式计算和桌面环境来降低进入高级计算CI的门槛，使用户从不同的新领域过渡到HPC变得容易。通过提供功能丰富的交互环境，如Open OnDemand和ThinLinc，用户可以通过Linux和Windows桌面，或通过他们的浏览器(例如，Jupyter，RStudio)，在Anvil上快速提高工作效率。复杂的科学软件环境和应用程序堆栈将通过在强大的可组合子系统中编排的容器来支持。ANVIL支持计算工作负载的云突发，以及使用公共云机器学习平台，包括GPU和FPGA加速器和软件工具，以自动进行超参数调整和算法选择，以进行探索性ML研究。普渡大学现有的生产质量科学门户将支持XSEDE研究人员在线共享他们的数据和工具，并在课堂教学和培训活动中方便地访问Anvil和其他XSEDE资源。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Understanding Factors that Influence Research Computing and Data Careers

了解影响研究计算和数据职业的因素

• DOI: 10.1145/3491418.3530292
• 发表时间: 2022
• 期刊: Practice and Experience in Advanced Research Computing (PEARC22
• 作者: Chaudhry, Shafaq;Pazouki, Arman;Schmitz, Patrick;Hillery, Elizabett;Kee, Kerk
• 通讯作者: Kee, Kerk

Defining Performance of Scientific Application Workloads on the AMD Milan Platform

定义 AMD Milan 平台上科学应用程序工作负载的性能

• DOI: 10.1145/3437359.3465596
• 发表时间: 2021
• 期刊: Practice and Experience in Advanced Research Computing
• 作者: Wu, Tsai-Wei;Lien Harrell, Stephen;Lentner, Geoffrey;Younts, Alex;Weekly, Sam;Mertes, Zoey;Maji, Amiya;Smith, Preston;Zhu, Xiao
• 通讯作者: Zhu, Xiao

Anvil - System Architecture and Experiences from Deployment and Early User Operations

Anvil - 系统架构以及部署和早期用户运营的经验

• DOI: 10.1145/3491418.3530766
• 发表时间: 2022
• 期刊: PEARC '22: Practice and Experience in Advanced Research Computing
• 作者: Song, X. Carol;Smith, Preston;Kalyanam, Rajesh;Zhu, Xiao;Adams, Eric;Colby, Kevin;Finnegan, Patrick;Gough, Erik;Hillery, Elizabett;Irvine, Rick
• 通讯作者: Irvine, Rick

Cyberinfrastructure for sustainability sciences

可持续科学的网络基础设施

• DOI: 10.1088/1748-9326/acd9dd
• 发表时间: 2023
• 期刊: Environmental Research Letters
• 影响因子: 6.7
• 作者: Song, Carol X.;Merwade, Venkatesh;Wang, Shaowen;Witt, Michael;Kumar, Vipin;Irwin, Elena;Zhao, Lan;Walton, Amy
• 通讯作者: Walton, Amy