SHF: Large: Collaborative Research: Next Generation Communication Mechanisms exploiting Heterogeneity, Hierarchy and Concurrency for Emerging HPC Systems

该奖项得到了CIF21软件重用风险企业的部分支持，该风险企业的目标是通过它们的重用来支持通往可持续软件元素的道路，并强调可重用软件元素在可持续软件网络基础设施中的关键作用，以支持计算和数据启用的科学和工程。基于MPI(消息传递接口)的并行编程正在学术界、政府(国防和非国防用途)以及可扩展机器学习和大数据分析中得到越来越频繁的使用。密集多核(DMC)架构(如Intel的Knters Landing(KnL))和加速器/协处理器架构(如NVIDIA GPGPU)的出现使高计算密度系统的设计成为可能。这一点，再加上支持远程直接内存访问(RDMA)的商用网络技术的可用性，如InfiniBand、ROCE和带有iWARP的10/40GigE，正在推动多Petaflop和Exaflop系统的增长。这些DMC架构具有以下独特特征：更深层次的分层内存；革命性的网络互连；以及计算能力和数据移动成本的异构性(芯片级和节点级的异构性)。对于这些新兴系统，MPI和其他编程模型的组合被称为MPI X(其中X可以是PGAS、TASKS、OpenMP、OpenACC或CUDA)。当前一代的MPI X编程模型的通信协议和机制不能有效地支持新兴的DMC体系结构。这带来了以下广泛的挑战：1)如何为下一代DMC体系结构设计高性能和可扩展的通信机制，以支持MPI X(包括基于任务的)编程模型？2)如何利用所提出的通信机制来设计和共同设计当前和下一代应用程序？俄亥俄州立大学(OSU)和俄亥俄超级计算机中心(OSC)的计算机科学家以及德克萨斯高级计算中心(TACC)、圣地亚哥超级计算机中心(SDSC)和加州大学圣地亚哥分校(UCSD)的计算科学家提出了一个协同和综合的研究计划，以创新的解决方案来应对上述广泛的挑战。这项研究将由一组来自NSF计算科学研究人员的应用程序推动，这些研究人员在OSC和OSU的Stampede和Comet以及其他系统上进行大规模模拟。建议的设计将被整合到广泛使用的MVAPICH2库中，并供公众使用。多名研究生和本科生将在该项目下接受培训，成为HPC未来的科学家和工程师。在TACC、SDSC和OSC建立的全国性培训和推广方案将用于向XSEDE用户传播这项研究的结果。将在XSEDE、SC等会议上组织教程，与社区分享研究成果和经验。

This award was partially supported by the CIF21 Software Reuse Venture whose goals are to support pathways towards sustainable software elements through their reuse, and to emphasize the critical role of reusable software elements in a sustainable software cyberinfrastructure to support computational and data-enabled science and engineering.Parallel programming based on MPI (Message Passing Interface) is being used with increased frequency in academia, government (defense and non-defense uses), as well as emerging uses in scalable machine learning and big data analytics. The emergence of Dense Many-Core (DMC) architectures like Intel's Knights Landing (KNL) and accelerator/co-processor architectures like NVIDIA GPGPUs are enabling the design of systems with high compute density. This, coupled with the availability of Remote Direct Memory Access (RDMA)-enabled commodity networking technologies like InfiniBand, RoCE, and 10/40GigE with iWARP, is fueling the growth of multi-petaflop and ExaFlop systems. These DMC architectures have the following unique characteristics: deeper levels of hierarchical memory; revolutionary network interconnects; and heterogeneous compute power and data movement costs (with heterogeneity at chip-level and node-level). For these emerging systems, a combination of MPI and other programming models, known as MPI+X (where X can be PGAS, Tasks, OpenMP, OpenACC, or CUDA), are being targeted. The current generation communication protocols and mechanisms for MPI+X programming models cannot efficiently support the emerging DMC architectures. This leads to the following broad challenges: 1) How can high-performance and scalable communication mechanisms for next generation DMC architectures be designed to support MPI+X (including Task-based) programming models? and 2) How can the current and next generation applications be designed/co-designed with the proposed communication mechanisms?A synergistic and comprehensive research plan, involving computer scientists from The Ohio State University (OSU) and Ohio Supercomputer Center (OSC) and computational scientists from the Texas Advanced Computing Center (TACC), San Diego Supercomputer Center (SDSC) and University of California San Diego (UCSD), is proposed to address the above broad challenges with innovative solutions. The research will be driven by a set of applications from established NSF computational science researchers running large scale simulations on Stampede and Comet and other systems at OSC and OSU. The proposed designs will be integrated into the widely-used MVAPICH2 library and made available for public use. Multiple graduate and undergraduate students will be trained under this project as future scientists and engineers in HPC. The established national-scale training and outreach programs at TACC, SDSC and OSC will be used to disseminate the results of this research to XSEDE users. Tutorials will be organized at XSEDE, SC and other conferences to share the research results and experience with the community.