Collaborative Research: Topology-Aware MPI Communication and Scheduling for Petascale Systems

该奖项是根据2009年美国复苏和再投资法案（公法111-5）资助的。现代网络（如InfiniBand和10 GigE）具有在运行时提供拓扑、路由以及网络状态信息的能力。这导致了以下广泛的挑战：下一代千万亿次系统能否提供拓扑感知的MPI通信，映射和调度，可以提高性能和可扩展性的应用范围？这一挑战引出了以下研究问题：1）千万亿次应用的拓扑感知通信和调度需求是什么？2)如何利用静态和动态网络信息设计网络拓扑和状态管理框架？3)如何在MPI库中设计拓扑感知的点对点和集体通信方案（如广播，all-to-all，all-reduce）？4)如何设计拓扑感知的任务映射和调度方案？如何定义和设计灵活的拓扑信息接口？一项协同而全面的研究计划，涉及来自俄亥俄州州立大学（OSU）的计算机科学家以及来自德克萨斯州高级计算中心（TACC）和加利福尼亚大学的计算科学家。，圣地亚哥，圣地亚哥超级计算机中心（SDSC），建议解决上述挑战。该研究将由一组应用程序（PSDNS，UCSDH 3D，AWM-Olsen和MPCUGLES）驱动，这些应用程序来自已建立的NSF计算科学研究人员，他们在Ranger系统和其他NSF HEC系统上运行大规模模拟。所提出的研究的变革性影响是开发拓扑感知的MPI软件和框架，用于使用派生的拓扑信息进行调度集成，以最大限度地提高千万亿次应用程序的性能。拟议的研究是计算机科学家和计算科学家之间的协作和协同活动，因此，将对设计，部署和使用下一代千万亿次系统的指导方针产生重大影响。所提出的研究方向及其解决方案将用于研究人员的课程，以培训研究生和本科生。在TACC和SDSC建立的全国范围的培训和推广计划将用于向HEC用户和开发人员传播这项研究的结果。研究结果还将分发给研究人员的多个合作组织（国家实验室和工业界），以便对其软件产品和应用产生影响。修改后的MVAPICH 2库（目前被840多个组织使用）和SGE调度器插件将以开源方式提供给HEC社区。本研究的案例研究将在MPI论坛（OSU是该论坛的成员）上展示，以指导即将到来的MPI-3标准和其他MPI库的设计。

Abstract This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5). Modern networks (like InfiniBand and 10GigE) have capability to provide topology, routing and also network status information at run-time. This leads to the following broad challenge: Can the next generation petascale systems provide topology-aware MPI communication, mapping and scheduling which can improve performance and scalability for a range of applications? This challenge leads to the following research questions: 1) What are the topology- aware communication and scheduling requirements of petascale applications? 2) How to design a network topology and state management framework with static and dynamic network information? 3) How to design topology-aware point-to-point and collective communication schemes (such as broadcast, all-to-all, all-reduce) in an MPI library? 4) How to design topology-aware task mapping and scheduling schemes? and 5) How to define and design a flexible topology information interface? A synergistic and comprehensive research plan, involving computer scientists from The Ohio State University (OSU) and computational scientists from the Texas Advanced Computing Center (TACC) and The Univ. of Calif., San Diego, San Diego Supercomputer Center (SDSC), is proposed to address the above challenges. The research will be driven by a set of applications (PSDNS, UCSDH3D, AWM-Olsen and MPCUGLES) from established NSF computational science researchers running large scale simulations on the Ranger system and other NSF HEC systems. The transformative impact of the proposed research is to develop topology-aware MPI software and a framework for using derived topology information for scheduling integration in order to maximize petascale application performance. The proposed research is a collaborative and synergistic activity between computer scientists and computational scientists and thus, will have significant impact in deriving guidelines for designing, deploying and using next generation petascale systems. The proposed research directions and their solutions will be used in curriculum of the investigators to train graduate and undergraduate students. The established national-scale training and outreach programs at TACC and SDSC will be used to disseminate the results of this research to HEC users and developers. Research results will also be disseminated to the multiple collaborating organizations of the investigators (national laboratories and industry) to enable impact on their software products and applications. The modified MVAPICH2 library (currently being used by more than 840 organizations) and SGE scheduler plug-in will be available to the HEC community in an open-source manner. Case-studies from this research will be presented at the MPI Forum (OSU is a member of this forum) to influence the design of the upcoming MPI-3 standard and other MPI libraries.