SDCI Net: Collaborative Research: An integrated study of datacenter networking and 100 GigE wide-area networking in support of distributed scientific computing

随着超级计算速度提高到Peta和Exaflop，科学家们正在扩大他们的模拟规模和范围，这导致了不断增长的数据集，需要将其转移到科学家自己实验室的本地计算机上。该项目的第一个目标是通过与社区地球系统模型(CESM)项目的科学家合作，使用数据收集和分析来确定导致端到端应用程序级别吞吐量较差和/或不一致的瓶颈。在了解了端到端链中最薄弱的组件后，我们计划在受控环境中进行试验，该试验床由NERSC的高端集群(能够以接近100Gbps的速度将数据采购/下沉到磁盘)和通过DOE 100Gbps高级网络计划(ANI)原型网络连接的其他高性能计算系统组成。多种数据中心网络技术，如融合以太网上的远程直接存储器访问(ROCE)和互联网广域RDMA协议(IWARP)，将分别与高速(100 Gb/S)广域网络解决方案(如专用虚电路和IP路由路径)相结合，以比较文件传输和广域MPI/O的性能。扩展套接字API(ES-API)的新软件模块将被原型化，并集成到文件传输应用程序中。最后，将组织试验，将确定的最佳解决方案移交给CESM和其他科学家。拟议项目的智力价值包括：i)一种系统的科学方法，以确定CESM科学家经历的端到端应用程序级别性能较差的原因；ii)开发综合数据中心和广域联网解决方案以解决所确定的问题；以及iii)使这些解决方案能够被CESM和其他科学项目利用。拟议活动的更广泛影响包括：i)创建关于数据中心网络的课程模块，并让所有三所机构的本科生参与这项研究；ii)多样性和外展计划；以及iii)积极促进为CESM项目和其他科学家开发的解决方案。

As supercomputing speeds increase to peta- and exaflops, scientists are increasing their scale and range of simulations, which are resulting in ever growing datasets that need to be moved to local computers at the scientists' own laboratories. The first goal of this project is to identify bottlenecks that result in poor and/or inconsistent end-to-end application-level throughput using data collection and analysis by working in conjunction with scientists in the Community Earth System Model (CESM) project. With knowledge of the weakest components in the end-to-end chain, we plan to experiment in a controlled environment using a testbed that consists of a high-end cluster at NERSC, which is capable of sourcing/sinking data to disks at close to 100Gbps speeds, and other high-performance computing systems connected via the DOE 100Gbps Advanced Networking Initiative (ANI) prototype network. Multiple datacenter networking technologies such as Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) and Internet Wide-Area RDMA Protocol (iWARP) will be combined with high- speed (100Gb/s) wide-area networking solutions, such as dedicated virtual circuits and IP-routed paths, respectively, for a comparative performance study of file transfers and wide-area MPI I/O. A new software module of the Extended-Sockets API (ES-API), which offers RDMA features such as zero-copy operations, will be prototyped and integrated into file transfer applications. Finally, trials will be organized to transfer the best identified solutions to CESM and other scientists. The intellectual merit of the proposed project consists of: i) a systematic scientific approach to determine the reasons for poor end-to-end application-level performance experienced by CESM scientists, ii) development of integrated datacenter and wide-area networking solutions to address the identified problems, and iii) the enabling of these solutions to be utilized by CESM and other science projects. The broader impacts of the proposed activities consist of i) the creation of a course module on datacenter networking, and the involvement of undergraduate students in this research at all three institutions, ii) diversity and outreach programs, and iii) the active promotion of the developed solutions to the CESM project and other scientists.