SDCI Net: Collaborative Research: An integrated study of datacenter networking and 100 GigE wide-area networking in support of distributed scientific computing

随着超级计算速度提高到peta和exaflops，科学家们正在增加他们的模拟规模和范围，这导致越来越多的数据集需要转移到科学家自己实验室的本地计算机上。该项目的第一个目标是通过与社区地球系统模型（CESM）项目中的科学家合作，使用数据收集和分析来确定导致端到端应用级吞吐量差和/或不一致的瓶颈。了解了端到端链中最薄弱的组件后，我们计划在一个受控环境中进行实验，使用一个测试平台，该平台由NERSC的高端集群组成，能够以接近100 Gbps的速度将数据源/宿到磁盘，以及通过DOE 100 Gbps高级网络计划（ANI）原型网络连接的其他高性能计算系统。多种数据中心网络技术，如融合以太网远程直接内存访问（RDMA）（RoCE）和互联网广域RDMA协议（iWARP）将分别与高速（100 Gb/s）广域网络解决方案（如专用虚拟电路和IP路由路径）相结合，用于文件传输和广域MPI I/O的比较性能研究。扩展套接字API（ES-API）的一个新的软件模块将被原型化并集成到文件传输应用程序中，该模块提供零拷贝操作等RDMA功能。最后，将组织试验，将确定的最佳解决方案转让给CESM和其他科学家。拟议项目的智力价值包括：i）系统的科学方法，以确定CESM科学家经历的端到端应用级性能差的原因，ii）开发集成数据中心和广域网解决方案，以解决已确定的问题，以及iii）使这些解决方案能够被CESM和其他科学项目利用。拟议活动的更广泛影响包括：i）创建数据中心网络课程模块，以及所有三个机构的本科生参与这项研究，ii）多样性和推广计划，以及iii）积极推广CESM项目和其他科学家的解决方案。

As supercomputing speeds increase to peta- and exaflops, scientists are increasing their scale and range of simulations, which are resulting in ever growing datasets that need to be moved to local computers at the scientists' own laboratories. The first goal of this project is to identify bottlenecks that result in poor and/or inconsistent end-to-end application-level throughput using data collection and analysis by working in conjunction with scientists in the Community Earth System Model (CESM) project. With knowledge of the weakest components in the end-to-end chain, we plan to experiment in a controlled environment using a testbed that consists of a high-end cluster at NERSC, which is capable of sourcing/sinking data to disks at close to 100Gbps speeds, and other high-performance computing systems connected via the DOE 100Gbps Advanced Networking Initiative (ANI) prototype network. Multiple datacenter networking technologies such as Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) and Internet Wide-Area RDMA Protocol (iWARP) will be combined with high- speed (100Gb/s) wide-area networking solutions, such as dedicated virtual circuits and IP-routed paths, respectively, for a comparative performance study of file transfers and wide-area MPI I/O. A new software module of the Extended-Sockets API (ES-API), which offers RDMA features such as zero-copy operations, will be prototyped and integrated into file transfer applications. Finally, trials will be organized to transfer the best identified solutions to CESM and other scientists. The intellectual merit of the proposed project consists of: i) a systematic scientific approach to determine the reasons for poor end-to-end application-level performance experienced by CESM scientists, ii) development of integrated datacenter and wide-area networking solutions to address the identified problems, and iii) the enabling of these solutions to be utilized by CESM and other science projects. The broader impacts of the proposed activities consist of i) the creation of a course module on datacenter networking, and the involvement of undergraduate students in this research at all three institutions, ii) diversity and outreach programs, and iii) the active promotion of the developed solutions to the CESM project and other scientists.