SHF: Small: Understanding and Exploiting Software Defined Networks (SDN) in High Performance Computing (HPC) Environments

在现代社会，许多科学和技术领域，如物理学、生物学和气象学，依靠强大的超级计算机来进行模拟和数据分析，以推进知识和服务社会。这些超级计算机使用专门的硬件，制造和操作都很昂贵。该项目旨在开发新技术，通过利用软件定义网络（SDN）技术的能力来构建高性能和低成本的超级计算机，该技术已证明其在数据中心中的有效性，以降低设置和管理成本。如果成功，这项研究将改变超级计算机的构造方式，并降低成本。这将使高性能计算（HPC）为广大受众所接受，并有助于许多领域的进步。该项目还将通过整合研究和教育的课程开发工作来支持教育。SDN以逻辑集中的方式管理网络流量和资源，使其能够为给定的优化目标寻求全球接近最优的资源管理解决方案。这可以提高现有互连网络技术的性能，包括高端高性能计算系统中的专有技术。此外，与专有技术相比，SDN的广泛商用将加速技术进步，提高性能并降低成本。在这个项目中，有效的SDN技术可以导致比目前的高性能计算系统更高的性能开发；建立了高性能计算环境下SDN在不同约束条件下的性能界限。此外，还开发和评估了可以与当前技术一起部署的实用技术。这些技术分布在软件堆栈的不同层（包括网络、操作系统和运行时系统），并协同工作以利用SDN功能。最后，实现并部署了一个具有sdn功能的高性能计算系统，该系统集成了所开发的技术。通过分析、设计、开发和实验，明确了SDN相对于高性能计算系统组网技术的优缺点。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In modern society, many science and technology domains such as physics, biology, and meteorology rely on powerful supercomputers to perform simulations and data analysis to advance knowledge and to serve the society. These supercomputers use specialized hardware and are expensive to build and operate. This project aims at developing novel techniques to build high-performance and low-cost supercomputers by exploiting the capabilities of the Software Defined Networking (SDN) technology that has demonstrated its effectiveness in data centers to reduce cost of setup and management. If successful, this research will change how supercomputers are constructed, and lower the cost. This will make high-performance computing (HPC) accessible to a broad audience and help the advances in many domains. The project will also support education through a curriculum development effort that integrates research and education. SDN manages network traffic and resources in a logically centralized manner, enabling it to seek globally near-optimal resource management solutions for a given optimization objective. This can lead to improved performance over existing interconnection network technologies, including the proprietary ones in high-end HPC systems. Additionally, SDN's widespread commodity use will accelerate technological advances with improved performance and reduced cost in comparison to proprietary technologies. In this project, effective SDN techniques that can lead to higher performance than the current HPC systems are developed; the performance bounds of SDN under different constraints in the HPC environment are established. Additionally, practical techniques that can be deployed with the current technology are developed and evaluated. These techniques spread across different layers of the software stack including network, operating system, and run-time system, and work in concert to exploit the SDN capabilities. Finally, a working proof-of-concept SDN-capable HPC system that incorporates the developed techniques is implemented and deployed. Through analysis, design, development, and experimentation, the strengths and weaknesses of SDN in comparison to the networking technology in HPC systems are clearly identified.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.