SHF: Small: Understanding and Exploiting Software Defined Networks (SDN) in High Performance Computing (HPC) Environments

在现代社会中，许多科学和技术领域，如物理学，生物学和气象学，依靠强大的超级计算机来执行模拟和数据分析，以促进知识和服务于社会。这些超级计算机使用专门的硬件，建造和运行成本高昂。该项目旨在开发新技术，通过利用软件定义网络（SDN）技术的功能来构建高性能和低成本的超级计算机，该技术已在数据中心证明其有效性，以降低设置和管理成本。如果成功，这项研究将改变超级计算机的构造方式，并降低成本。这将使高性能计算（HPC）可供广大受众使用，并有助于许多领域的进步。该项目还将通过结合研究和教育的课程编制工作支持教育。SDN以逻辑集中的方式管理网络流量和资源，使其能够针对给定的优化目标寻求全局接近最优的资源管理解决方案。这可以提高现有互连网络技术的性能，包括高端HPC系统中的专有技术。此外，SDN的广泛商品使用将加速技术进步，与专有技术相比，性能更高，成本更低。在这个项目中，有效的SDN技术，可以导致更高的性能比当前的HPC系统的开发，在HPC环境下的不同约束下的SDN的性能界限建立。此外，开发和评估了可以使用当前技术部署的实用技术。这些技术分布在软件栈的不同层，包括网络、操作系统和运行时系统，并协同工作以利用SDN功能。最后，一个工作的概念验证的SDN能力的HPC系统，结合开发的技术实施和部署。通过分析、设计、开发和实验，SDN与HPC系统中的网络技术相比的优势和劣势得到了明确的识别。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响评审标准进行评估，被认为值得支持。

In modern society, many science and technology domains such as physics, biology, and meteorology rely on powerful supercomputers to perform simulations and data analysis to advance knowledge and to serve the society. These supercomputers use specialized hardware and are expensive to build and operate. This project aims at developing novel techniques to build high-performance and low-cost supercomputers by exploiting the capabilities of the Software Defined Networking (SDN) technology that has demonstrated its effectiveness in data centers to reduce cost of setup and management. If successful, this research will change how supercomputers are constructed, and lower the cost. This will make high-performance computing (HPC) accessible to a broad audience and help the advances in many domains. The project will also support education through a curriculum development effort that integrates research and education. SDN manages network traffic and resources in a logically centralized manner, enabling it to seek globally near-optimal resource management solutions for a given optimization objective. This can lead to improved performance over existing interconnection network technologies, including the proprietary ones in high-end HPC systems. Additionally, SDN's widespread commodity use will accelerate technological advances with improved performance and reduced cost in comparison to proprietary technologies. In this project, effective SDN techniques that can lead to higher performance than the current HPC systems are developed; the performance bounds of SDN under different constraints in the HPC environment are established. Additionally, practical techniques that can be deployed with the current technology are developed and evaluated. These techniques spread across different layers of the software stack including network, operating system, and run-time system, and work in concert to exploit the SDN capabilities. Finally, a working proof-of-concept SDN-capable HPC system that incorporates the developed techniques is implemented and deployed. Through analysis, design, development, and experimentation, the strengths and weaknesses of SDN in comparison to the networking technology in HPC systems are clearly identified.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.