CC-NIE Integration: High Performance Computing with Data and Networking Acceleration (HPCDNA)

未来的网络基础设施越来越多地围绕着大数据、计算和高性能网络的集成。数据和网络加速的高性能计算（HPCDNA）项目侧重于校园科学和计算需求背景下的这些问题。许多研究人员拥有特定于应用的数据和各种所需的计算环境，包括专门构建的小型实验室计算资源、中等规模的校园高性能计算（HPC）、云计算和国家规模的分布式或集中式资源。无法灵活、无缝地将数据获取到最合适的计算资源，这通常是决定在何处运行计算和执行哪些计算的限制因素。HPCDNA项目正在开发技术，以极大地提高在这些不同的计算资源中利用公共数据集的能力。HPCDNA架构和技术包括一个基于分布式高性能并行文件系统的网络嵌入式存储（NES）系统，该系统部署在中大西洋十字路口区域网络的核心。该系统通过与校园HPC、校园网和广域网的紧密集成，增强了高性能。网元客户端接口正在开发中，实现存储与网络动态发放的协调，通过工作流流程实现多种高性能的访问方式。预计HPCDNA系统将促进研究人员扩展HPC和云的使用，这将允许采用更雄心勃勃的目标，因为他们的计算环境可以随着他们的问题空间而扩展。

Future cyber-infrastructures are increasingly expected to revolve around the integration of big data, computation, and high performance networking. The High Performance Computing with Data and Networking Acceleration (HPCDNA) project focuses on these issues in the context of campus science and computational requirements. Many researchers have application specific data and a variety of desired compute environments, including purpose built small lab compute resources, medium scale campus High Performance Computing (HPC), cloud computing, and national scale distributed or centralized resources. The inability to flexibly and seamlessly get data to the most appropriate compute resource is often the limiting factor determining where the computation is run and what computation is performed. The HPCDNA project is developing technologies to greatly improve the ability to utilize common data sets across this diverse set of computational resources.The HPCDNA architecture and technologies include a Network Embedded Storage (NES) system based on a distributed high performance parallel file system deployed in the core of the Mid-Atlantic Crossroads regional network. This system is enhanced for high performance via tight integration with campus HPC, campus networks, and wide area networks. An NES client interface is being developed, which coordinates storage with dynamic network provisioning, and enables a variety of high performance access methods by workflow processes. It is expected that the HPCDNA system will facilitate expanded use of HPC and clouds by researchers, which will allow the adoption of more ambitious goals with the knowledge that their computing environments can scale up along with their problem space.