MRI:Development of Quartzite, a Campus-wide, Terabit-Class, Field-Programmable, Hybrid Switching Instrument for Comparative Studies

该项目旨在建设一个支持科学应用和大容量数据实验的校园级超高速光纤网络，开发一种实验性的下一代仪器，以有效地研究和比较校园级太比特级lambda网络架构，从纯光电路到纯分组交换网络（以及两者之间的一系列混合组合）。当前的商业存储系统方法在性能水平和数据抽象方面都不能扩展。所建议的方法建立在无共享计算集群的基础上，这些集群来自数据系统、可视化墙和高端仪器接口，具有服务和摄取应用程序所需的大量数据的原始能力。构建同时高效交换10gb流的下一代交换机，旨在构建21世纪的光子仪器，以探索在带宽丰富的基础设施中网络和应用设计的实际权衡。为了支持大型科学问题和大型模拟，该项目构建了下一代实验性仪器“石英岩”。在促进比较研究的同时，数据密集型应用程序的面包板Quartzite可以将资源拼接在一起，将它们虚拟地引入。因此，这个波长选择性交换机创建、通信和交付项目将混合网络结构添加到独特的校园规模平台上，并使研究网络架构和应用程序设计成为可能，在带宽丰富的基础设施中，以及跨集群共享大型数据集。这项工作涉及高风险，但有可能产生更大的影响，因为数据密集型科学探索可以带入科学家的实验室，通过使用按需高速数据流来利用校园到国际规模的资源。作品探讨了以下问题：终端用户应用程序如何利用剩余的按需带宽，如何最佳地构建分布式系统，何时需要非共享分组网络，如何处理混合结构的控制，应用程序能否真正利用高速并行基础设施，动态重新配置校园网以满足瞬时容量需求是否可行，将直接电路暴露给各个端点是否有益，以及如何实现这些问题。采用新颖的分组调度策略，显著提高了共享链路的容量。更广泛的影响：基于石英岩的比较将影响未来研究型大学网络的网络结构，极大地提高全国范围内数据密集型研究的能力。与工业合作伙伴合作，混合石英岩心系统和软件将为我们所有人服务。

This project, building a campus wide ultra high-speed optical fiber network that supports scientific application and experiments of high volume data, develops an experimental next-generation instrument to efficiently investigate and compare campus-scale terabit-class lambda network architectures that span from optical-circuits-only to packet-switched-only networks (and a range of hybrid combinations in between). Current commercial approaches to storage systems do not scale in either performance levels or data abstractions. The proposed approach builds on the foundation of the shared-nothing compute cluster emerging from data systems, visualization walls, and high-end instrument interfaces, having raw horsepower to serve and ingest high volumes of data required by applications. Constructing a next generation switch for simultaneously switching 10Gbs streams efficiently, the work aims at building a 21st century photonic instrument to explore the practical tradeoffs of network and application design in bandwidth-rich infrastructure. Supporting large scientific problems and enabling big simulations, the project constructs Quartzite, the experimental, next-generation instrument. While fostering comparative studies, Quartzite, a data-intensive application breadboard, enables stitching together resources, bringing them virtually in. Thus, this wavelength-selective switch creation, communication and delivery project, adds hybrid-networking structure to a unique campus-scale platform and enables the study of network architecture and application design in a band-width-rich infrastructure and the sharing of large data sets across clusters. The work involves high risk, with a promise of even higher impact, since data intensive scientific exploration can be brought into the scientists' lab, by using on-demand high-speed data flows to harness campus- to international-scale resources. The work explores the following issues: How surplus of on-demand bandwidth can be exploited by end user applications, How distributed systems can be best architected, When is a non-shared packet network needed, How should control of a hybrid fabric be handled, Can applications truly exploit a high-speed parallel infrastructure, Is dynamic reconfiguring of campus network to meet transient capacity demands practical, Is it beneficial to expose direct circuits to individual endpoints, and Do novel packet scheduling strategies for shared links dramatically improve the capacity.Broader Impact: The Quartzite-enabled comparisons will influence the network structure of future research university networks, greatly increasing the capability for data-intensive research throughout the country. Working with industrial partners, the hybrid Quartzite core system and software will service us all.