Collaborative Research: CNS Core: Small: A New Architecture for Petabyte-scale File Transfer Evaluated in FABRIC

合作研究：CNS 核心：小型：在 FABRIC 中评估的 PB 级文件传输新架构

• 批准号: 2215671
• 负责人: Violet Syrotiuk
• 金额: $ 32.4万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2215671&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Small

File transfer is a fundamental operation of the Internet. Important scientific instruments, such as the James Webb Space Telescope and the Large Hadron Collider, generate massive files daily. As files increase in size, it is more likely errors are introduced during their transfer, such as the ones not caught by the checksum process. Reliability is especially important when the files store data that capture a rare event, such as rare gravitational waves whose loss could significantly impact scientific conclusions if they go undetected. This collaborative project brings together investigators from Arizona State University and Boston University to develop a multi-layer error detection (MLED) architecture. MLED will not only eliminate detectable errors, it also provably reduces the probability of undetected errors.Each layer in the MLED architecture is parameterized by a policy that includes the scope of the error detection method among other requirements. When aggregated, the layers improve error detection capabilities and significantly reduce the probability of undetected errors. This project has three research directions: (1) to develop the theory of MLED to study trade-offs among policy parameters to optimize the probability of undetected errors and file transfer delay; (2) to design a software-defined implementation of MLED exploiting features in FABRIC, a new testbed of networked computing systems; (3) to evaluate MLED to investigate and understand the design trade-offs that must be navigated to harness FABRIC, to validate the theory, and to compare to other tools across the spectrum of file transfer solutions. This project impacts the science of understanding networked computing systems through the design and implementation of the MLED architecture using features of FABRIC that are unavailable in other network testbeds. Reliable file transfer is especially important when the conclusions drawn from the data critically depend on their correctness, with potential consequences to scientific knowledge. In addition, MLED advances data science to better understand undetected errors, their propagation through file transfer, and management of massive data sets. FABRIC Across Borders offers the opportunity to use MLED for petabyte-scale file transfer across the oceans. Outreach activities will share experience using FABRIC with the testbed community and promote the adoption of MLED for file transfer. By the end of this project, it will have contributed towards NSF and national research priorities on networked computing systems, and helped educate graduate and undergraduate students, while broadening participation of underrepresented minorities.Project website: https://www.public.asu.edu/~syrotiuk/mled.html. Jupyter Notebooks will be used throughout the project to prototype, explore, and document the experimental process. To enable reproducibility the notebooks and code repositories will be shared on the project website, in addition to resulting publications. This website will be maintained for at least three years following the project end date.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

文件传输是互联网的一项基本操作。重要的科学仪器，如詹姆斯·韦伯太空望远镜和大型强子对撞机，每天都会产生大量的文件。随着文件大小的增加，更有可能在传输过程中引入错误，比如校验和过程没有捕获到的错误。当文件存储捕获罕见事件的数据时，可靠性尤为重要，例如罕见的引力波，如果它们未被发现，其丢失可能会严重影响科学结论。该合作项目汇集了来自亚利桑那州立大学和波士顿大学的研究人员，共同开发多层错误检测（MLED）架构。MLED不仅可以消除可检测的错误，还可以降低未检测到错误的概率。MLED体系结构中的每一层都由一个策略参数化，该策略包括错误检测方法的范围以及其他需求。当聚合时，这些层提高了错误检测能力，并显著降低了未检测到错误的概率。本课题有三个研究方向：(1)发展MLED理论，研究策略参数之间的权衡，以优化未被检测到的错误概率和文件传输延迟；(2)利用网络计算系统的新测试平台FABRIC的特性，设计一种软件定义的MLED实现；(3)评估MLED，以调查和理解必须导航的设计权衡，以利用FABRIC，验证理论，并与跨文件传输解决方案的其他工具进行比较。该项目通过使用FABRIC的特性设计和实现MLED架构，影响了理解网络计算系统的科学，而这些特性在其他网络测试平台中是不可用的。当从数据中得出的结论严重依赖于其正确性时，可靠的文件传输尤为重要，这对科学知识有潜在的影响。此外，MLED还推动了数据科学的发展，以更好地理解未检测到的错误、它们通过文件传输的传播以及大量数据集的管理。FABRIC Across Borders提供了使用MLED进行pb级跨洋文件传输的机会。外展活动将与测试平台社区分享使用FABRIC的经验，并促进采用MLED进行文件传输。到这个项目结束时，它将为美国国家科学基金会和国家网络计算系统的研究重点做出贡献，并帮助教育研究生和本科生，同时扩大代表性不足的少数民族的参与。项目网站：https://www.public.asu.edu/~syrotiuk/mled.html。Jupyter笔记本将在整个项目中用于原型，探索和记录实验过程。为了实现再现性，除了最终的出版物之外，笔记本和代码库将在项目网站上共享。本网站将在项目结束日期后至少维持三年。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。