CAREER: Efficient and Reliable Data Transfer Services for Next Generation Research Networks

研究网络对于数据密集型、分布式和协作科学项目至关重要，因为它们在研究和教育机构之间提供高速连接。然而，研究网络的用户无法有效地利用可用资源，因为现有的传输应用程序在高速下遭受可扩展性问题。该项目为下一代研究网络设计和开发了一个可扩展的、强大的数据传输框架，以提高其利用率。研究网络中增强的网络性能允许无缝执行下一代分布式科学应用，从而为迅速取得突破性发现铺平道路。该项目还通过减少共享数据所需的时间，促进地理上分散的机构的科学家之间的合作。除了研究贡献外，该项目还将强大的教育计划紧密结合到其研究计划中。该计划包括为科学家提供培训，帮助他们在处理大规模数据时更好地利用先进的网络基础设施资源，为初中和高中学生开发游戏，教授网络概念，以及为高中学生提供暑期学校，为代表性不足的群体教授编程和网络。随着数据密集型分布式科学趋势的持续，开发可以扩展到下一代太比特每秒网络及更高级别的数据传输服务变得越来越重要。为了实现这一目标，该项目集中了四个关键的研究方向：首先，它创新了一个模块化的文件传输架构，将I/O操作与网络传输分离，以实现动态和组件特定的调优。其次，它实现了对延迟敏感的分布式工作流的服务质量支持，以满足其严格的性能要求。第三，通过网络缓存/计算和概率错误检查机制，开发了可扩展的，安全的，低开销的文件传输完整性验证。第四，它将开发的算法集成到常用的工作流程管理系统中，以增加其在更广泛的科学界的采用。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Research networks are crucial for data intensive, distributed, and collaborative science projects as they provide high speed connectivity between research and education institutions. However, users of research networks are unable to efficiently utilize available resources as existing transfer applications suffer from scalability issues at high speeds. This project designs and develops a scalable and robust data transfer framework for next-generation research networks to improve their utilization. Enhanced network performance in research networks allows seamless execution of next generation distributed science applications, thereby paving the way for breakthrough discoveries to be made swiftly. This project also promotes collaboration between scientists at geographically separated institutions by means of reducing the time it takes to share data. In addition to research contributions, this project has strong education plan tightly integrated into its research plan. The plan involves trainings for scientists to help them better utilize advanced cyberinfrastructure resources when dealing with large scale data, game development for middle and high school students to teach networking concepts, and summer schools for high school students for underrepresented groups to teach programming and networking. As trend towards data intensive distributed science continues, it is becoming increasingly important to develop data transfer services that can scale to next generation terabit per second networks and beyond. To achieve this goal, this project focuses four key research directions: First, it innovates a modular file transfer architecture to separate I/O operations from network transfers to enable dynamic and component specific tuning. Second, it implements Quality of Service support for delay sensitive distributed workflows to meet their stringent performance requirements. Third, it develops scalable, secure, and low overhead integrity verification for file transfers through in network caching/computing and probabilistic error checking mechanisms. Fourth, it integrates the developed algorithms to commonly used workflow management systems to increase its adoption by a broader science community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.