Large Scale Data Transfer: Addressing a Critical Need with a Low Cost Internet2 Service

自从互联网出现以来，在用户之间发送文件的问题对于大多数服务器来说都太大了，这一问题一直困扰着用户。通常，让发送者和接收者在公共服务上建立帐户以允许交换他们各自的身份的简单过程导致实际发送文件的延迟，因为在可以启动文件交换之前需要知道该信息。Internet2在这项工作中致力于创建一个通用的批量数据传输服务，使研究人员能够在短时间内将10MB-1TB大小的数据集暂存。将提供100TB的初始数据存储作为短期存储库，以实现可靠的实施，以满足社区的需求，因为所有空间都不会长期分配，并将很快可供重复使用。这项工作侧重于适度的功能目标(仅解决交换文件的问题，而不是其长期存储和共享的问题)，同时强调覆盖全球，以便在没有高级用户规划或配置的情况下在大量人群中工作。这项工作的学术价值在于它独一无二地解决了一个影响广泛研究领域的问题，特别是那些处理大型文件数据传输的社区，以及它将通过一个已经在国际上部署的解决方案提供的价值。独立于任何社区基础设施的简单但全面的解决方案的存在将是整个R&A；E网络基础设施的关键组成部分。实现这一目标的核心软件FileSender已经发展成为一个国际开源社区，正在获得巨大的吸引力。通过在美国提供基于FileSender的服务，该项目将利用并扩展这一潜力，使其成为整个国际研究和教育界无处不在的通用解决方案。

Since the inception of the Internet, the problem of sending files between users that were too large for most servers has plagued users. Often the simple process of getting the sender and recipient to set up accounts on a common service to allow exchange of their respective identities results delays in actually sending a file, as this information needs to be known before a file exchange can be initiated. Internet2 pursues in this work the creation of a general bulk data transfer service that will allow researchers to stage datasets in the 10MB-1TB size range for short periods of time. An initial data store of 100TB will be provided as a short-term repository, allowing robust implementation to meet the needs of the community as none of the space will be allocated long-term and will be available for re-use quickly. This work focuses on modest functional goals (solving only the problem of exchanging files, rather than their long term storage and sharing) while emphasizing a global reach to work across a large population with no advanced user planning or provisioning. The intellectual merit of this work lies in its unique solution to a problem affecting a broad scope of research communities, particularly those that deal with large file data transfers, and in the value it will offer through a solution that already enjoys international deployment. The existence of a simple but comprehensive solution that is independent of any community infrastructure will be a critical component of the overall R&E cyberinfrastructure. FileSender, the core software to accomplish this, has evolved into an international open-source community that is gaining significant traction. By providing a service based on FileSender in the U.S., this project will capitalize on and extend the potential for this to become a ubiquitous and common solution throughout the international research and education community.