EAGER: CloudJoin: Migrating CISE Computing Research Infrastructure to the Cloud

在过去的十年里，大规模数据中心计算和网络的进步加速了整个计算机科学的创新。大规模的数据密集型系统实验对于加速科学越来越重要，培养一支能够在大型基于云的应用平台上解决大数据问题的劳动力也越来越重要，这些应用平台对美国经济越来越重要。为了在大规模计算机系统研究的时代保持相关性，系统研究界向CloudLab和Chameleon Cloud等计算研究基础设施(CRI)的运营商施压，要求其支持更大规模的实验。但是，尽管商业计算云平台呈指数级增长，但研究基础设施的增长幅度不大。不幸的是，这种“规模”差距似乎可能会随着时间的推移而扩大，这可能会阻碍学术研究人员在大规模计算系统研究中直接创新。CloudJoin项目寻求通过创建具有弹性计算的混合CRI来开发一种长期、可持续的方法来缩小规模差距。该项目将开发云CRI和商业计算云集成的方法、工具和最佳实践。CloudJoin承诺允许实验以低成本扩大规模，同时保持实验的重复性，并为计算系统研究实验者保留一个智力上的“家”。该项目探索了三个云集成研究推动力。在第一个研究重点中，我们问“CRI运营商应该如何设计和实现其基础设施到公共云的连接？”第二个要点是：“实验者应该如何在CRI和公共云之间划分计算和存储资源？”最后一个重点是通过探索和拥抱专业质量的多云管理平台来解决扩展实验的复杂性，这些平台配备了监控、日志记录、警报、错误报告和配置文件功能。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Advances in large-scale data center computing and networking has accelerated innovation across computer science during the past decade. Large-scale, data-intensive systems experimentation is increasingly important to accelerating science, as is developing a workforce capable of solving big data problems on the large cloud-based application platforms that are increasingly central to the US economy. To stay relevant in an age of at-scale computer systems research, the systems research community has pressed operators of Computing Research Infrastructures (CRI) such as CloudLab and Chameleon Cloud with growing demands to support larger size experiments. But while commercial compute cloud platforms have grown exponentially, research infrastructures have grown modestly. Unfortunately, this `scale' gap seems likely to grow over time, threatening to inhibit academic researchers from directly innovating in large-scale computing systems research.The Cloudjoin project seeks to develop a long-term, sustainable approach to shrink the scaling gap by creating hybrid CRIs with elastic computing. The project will develop methods, tools, and best practices for integration of cloud CRI and commercial compute clouds. Cloudjoin promises to permit experiments to scale at low cost while maintaining experiment reproducibility and preserving an intellectual `home' for computing systems research experimenters. The project explores three cloud integration research thrusts. In the first research thrust we ask "How should a CRI operator architect and implement her infrastructure's connectivity to a public cloud?" The second thrust asks: "How should an experimenter partition compute and storage resources between CRI and the public cloud?" The final thrust addresses the complexities of scaling experiments by exploring and embracing professional quality, multi-cloud management platforms equipped with monitoring, logging, alerting, error reporting and profiling capabilities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.