CRII: CNS: Toward a Scalable Geo-distributed Data Analytics System

该奖项的全部或部分资金来自2021年美国救援计划法案(公法117-2)。许多自动化人们日常活动的应用程序依赖于云来服务全球用户。例如，对延迟敏感的服务，如Netflix、Uber和Airbnb，利用亚马逊地理分布的数据中心中的云资源，而不是管理自己的数据中心。这些云应用提供商必须分析高度分散的大规模数据，包括用户的活动日志和系统日志，即地理分布式分析(GDA)。GDA任务会影响用户体验、系统运行状况和运营成本；必须以经济高效的方式及时完成这些任务。然而，由于异类云资源的复杂性以及云环境和应用程序中不可预测的变化，实现这一目标具有挑战性。该项目旨在设计和实现一种新型的GDA系统，该系统能够确定和管理最优的云资源配置，并考虑到广域网(WAN)和多样化的计算资源。此外，系统将考虑可用的云资源、输入数据位置、应用程序目标和查询执行动态，做出任务调度决策，以及时且经济高效地处理GDA查询。因此，GDA应用程序将能够以最小的努力实现性价比目标，同时处理和克服上述挑战。该项目的成功完成将促进对(1)如何使用历史数据准确预测GDA工作负载，以及(2)如何在GDA中优化分配和高效利用各种不同的云资源的理解。设计和原型实施将以开放源码系统的形式公开提供。这项研究的结果将使几个科学、商业和社会领域的研究人员和软件开发人员能够轻松地运行和评估各种GDA算法。该项目的研究成果也将纳入本科生和研究生的大学课程。该项目还将提供多种机会，扩大STEM教育的参与度和多样性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Many applications that automate people's daily activities rely on the cloud to serve users around the globe. For example, latency-sensitive services, such as Netflix, Uber, and Airbnb, utilize cloud resources in Amazon’s geo-distributed data centers instead of managing their own data centers. These cloud application providers must analyze highly diffuse, large-scale data, including users’ activities logs and system logs — i.e., geo-distributed analytics (GDA). GDA tasks affect user experiences, system health, and operating cost; they must be completed in a timely and cost-efficient manner. However, achieving this is challenging due to the complexities of heterogeneous cloud resources and unpredictable changes in the cloud environments and applications. This project aims to design and implement a novel GDA system that determines and manages the optimal cloud resource configurations, with consideration of the wide-area network (WAN) and diverse compute resources. In addition, the system will make task scheduling decisions to process GDA queries in a timely and cost-efficient manner considering available cloud resources, input data locations, application goals, and query execution dynamics. Thus, GDA applications will be able to achieve cost-performance goals with minimal effort, while addressing and overcoming the challenges above. The successful completion of this project will advance the understanding of (1) how GDA workloads can be accurately predicted using historical data, and (2) how diverse and heterogeneous cloud resources can be optimally allocated and efficiently utilized in GDA. A design and prototype implementation will be publicly available in the form of an open-source system. The outcomes of this research will allow researchers and software developers across several scientific, commercial, and social domains to easily run and evaluate diverse GDA algorithms. The research outcomes of this project will also be integrated into the university course curriculum for both undergraduate- and graduate-level students. This project will also open diverse opportunities that broaden participation and diversity in STEM education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.