CRII: CNS: Toward a Scalable Geo-distributed Data Analytics System

点击翻译按钮获取中文摘要

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Many applications that automate people's daily activities rely on the cloud to serve users around the globe. For example, latency-sensitive services, such as Netflix, Uber, and Airbnb, utilize cloud resources in Amazon’s geo-distributed data centers instead of managing their own data centers. These cloud application providers must analyze highly diffuse, large-scale data, including users’ activities logs and system logs — i.e., geo-distributed analytics (GDA). GDA tasks affect user experiences, system health, and operating cost; they must be completed in a timely and cost-efficient manner. However, achieving this is challenging due to the complexities of heterogeneous cloud resources and unpredictable changes in the cloud environments and applications. This project aims to design and implement a novel GDA system that determines and manages the optimal cloud resource configurations, with consideration of the wide-area network (WAN) and diverse compute resources. In addition, the system will make task scheduling decisions to process GDA queries in a timely and cost-efficient manner considering available cloud resources, input data locations, application goals, and query execution dynamics. Thus, GDA applications will be able to achieve cost-performance goals with minimal effort, while addressing and overcoming the challenges above. The successful completion of this project will advance the understanding of (1) how GDA workloads can be accurately predicted using historical data, and (2) how diverse and heterogeneous cloud resources can be optimally allocated and efficiently utilized in GDA. A design and prototype implementation will be publicly available in the form of an open-source system. The outcomes of this research will allow researchers and software developers across several scientific, commercial, and social domains to easily run and evaluate diverse GDA algorithms. The research outcomes of this project will also be integrated into the university course curriculum for both undergraduate- and graduate-level students. This project will also open diverse opportunities that broaden participation and diversity in STEM education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.