CAREER: Intermittent Query Processing

促进数据存储和分析的数据库系统是现代数据密集型应用的支柱，从机器学习到物联网，并且代表了一个重要的市场力量，估计全球市场价值超过500亿美元。许多新的数据库应用程序需要对不断变化的数据进行快速复杂的分析。然而，硬件性能增长的放缓，加上数据和数据密集型应用程序的持续增加，造成了一场危机，因为现有的数据库系统不太适合那些希望权衡性能和资源效率的应用程序。为了解决这种对新数据的查询与受控资源利用率之间的差距，需要一种新的数据库系统范式。该奖项研究了一种新的查询执行范式，间歇查询处理，以回答这个问题：数据库系统如何在最小化资源利用率的同时，为突发数据到达或周期性查询的持续查询提供低延迟？该框架允许查询最初作为批处理执行，随后进行间歇性处理以更新结果，同时通过选择性地持久化来自初始查询执行的工件以及通过利用关于预期数据到达的知识和查询如何适合增量计算的模型来最小化更新查询结果所需的资源。该项目将研究预测查询和数据到达的方法，查询部分重叠的工作负载的调度和放置决策，状态共享的机会，资源分配和新的查询优化技术。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Database systems that facilitate the storage and analysis of data are the backbone of modern data-intensive applications, from machine learning to the Internet of Things, and represent a significant market force with an estimated worldwide market value of over 50 billion USD. Many new database applications demand fast complex analytics over changing data. However, the slowdown in the growth of hardware performance, coupled with the continued increase of data and data-intensive applications creates a crisis, as existing database systems are not well suited for applications that want to trade-off performance and resource efficiency. To address this gap of queries over fresh data with controlled resource utilization, a new database system paradigm is needed.This award investigates a new query execution paradigm, Intermittent Query Processing, to answer the question: how should database systems provide low latency for ongoing queries with bursty data arrival or periodic queries while minimizing resource utilization? This framework allows a query to initially execute as a batch, with subsequent intermittent processing to update the result while minimizing the resources required to update the query result by selectively persisting artifacts from the initial query execution and by exploiting knowledge about expected data arrivals and models for how amenable a query is for incremental computation. This project will investigate methods for predicting query and data arrivals, scheduling and placement decisions for workloads where queries partially overlap, opportunities for state sharing, resource allocation, and new query optimization techniques.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.