BIGDATA: F: Bringing Interactive Data Management to Scientists, Analysts, and the Masses: A Holistic Unification of Spreadsheets and Databases

随着数据集在我们日常生活的许多领域（包括科学、金融和商业）的激增，管理表格数据（即由一组表或关系表示的数据）变得越来越重要。表格数据管理的最新技术跨越了两种截然不同的范式——电子表格和数据库——它们有着截然不同的优势和劣势；两者都取得了巨大的成功，但都面临着根本性的局限。迄今为止，大型数据的交互式临时管理仍然是繁琐和困难的。该项目旨在全面统一数据库和电子表格技术，从而为科学家、分析师和普通用户带来电子表格的易用性和交互性，以及数据库的可扩展性、表现力和协作能力。该项目开发了DataSpread，这是一个将数据库与电子表格整体统一的系统——将数据库的可扩展性、表现力和一致性的传统优势与数十亿终端用户非常熟悉的电子表格的交互性结合起来，并在此过程中重新思考面向用户的交互，而不是电子表格，该项目将通过广泛的用户研究开发和评估。研究小组研究了新模型、算法、结构和交互原语的发展，以实现统一。具体来说，本研究开发了新的方法，用于紧凑地表示电子表格数据和计算电子表格上的查询，位置感知索引结构，以及有效地将更新传播到用户视图的机制。该项目还研究了新的交互原语的设计，以取代成熟的SQL，这更有利于电子表格界面，并支持以用户为中心的事务。虽然旨在开发通用技术来推进数据管理的最新技术，但该项目将由合作者的现实挑战驱动——用例、痛点、实际部署、联合开源开发以及关键的评估和反馈。

With the proliferation of datasets in many spheres of our everyday life, including science, finance, and commerce, managing tabular data, i.e., data represented by a set of tables or relations, has become increasingly essential. The state of the art in tabular data management spans two distinct paradigms with a sharp divide--spreadsheets and databases--with drastically different strengths and weaknesses; both enjoying tremendous successes but facing fundamental limitations. To date, interactive ad-hoc management of large data remains cumbersome and difficult. This project aims to holistically unify database and spreadsheet technology, and thus bring to scientists, analysts, and lay users, the ease of use and interactivity of spreadsheets with the scalability, expressiveness, and collaboration capabilities of databases.The project develops DataSpread, a system to holistically unify databases with spreadsheets--to marry the traditional strengths of scalability, expressiveness, and consistency of databases with the interactivity of spreadsheets that billions of end-users are so familiar with, and in the process rethink user-facing interactions beyond spreadsheets which the project will develop and evaluate via extensive user studies. The research team studies the development of new models, algorithms, structures, and interaction primitives to make the unification practical. Specifically, this research develops new methods for compactly representing spreadsheet data and computing over queries on a spreadsheet, positionally aware indexing structures, and mechanisms for efficiently propagating updates to the user viewport. The project also studies the design of new interaction primitives to replace full-fledged SQL, which is more conducive to a spreadsheet interface, and in support for user-centric transactions. While aiming at developing general techniques to advance the state of the art in data management, the project will be driven by the collaborators' real-world challenges--for use cases, pain points, practical deployment, joint open-source development, and critical evaluation and feedback.