III: Medium: 20/20: A System for Human-in-the-Loop Data Exploration

探索性数据分析在科学、工程和商业等广泛领域的数据驱动发现中发挥着关键作用。为了使数据分析在用户群不断扩大和多样化的时期成为商品，人类生产力和易用性必须成为任何数据库系统的首要设计考虑因素。不幸的是，用户友好的、旨在提高人类生产力的数据工具仍然非常缺乏。该项目将使不同技能水平的用户能够比现在更容易、更快地与大型数据集进行交互和探索。与其花费大量宝贵的时间来构建复杂的分析任务，这项工作将提供一个更敏捷，响应速度更快和用户友好的系统，该系统基于对视觉表示的直接操作（例如，图表、曲线图、地图）的数据集和分析结果。该系统还可以用作学习工具：例如，教师可以引导学生通过复杂的数据集来验证特定的假设。该项目将使更多用户更容易获得大规模数据探索。总体而言，它将加速电子商务、金融和科学等许多领域的发现和突破。这项研究将纳入本科和研究生课程。该项目的推广活动包括面向大学生和高中女生的特别研究和教育项目。该项目将建立一个新的数据库系统，旨在进行人机交互（HIL）操作。这项工作的目标是不断增长的以数据为中心的应用程序，在这些应用程序中，用户通常使用复杂的分析和机器学习技术直接操纵、分析和探索大型数据集。传统的数据库技术不适合于此目的。历史上，数据库假定（1）基于文本的输入（例如，SQL）和输出，（2）一个点（即，无状态）查询-响应范例，（3）批处理结果，以及（4）简单分析。项目团队将放弃这些基本假设，并构建一个支持可视化输入和输出、“对话式”交互、早期和渐进式结果以及复杂分析的系统。构建一个集成了这些功能的系统需要对整个数据堆栈进行彻底的重新思考，从可视化界面到“核心”，以及整合相关的算法。主要的研究挑战围绕着开发算法和优化，这些算法和优化利用HIL工作负载的独特特性来加快对大型数据集的分析。该团队将构建一个名为20/20的概念验证HIL数据库，该数据库将紧密集成并显著扩展Brown构建的两项现有技术：PanoramicData是一个触摸和笔数据可视化系统，将作为前端。第二个构建块是Tupleware主存分析系统，它将复杂的分析管道编译成可执行文件。Tupleware将作为后端分析组件。该团队预计，最终结果将为常见的分析工作负载提供比最先进的解决方案更快的速度（50%或更多）。项目网站（http：//database.cs.brown.edu/projects/20-20/）将包括关于项目、出版物、公共数据集和代码的信息。

Explorative data analysis plays a key role in data-driven discovery in a wide range of domains including science, engineering and business. In order for data analysis to become a commodity during a period when their user base is continually expanding and diversifying, human productivity and ease-of-use must become first-class design considerations for any database system. Unfortunately, data tools that are user friendly and designed to improve human productivity are still sorely lacking. This project will enable users at different skill levels to interact with and explore their large datasets far easier and faster than they do today. Rather than spending a lot of precious time to build complex analytics tasks, this work will offer a more agile, responsive and user-friendly system based on direct manipulation of the visual representations (e.g., charts, graphs, maps) of the data sets and analysis results. The system can also be used as a learning tool: e.g., a teacher could walk students through a complex dataset to verify specific hypothesis. This project will make large-scale data exploration more accessible to more users. Overall, it will accelerate discovery and breakthroughs in many domains such as e-commerce, finance and science. This research will be incorporated in undergraduate and graduate coursework. The outreach activities include special research and education­focused programs that are geared towards undergraduates and high school girls.This project will build a new class of database systems designed for Human-In-the-Loop (HIL) operation. The work targets an ever-growing set of data-centric applications in which users directly manipulate, analyze and explore large data sets, often using complex analytics and machine learning techniques. Traditional database technologies are ill suited to serve this purpose. Historically, databases assumed (1) text-based input (e.g., SQL) and output, (2) a point (i.e., stateless) query-response paradigm, (3) batch results, and (4) simple analytics. The project team will drop these fundamental assumptions and build a system that instead supports visual input and output, "conversational" interaction, early and progressive results, and complex analytics. Building a system that integrates these features requires a complete rethinking of the full data stack, from the visual interface to the "core", as well as incorporating pertinent algorithms. The primary research challenges revolve around developing algorithms and optimizations that leverage the unique characteristics of HIL workloads to speed up analysis over large data collections. The team will build a proof-of-concept HIL database called 20/20 that will tightly integrate and significantly extend two existing technologies built at Brown: PanoramicData is a touch and pen data visualization system and will serve as the front-end. The second building block is the Tupleware main-memory analytics system, which compiles complex analytics pipelines into executables. Tupleware will serve as the back­end analytics component. The team expects that the end result will offer a substantial speed­up (50% or more) over the state-of-the-art solutions for common analytics workloads. The project web site (http://database.cs.brown.edu/projects/20-20/) will include information on the project, publications, public datasets and code.