CAREER: FIREFLY - Rich Explanations for Database Queries

随着最近大数据的流行，包括数据分析师、科学家、决策者和普通互联网用户在内的一系列人越来越多地寻求对可用数据集中的趋势和异常的高层次解释。这样的用户通常在数据集上运行查询，计算聚合，在图表上绘制答案，并为她所观察到的寻找解释。例如，她可能会问：“为什么两个图相似或不同？”，“为什么一个点序列增加或减少？”，“为什么一个图中有一个突然的高峰或低谷？”等等。现有的数据分析系统侧重于大规模统计分析、多维数据聚合、交互式数据探索和复杂的可视化支持。然而，目前还没有工具可以为用户提供语义解释。这个项目开发了一个名为FIREFLY（正式互动丰富的即时解释）的工具包，它为用户提出的“为什么”问题提供了快速、丰富、深刻的解释。该工具提供的自动解释将帮助用户更有效地利用大数据，项目的研究成果将丰富大数据分析技术。此外，与该项目相结合的课程和研究经验将为不同层次的学生提供帮助，帮助他们成为未来的研究人员。将特别注意支持这一进程中的多样性。本项目介绍了一个新的视角，在数据分析原则的概念，因果关系，反事实和干预。FIREFLY旨在寻找输入元组上的属性概要作为解释，这样通过将数据库限制为包含这些概要不同值的元组，查询的答案和用户的观察结果会发生变化，从而解释观察结果。为了有效地返回有意义的概要作为解释，本项目将沿着三个主要研究方向发展理论，算法和优化：(1)将建立一个丰富的框架来支持有意义的解释、大类数据库查询和用户提出的各种问题；(2)将建立一个带有图形用户界面的交互工具，以帮助用户运行查询、提问和探索工具返回的解释；(3)将开发新技术来处理输入数据和解释本身的不确定性。

With the recent popularity of Big Data, a range of people including data analysts, scientists, decision makers, and ordinary Internet users are increasingly seeking high level explanations for trends and anomalies in available datasets. Such a user typically runs queries on the datasets, computes aggregates, plots the answers on a graph, and looks for explanations for what she observes. For example, she may ask: "Why are two graphs similar or different?", "Why is a sequence of points increasing or decreasing?", "Why is there a sudden spike or dip in a graph?", and so on. Existing data analysis systems focus on large-scale statistical analytics, multi-dimensional data aggregation, interactive data exploration, and sophisticated visualization support. However, there are no tools currently available that offer semantic explanations to users. This project develops a toolkit named FIREFLY (Formal Interactive Rich Explanations On-The-Fly) that provides fast, rich, insightful explanations in response to such 'why' questions asked by users. The automatic explanations provided by this tool will help users harness Big Data more effectively, and the research findings of the project will enrich Big Data analytics techniques. Furthermore, the courses developed in conjunction with this project and the research experience that it will provide students at various levels will help train them to be future researchers. Special attention will be paid to supporting diversity in this process. This project introduces a new perspective in data analysis principled upon the notions of causality, counterfactuals, and interventions. FIREFLY aims to find synopses of properties on input tuples as explanations, such that by restricting the database to tuples that entail a different value of these synopses, the answer to the query and the observation of the user changes, thereby explaining the observation. In order to efficiently return meaningful synopses as explanations, this project will develop theory, algorithms, and optimizations along three main research directions: (1) a rich framework will be established to support meaningful explanations, large classes of database queries, and a variety of questions asked by the users, (2) an interactive tool with a graphical user interface will be built to help users run queries, ask questions, and explore the explanations returned by the tool, and (3) new techniques will be developed to handle uncertainty in the input data and in the explanations themselves.