AitF: Collaborative Research: Fast, Accurate, and Practical: Adaptive Sublinear Algorithms for Scalable Visualization

随着人类工作的每个领域都产生了丰富的数据，数据探索--分析、理解并从数据中提取价值--变得绝对至关重要。数据可视化是迄今为止最常见的数据探索机制，无论是新手还是专业数据分析师都在使用。然而，在越来越大的数据集上进行数据可视化仍然很困难：即使是对大型数据集的简单可视化也可能是缓慢和非交互的，而对数据集的采样部分的可视化可能会误导分析师。该项目的目标是开发FastViz，这是一个可扩展的可视化引擎，不仅可以在同一时间对大几个数量级的数据集进行可视化，而且还可以确保所产生的可视化满足最终用户正确分析所必需的关键属性。为了确保立竿见影的使用，FastViz将被应用于三个现实世界的应用领域：电池科学、广告分析和基因组数据分析，并在PIS开发的开源可视化探索平台ZProfacage中实施。该项目的学生在结合算法和系统考虑因素以实现数据探索方面获得了宝贵的经验。FastViz的开发是由对系统考虑因素的同时调查推动的，例如能够实现各种形式的在线采样的索引和存储技术，以及(A)可视化生成的算法考虑，其中目标是产生递增改进的可视化，其中重要特征首先被显示，以及(B)可视化选择，其中的目标是从尚未生成的可视化集合中选择那些满足预期标准的可视化。在系统方面，FastViz将利用并回馈在线采样系统的最新发展，使之能够使用更强大的采样模式。在算法方面，FastViz将从测试、分发学习和次线性算法文献中吸取想法，就PI的最佳知识而言，这些文献尚未在实践中得到改编。开发的算法将遵守最优性保证，并在任何可能的情况下，实例最优性保证，确保它们将以最有效的方式适应数据特征。该项目将导致更好地了解采样算法开发和系统设计之间的相互作用，一方面促进采用更现实的模型和算法，并开发更强大的采样引擎，使算法中所需的模型能够实现。

With the wealth of data being generated in every sphere of human endeavor, data exploration--analyzing, understanding, and extracting value from data--has become absolutely vital. Data visualization is by far the most common data exploration mechanism, used by novice and expert data analysts alike. Yet data visualization on increasingly larger datasets remains difficult: even simple visualizations of a large dataset can be slow and non-interactive, while visualizations of a sampled fraction of a dataset can mislead an analyst. The project aims to develop FastViz, a scalable visualization engine, that will not only enable visualization on datasets that are orders of magnitude larger in the same time, but also ensure the resulting visualizations satisfy key properties essential for correct analysis by end-users. To ensure immediate utilization, FastViz will be applied to three real-world application domains: battery science, advertising analysis, and genomic data analysis, and implemented in Zenvisage, an open-source visual exploration platform developed by the PIs. Students in the project gain invaluable experience in combining the algorithmic and systems considerations that enable data exploration. FastViz's development is driven by simultaneous investigation of systems considerations, such as indexing and storage techniques that enable various forms of online sampling, and algorithmic considerations for (a) visualization generation, where the goal is to produce incrementally improving visualizations in which the important features are displayed first, and (b) visualization selection, where the goal is to select, from a collection of as yet not generated visualizations, those that satisfy desired criteria. On the systems front, FastViz will leverage and contribute back to recent developments on online sampling systems that enable the use of more powerful sampling modalities. On the algorithms front, FastViz will draw ideas from testing, distribution learning, and sublinear algorithms literature that, to the best knowledge of the PIs, have not been adapted in practice. The algorithms developed will obey optimality guarantees, and wherever possible, instance-optimality guarantees, ensuring that they will adapt to data characteristics in the most efficient way possible. The project will lead to a better understanding of the interplay between sampling algorithms development and systems design, facilitating the adoption of more realistic models and algorithms on the one hand, and the development of more powerful sampling engines that enable the models required within the algorithms.