AitF: Collaborative Research: Fast, Accurate, and Practical: Adaptive Sublinear Algorithms for Scalable Visualization

随着人类奋进的各个领域产生大量数据，数据探索-分析，理解和从数据中提取价值-已经变得绝对重要。数据可视化是迄今为止最常见的数据探索机制，新手和专家数据分析师都使用它。然而，在越来越大的数据集上进行数据可视化仍然很困难：即使是大型数据集的简单可视化也可能很慢且不具有交互性，而数据集的采样部分的可视化可能会误导分析师。该项目旨在开发FastViz，这是一个可扩展的可视化引擎，它不仅可以在同一时间内实现更大数量级的数据集的可视化，而且还可以确保所产生的可视化满足最终用户正确分析所必需的关键属性。为了确保立即使用，FastViz将应用于三个现实世界的应用领域：电池科学，广告分析和基因组数据分析，并在由PI开发的开源视觉探索平台Zenvision中实现。 该项目的学生在结合算法和系统考虑因素，使数据探索方面获得宝贵的经验。FastViz的开发是由系统考虑因素的同时调查驱动的，例如能够实现各种形式的在线采样的索引和存储技术，以及用于（a）可视化生成的算法考虑因素，其中目标是产生增量改进的可视化，其中首先显示重要特征，以及（B）可视化选择，其中目标是选择，从尚未生成的可视化的集合中选择满足期望标准的可视化。在系统方面，FastViz将利用并促进在线采样系统的最新发展，从而能够使用更强大的采样模式。 在算法方面，FastViz将从测试、分布学习和次线性算法文献中汲取想法，据PI所知，这些想法尚未在实践中得到应用。 开发的算法将遵守最优性保证，并在可能的情况下，实例最优性保证，确保它们将以最有效的方式适应数据特征。 该项目将导致更好地了解采样算法开发和系统设计之间的相互作用，一方面促进采用更现实的模型和算法，并开发更强大的采样引擎，使算法中所需的模型。

With the wealth of data being generated in every sphere of human endeavor, data exploration--analyzing, understanding, and extracting value from data--has become absolutely vital. Data visualization is by far the most common data exploration mechanism, used by novice and expert data analysts alike. Yet data visualization on increasingly larger datasets remains difficult: even simple visualizations of a large dataset can be slow and non-interactive, while visualizations of a sampled fraction of a dataset can mislead an analyst. The project aims to develop FastViz, a scalable visualization engine, that will not only enable visualization on datasets that are orders of magnitude larger in the same time, but also ensure the resulting visualizations satisfy key properties essential for correct analysis by end-users. To ensure immediate utilization, FastViz will be applied to three real-world application domains: battery science, advertising analysis, and genomic data analysis, and implemented in Zenvisage, an open-source visual exploration platform developed by the PIs. Students in the project gain invaluable experience in combining the algorithmic and systems considerations that enable data exploration. FastViz's development is driven by simultaneous investigation of systems considerations, such as indexing and storage techniques that enable various forms of online sampling, and algorithmic considerations for (a) visualization generation, where the goal is to produce incrementally improving visualizations in which the important features are displayed first, and (b) visualization selection, where the goal is to select, from a collection of as yet not generated visualizations, those that that satisfy desired criteria. On the systems front, FastViz will leverage and contribute back to recent developments on online sampling systems that enable the use of more powerful sampling modalities. On the algorithms front, FastViz will draw ideas from testing, distribution learning, and sublinear algorithms literature that, to the best knowledge of the PIs, have not been adapted in practice. The algorithms developed will obey optimality guarantees, and wherever possible, instance-optimality guarantees, ensuring that they will adapt to data characteristics in the most efficient way possible. The project will lead to a better understanding of the interplay between sampling algorithms development and systems design, facilitating the adoption of more realistic models and algorithms on the one hand, and the development of more powerful sampling engines that enable the models required within the algorithms.